Skip to content

Harden YDB Trino connector for production readiness - #246

Open
KirillKurdyukov wants to merge 10 commits into
mainfrom
codex/ydb-trino-integration-clean
Open

Harden YDB Trino connector for production readiness#246
KirillKurdyukov wants to merge 10 commits into
mainfrom
codex/ydb-trino-integration-clean

Conversation

@KirillKurdyukov

Copy link
Copy Markdown
Collaborator

Summary

  • define one-catalog/one-database default namespace semantics with safe nested YDB paths and verified cross-catalog federation
  • harden UPDATE/DELETE/MERGE key, retry, transaction, commit-ambiguity, batching, and retained-memory contracts
  • add YDB-native transactional INSERT staging and Date32/Datetime64/Timestamp64 support using forceSignedDatetimes=true
  • audit Trino 479 capabilities and document concrete unsupported boundaries instead of advertising incomplete behavior

The final review also aligned Timestamp64 predicate pushdown with the pinned JDBC 2.3.18 upper boundary and made MERGE buffer-limit failures release retained pages and terminate the sink before any remote mutation.

Validation

  • JAVA_HOME=$(/usr/libexec/java_home -v 25) DOCKER_HOST=unix://${HOME}/.colima/default/docker.sock TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock mvn --batch-mode --update-snapshots -f ydb-trino-adapter/pom.xml clean test
    • 364 total, 280 passed, 0 failures, 0 errors, 84 skipped
    • Connector: 291 total / 80 skipped
    • Smoke: 36 total / 4 skipped
    • Federation: 6 total / 0 skipped
  • JAVA_HOME=$(/usr/libexec/java_home -v 25) mvn --batch-mode -f ydb-trino-adapter/pom.xml -DskipTests package
    • plugin JAR, service descriptor, YdbPlugin, and YdbConfig verified

Compatibility and limitations

  • Trino 479, JDK 25, YDB JDBC 2.3.18 (SDK 2.3.20)
  • one catalog per configured YDB database; default is the only virtual schema
  • no TRUNCATE, dynamic database switching, cross-catalog snapshot/distributed commit, or replay after an ambiguous MERGE commit
  • DEFAULT capability remains false because the inherited Trino contract includes unsupported DEFAULT NULL, SET/DROP DEFAULT, and MERGE-default cases
  • the YDB test-helper image is intentionally unpinned; this local run is not presented as an image-version compatibility baseline

Related: trinodb/trino#28110

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant