SK-3020 Flowvault release/26.8.1 - #398
Open
skyflow-bharti wants to merge 50 commits into
Open
Conversation
* Backport pr-flowvault.yml CI workflow from flowvault-release/test-artifactory-path Verified via a dummy test PR (#381) against that branch - Build and Unit Tests jobs both passed, scoped correctly to flowvault/common/pom.xml path changes. This workflow only existed on the test-artifactory-path branch until now; it was never on main or any other release branch. * SK-2967 backport internal + public release workflow fixes Internal releases (both modules) Version now comes from the module's own pom plus the commit SHA, via scripts/current_module_version.sh, instead of get-previous-tag. Tag lookup is repo-wide with no module awareness, so it stamped an unrelated v3 tag onto both flowvault and v2 builds (3.0.0-beta.13-dev.<sha>). A module's own pom is inherently per-module. Verified: flowvault 1.0.0-dev.<sha>, v2 2.1.1-dev.<sha>. internal-release.yml no longer fires on tag pushes. tags-ignore was '*.*', and Actions' glob '*' does not match '/', so slash tags like flowvault/v1.0.0 were never ignored - a public release tag also started an internal release, resolving to the wrong module. Now '**'. Public/beta releases Triggered by publishing a GitHub Release rather than a raw tag push, so the branch comes from target_commitish - the branch chosen in the release UI. It was previously guessed by matching a branch tip to the tagged commit, which failed once anything landed after the tag (re-runs never worked) and silently pushed to an unrelated branch when several shared a tip. Tags are <module>/v<semver>[-beta.N]; the prefix selects the module and is stripped, so Maven Central sees a plain semver. Malformed tags are rejected instead of publishing something unintended. beta-release.yml merged in: 'release' events cannot be filtered by tag pattern, so both files would fire on every release, and beta/public were already handled identically in every conditional. Kind is derived from the tag. Empty-commit guard: when the pom already holds the release version - the normal case when it is set before tagging - a bare git commit exits 1 and aborted the release. Now treated as success. dry-run input added so this path can be exercised without publishing; Maven Central is immutable. Verified with a real published Release (since deleted): correct module/version/branch, 383 tests, GPG signed, zero uploads. Merged with the base rather than overwriting it: the javadoc-skip added on flowvault-release/26.8.1 is kept for internal builds, but NOT applied to beta/public - Sonatype requires the -javadoc.jar and rejects bundles without one. That matches the intent stated in the base's own comment, which its code contradicted. * SK-2967 stage Maven Central releases instead of auto-publishing Sets autoPublish=false for both flowvault and v2. Deployments now upload and validate, then wait in the Central Portal as VALIDATED until someone clicks Publish - or Drop, to discard them. Maven Central is immutable: a published version can never be edited or removed, and the only remedy for a bad release is shipping another one. With autoPublish=true there was no point at which anything could be checked before that became permanent. Sonatype's validation only checks that required fields are present, not that they are correct. flowvault/pom.xml currently carries <description>Skyflow V3 SDK for the Java programming language</description> on an artifact named skyflow-flowvault-java - copy-paste from the v3 module. That passes validation and would have shipped permanently on the first release of a brand-new coordinate. Exactly what the staging step is for. Note this changes v2's behaviour too: its releases previously went live automatically once mvn deploy succeeded, and now need the extra click. * SK-2967 follow the v2 -> skyvault rename release.yml mapped the skyvault tag prefix to MODULE="v2", a directory that no longer exists - 'mvn -pl v2' would have failed outright. Both prefixes now map straight to their directory name. internal-release.yml already resolved to skyvault: that came from the base and survived the merge, so only release.yml carried the stale mapping. Also refreshed comments that still said v2. Tag examples like skyvault/v2.1.2 are unchanged and correct - the v2 there is the version 2.1.2, not the module. * SK-2967 add skyvault-release/* trigger for internal releases Gives skyvault the same per-module release branch flowvault already had, so both read the same way: flowvault-release/* and skyvault-release/*. release/* is kept as a legacy alias, still mapping to skyvault. Module resolution is now an explicit case with no catch-all. The previous if/else defaulted anything that was not flowvault-release/* to skyvault, which is exactly how a flowvault tag once ended up publishing skyvault - unrecognised refs silently became the wrong module. An unmatched branch now fails loudly, and the resolved module is logged.
The internal release force-pushes its own version-bump commit, that push re-triggers the workflow, and it bumps and pushes again - roughly every 30 seconds. It ran ~60 times on this branch before being cancelled. The push uses PAT_ACTIONS, and PAT-authenticated pushes DO trigger workflows; GITHUB_TOKEN pushes deliberately do not. So the guard only becomes load-bearing once the PAT is wired up correctly - which is why the loop started the moment that was fixed, not before (while the secret resolved to empty, checkout failed and nothing could recurse). The guard existed on the v3 branch with a comment describing this exact failure mode, and was lost in the modularization.
* Split monolithic README into per-package docs for skyvault and flowvault
Ports the README restructure from devesh/v2-flowvault-readmes onto this
release branch, which already carries the v2 -> skyvault rename:
- root README.md becomes a short landing page pointing at each package
- skyvault/README.md holds the full skyflow-java documentation
- flowvault/README.md documents the skyflow-flowvault-java bulk surface
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Address README review: self-contained flowvault docs, per-record error model
Resolves review comments on the flowvault README:
- Quickstart declared `VaultController vault` twice. The second line also
called `skyflowClient.vault("<VAULT_ID>")`, which flowvault does not
expose — its `vault()` takes no argument and always resolves to the
first configured vault. Both fixed, and the single-vault behaviour is
now stated.
- Auth, Credentials, VaultConfig and error handling lived only in the
skyvault README, so flowvault users had to switch documents to get
started. flowvault now documents them inline: credential types and
their setters, credential resolution order, bearer token generation,
a full VaultConfig / Skyflow.builder() reference, timeout and retry
resolution with defaults, and logging. Only the exotic token variants
(context, scoped, signed) still point at skyvault.
- Added a note that flowvault is a new SDK versioned independently, so a
1.x flowvault next to a 2.x skyvault does not read as a downgrade. In
both the flowvault README and the root package table.
- Rewrote Error Handling around flowvault's actual model: request-level
SkyflowException versus per-record success/failure inside a call that
returns normally, including the trap that a call where every record
failed still does not throw. Documents index/httpCode/error/requestId
per record, the per-operation summaries, consumption loops for both
the one-level and two-level (tokenize) shapes, and the retry helpers.
Also brings the API surface in line with this branch, which renamed
several types after the README was first written: BulkInsertRequestRecord
(was BulkInsertRecord), tableName (was table), UpsertOptions (was
upsert/upsertType), TokenGroupRedactions (was BulkTokenGroupRedactions),
Bulk*Options (was InsertOptions/TokenizeOptions/...), CustomHeaderKey
.SkyflowAccountId/.RequestIdHeader, and the response shape, which is now
a single `records` list rather than separate `success` and `errors` lists.
Verified: all 20 Java snippets compile against the branch's classes, and
every sample JSON response matches the real toString() output.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Document bulk batch size and concurrency configuration
The flowvault README noted that bulk calls are split into batches sent
concurrently, but never documented how to configure that. Add a
"Batching and concurrency" section covering the per-operation env vars,
their defaults and maximums, and how effective values are resolved.
Values verified against Constants.java and the configure*ConcurrencyAndBatchSize
methods in VaultController: batch size is min(value, max) with a warning-and-
clamp above the max and a warning-and-default for invalid input; concurrency is
additionally capped at ceil(itemCount / batchSize), so it never exceeds the
number of batches to run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Cuts the incident narratives down to the constraint each one exists to protect, 86 comment lines to 36. Every non-comment line is byte-identical to before - verified by stripping comments and blanks from both sides. Also removes a duplicated comment above the fixtures step that had become wrong: it described a glob-based implementation while the step uses a hardcoded module list.
SK-3026 add tokens key in insert
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Start with a concise summary of the PR. The first three sections are required. The questions present in each section is there to help you guide you what to add. They are meant to be overwritten by your comments.
Why
Goal
Testing
Tech debt