samples: credential helper, pagination fixes, and new jobs/subscriptions samples - #1843
Open
jacalata wants to merge 16 commits into
Open
samples: credential helper, pagination fixes, and new jobs/subscriptions samples#1843jacalata wants to merge 16 commits into
jacalata wants to merge 16 commits into
Conversation
Introduces samples/_shared.py with resolve_credentials(args), which fills missing sign-in values from env vars (TABLEAU_SERVER, TABLEAU_TOKEN_NAME, etc.) or a .env-style file, and falls back to interactive getpass so secrets never touch shell history. CLI args still work for CI use. Wires the new helper into login.py, publish_workbook.py, and publish_datasource.py to establish the pattern; the remaining samples still accept the same CLI args and continue to work as before. Addresses #1551 item 1.
Several samples called `server.<endpoint>.get()` and named the result `all_workbooks`, `all_datasources`, etc. This only returns the first page (default 100 items); if the item of interest was not on that page it was silently missed and the sample failed with a "not found" message. Replace those calls with `TSC.Pager(server.<endpoint>)` so every page is walked. Where a total count was being displayed we still make one plain `.get()` up front so the total_available field is available without paging through the whole site twice. Also corrects an unrelated typo in getting_started/3_hello_universe.py where the "workbooks" section actually queried datasources. Addresses #1551 item 2 (and #1531).
The existing samples cover workbooks, datasources, schedules, extracts,
projects, users, groups, favorites, and webhooks, but there was no
sample for two frequently asked-about endpoints:
* list_jobs.py -- lists background jobs (extract refreshes, publishes,
flow runs, etc.), demonstrating the .filter() queryset with
date/status/type filters and the wait_for_job helper.
* manage_subscriptions.py -- list/create/delete site subscriptions,
demonstrating the SubscriptionItem + Target pattern and paginated
listing with TSC.Pager.
Both samples use the new samples/_shared.py credential resolver so the
sign-in pattern matches the rest of the samples.
Addresses #1551 item 3.
…#1853) - increase operations-per-run beyond the default 30 - add permission for action to write its cache Taken together, these should let the action run on the oldest issues and PRs we have.
…) (#1852) The REST Query Job response schema documents a structured status-notes block: <statusNotes> <statusNote type="CountOfUsersAddedToSite" value="5" text="..." /> <statusNote type="CountOfUsersSkipped" value="1" text="..." /> </statusNotes> JobItem was parsing only the sibling legacy `<notes>` element (emitted by some job types like extractRefreshJob), missing the modern statusNotes entirely. For UserImport jobs and any other multi-row job where individual rows have distinct outcomes, `job.notes` came back as an empty list even when the server had sent detailed structured status. Add `JobItem.status_notes: list[dict]`, each dict with keys `type`, `value`, `text` (any of which may be None if the server omitted them). The legacy `notes: list[str]` attribute is unchanged for backwards compatibility -- it still parses the `<notes>` element still emitted by extract-refresh and similar older job types. Verified against the public REST doc: https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api_ref_jobs_tasks_and_schedules.htm#query_job The existing job_get_by_id.xml test asset already contained a statusNotes block; the get_by_id test now asserts the structured value in addition to the legacy notes list. Two new tests cover the absent case (yields []) and the multi-note case with attribute omissions. Discovered while planning tabcmd createsiteusers nowait / silent-progress work (tableau/tabcmd#35); a live probe against Tableau Server 2025.1 confirmed the server emits this schema for UserImport jobs. Fixes #1850.
* fix: normalize CRLF line endings to LF in six Python files Several files had CRLF line endings baked in from a prior black-version bump commit, causing ^M noise in diffs. Also reformats the multi-line .format() calls in data_alert_item.py and subscription_item.py to match current black formatting. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * Run type and style checks on Python 3.13 now * Black format, run via Python 3.13 * chore: add .gitattributes to enforce LF line endings for Python files Prevents a repeat of the CRLF regression fixed in the prior commit by normalizing line endings at commit time. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Restore -t for --site, -u for --username, -p for --password; drop short flags on --token-name and --token-value. This matches tabcmd's canonical short flags in tabcmd/execution/parent_parser.py so users running both tools have one convention to remember. The initial refactor picked new short flags without noticing that the old samples/login.py already followed tabcmd's convention (-p was --password, -t was --site). Reassigning -p to --token-name meant `python login.py -p <password>` silently sent the password as a token name. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Tableau REST API supports a `refreshExtractTriggered="true"` attribute on subscription payloads that makes the subscription fire when its referenced schedule's extract refresh completes, rather than on the schedule's time trigger. On Tableau Cloud, this is the wire form of an "On Extract Refresh" subscription. TSC never exposed this attribute; users trying to create these subscriptions were passing `schedule_id=None` and hitting a confusing wire error deep in the endpoint layer. Changes: - `SubscriptionItem.on_extract_refresh(...)` classmethod factory constructs a subscription with an extract-refresh schedule id and the flag set. - `refresh_extract_triggered` exposed as a property with a docstring covering the two ways the server surprises callers (server rejects True with a non-extract schedule; server silently clears the flag when a schedule change is included in an update). - `Subscriptions.create()` and `.update()` now raise `ValueError` up front when `schedule_id` is missing, so the wire error becomes an actionable client-side message. - `create_req` emits `refreshExtractTriggered="true"` only when set; `update_req` emits both true and false so callers can turn the flag off on an existing subscription. - `_parse_element` reads the attribute back into the property; parse continues to accept inline-schedule responses (schedule_id=None). Tests cover: factory sets flag + schedule id; default false; create_req emit-when-set/omit-when-false; update_req always emits; parse round-trip for both true and missing; parse of inline-schedule responses; create() and update() reject missing schedule_id. Related to #1658. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Updates the samples/ scripts to demonstrate safer credential handling, correct pagination patterns, and add new examples for jobs and subscriptions, without changing the core tableauserverclient library.
Changes:
- Introduces
samples/_shared.pyhelpers for common CLI args plus credential resolution from CLI/env/.env/interactive prompt. - Fixes several samples that used
.get()(first page only) by switching toTSC.Pager(...)(or iterable QuerySet) to traverse full result sets. - Adds new sample scripts for listing background jobs and managing subscriptions.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| samples/_shared.py | New shared credential + argparse helpers used by multiple samples |
| samples/login.py | Uses shared helpers; updates login flow to avoid secrets on CLI |
| samples/publish_workbook.py | Uses shared helpers; fixes project discovery to page through projects |
| samples/publish_datasource.py | Uses shared helpers; removes ad-hoc env reading and debug overrides |
| samples/refresh_tasks.py | Pages through tasks via TSC.Pager instead of first-page .get() |
| samples/move_workbook_sites.py | Pages through sites via TSC.Pager instead of first-page .get() |
| samples/update_workbook_data_freshness_policy.py | Uses TSC.Pager to list all workbooks |
| samples/extracts.py | Uses TSC.Pager to list all workbooks |
| samples/explore_workbook.py | Uses TSC.Pager for projects/workbooks/custom views paging correctness |
| samples/explore_webhooks.py | Uses TSC.Pager to list all webhooks |
| samples/explore_favorites.py | Uses TSC.Pager to list all workbooks/datasources |
| samples/explore_datasource.py | Uses TSC.Pager for projects/datasources paging correctness |
| samples/getting_started/3_hello_universe.py | Fixes incorrect endpoint (workbooks vs datasources) |
| samples/list_jobs.py | New sample demonstrating jobs listing/filtering and wait-for-job |
| samples/manage_subscriptions.py | New sample demonstrating list/create/delete subscriptions with paging |
Suppressed comments (3)
samples/publish_workbook.py:34
add_common_arguments()already reserves-ufor--username, so reusing-uhere causes argparse to raise a conflicting option error and the script won’t start.
group = parser.add_mutually_exclusive_group(required=False)
group.add_argument("--thumbnails-user-id", "-u", help="User ID to use for thumbnails")
group.add_argument("--thumbnails-group-id", "-g", help="Group ID to use for thumbnails")
samples/_shared.py:133
- This claims a
.envfile next to the sample is loaded automatically, but the implementation only checks the current working directory. If users runpython samples/<script>.pyfrom the repo root,samples/.envwill be ignored.
# Load `.env` file if one is requested or available.
env_file = getattr(args, "env_file", None)
if env_file:
_load_env_file(Path(env_file))
else:
default_env = Path.cwd() / ".env"
if default_env.is_file():
_load_env_file(default_env)
samples/_shared.py:149
resolve_credentialswill callinput()/getpass.getpass()even when stdin is not a TTY, which can hang non-interactive runs despite the docstring saying prompts happen only when stdin is a terminal.
if not allow_prompt:
return
# Prompt for what's still missing. We only prompt for the pieces we
# actually need: server URL, and one of token or username/password.
if not getattr(args, "server", None):
args.server = input("Tableau server URL: ").strip()
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- Docstring on `refresh_extract_triggered` now warns about the manual- build update() footgun: because every subscriptions.update() payload carries the attribute, a caller who builds a fresh SubscriptionItem locally, stamps _id, and updates will silently flip an existing on-extract-refresh subscription off. Fetch first. - Soften create()'s "schedule_id is required" error so someone who just forgot to set schedule_id on a time-based subscription doesn't get steered exclusively toward SubscriptionItem.on_extract_refresh(...); the factory is now mentioned as a conditional pointer. - __init__'s schedule_id parameter is now typed str | None, matching the real state: _parse_element sets it to None on inline-schedule responses. Drop the two `# type: ignore` markers in test/test_subscription.py that were papering over the earlier lie. - create_req asserts schedule_id non-None to satisfy mypy after the parameter widening; subscriptions.create() already guards this path before request emission. - Add samples/create_extract_refresh_subscription.py demonstrating the full flow: sign in, resolve view/workbook and user by name, pick an extract-refresh schedule from the schedules list, build the subscription via on_extract_refresh(), post it. Highest-leverage discoverability artifact for callers searching "on extract refresh". - CHANGELOG entry.
Introduces samples/_shared.py with resolve_credentials(args), which fills missing sign-in values from env vars (TABLEAU_SERVER, TABLEAU_TOKEN_NAME, etc.) or a .env-style file, and falls back to interactive getpass so secrets never touch shell history. CLI args still work for CI use. Wires the new helper into login.py, publish_workbook.py, and publish_datasource.py to establish the pattern; the remaining samples still accept the same CLI args and continue to work as before. Addresses #1551 item 1.
Several samples called `server.<endpoint>.get()` and named the result `all_workbooks`, `all_datasources`, etc. This only returns the first page (default 100 items); if the item of interest was not on that page it was silently missed and the sample failed with a "not found" message. Replace those calls with `TSC.Pager(server.<endpoint>)` so every page is walked. Where a total count was being displayed we still make one plain `.get()` up front so the total_available field is available without paging through the whole site twice. Also corrects an unrelated typo in getting_started/3_hello_universe.py where the "workbooks" section actually queried datasources. Addresses #1551 item 2 (and #1531).
The existing samples cover workbooks, datasources, schedules, extracts,
projects, users, groups, favorites, and webhooks, but there was no
sample for two frequently asked-about endpoints:
* list_jobs.py -- lists background jobs (extract refreshes, publishes,
flow runs, etc.), demonstrating the .filter() queryset with
date/status/type filters and the wait_for_job helper.
* manage_subscriptions.py -- list/create/delete site subscriptions,
demonstrating the SubscriptionItem + Target pattern and paginated
listing with TSC.Pager.
Both samples use the new samples/_shared.py credential resolver so the
sign-in pattern matches the rest of the samples.
Addresses #1551 item 3.
Restore -t for --site, -u for --username, -p for --password; drop short flags on --token-name and --token-value. This matches tabcmd's canonical short flags in tabcmd/execution/parent_parser.py so users running both tools have one convention to remember. The initial refactor picked new short flags without noticing that the old samples/login.py already followed tabcmd's convention (-p was --password, -t was --site). Reassigning -p to --token-name meant `python login.py -p <password>` silently sent the password as a token name. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Round of fixes for the sample-scripts refactor after fresh-eyes review. Blocker: publish_workbook.py reused `-u` for --thumbnails-user-id while _shared.py add_common_arguments already binds `-u` to --username, so argparse raised ArgumentError on module load and the script would not start. Renamed to `-U`. Real bugs: - _shared.py .env search now checks cwd, samples/, and repo root (in that order) so the docstring stops lying about "next to the sample or cwd." - resolve_credentials now gates input()/getpass on sys.stdin.isatty() as the docstring already promised, so piped/CI invocations no longer hang forever. - manage_subscriptions.py --attach-image switched to argparse.BooleanOptionalAction so users can actually pass --no-attach-image; the previous store_true+default=True made the flag a permanent True. - Header docstring in _shared.py no longer claims "no existing command line breaks" (which was false: -p migrated from --token-name to --password in an earlier commit). Documented the tabcmd-aligned short flags instead. - Corrected Python-version headers on login.py, list_jobs.py, manage_subscriptions.py, publish_workbook.py, refresh_tasks.py, move_workbook_sites.py, publish_datasource.py, and update_workbook_data_freshness_policy.py -- repo floor is 3.10 per pyproject.toml. - list_jobs._wait_for_job: reordered excepts so JobCancelledException (a subclass of JobFailedException) is caught first, otherwise cancelled jobs were reported as failed with the wrong exit code. - login.py sign-in banner now branches on JWTAuth as well, so JWT logins no longer print "Username: None". Header env-var list updated to include TABLEAU_JWT / TABLEAU_JWT_FILE. New JWT support: _shared.py add_common_arguments now exposes --jwt and --jwt-file, resolves TABLEAU_JWT / TABLEAU_JWT_FILE from env, reads a JWT file path into args.jwt during resolve_credentials, and returns TSC.JWTAuth from build_auth when a JWT is present. JWT takes priority over PAT and username/password. Extract-refresh subscription: manage_subscriptions.py create now accepts --on-extract-refresh, which calls SubscriptionItem.on_extract_refresh() to construct a subscription that fires when the referenced extract-refresh schedule completes (the flow introduced in #1861). Rebased this branch onto jac/subscription-refresh-extract-triggered so the flag lands on top of the new API without conflicts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up round of fixes on top of the fresh-eyes review pass. Each
change maps to a specific finding from that review.
Migrate stragglers to _shared (M4). Eight samples still had their own
inline argparse and inline PersonalAccessTokenAuth construction:
explore_{datasource,favorites,webhooks,workbook}.py, extracts.py,
move_workbook_sites.py, refresh_tasks.py, and
update_workbook_data_freshness_policy.py. All now call
_shared.add_common_arguments and _shared.build_auth so the
tabcmd-aligned short-flag convention (-s -t -u -p -l) applies
uniformly and any future credential-handling fix lives in one place.
Skip getting_started/3_hello_universe.py: intentionally a hardcoded
starter with no argparse, aimed at teaching new users to edit the
source directly. Different pedagogy from the CLI samples.
Fix explore_favorites empty-site handling (L8). The favorite-datasource
add and delete calls used to run unconditionally with my_datasource
initialized to None, so on an empty site the sample failed partway.
Both calls are now guarded (add inside the existing
`if all_datasource_items:` block, delete under a new
`if my_datasource is not None:` check).
Drop verify=False TLS bypass (L11). Removed http_options={"verify": False}
from publish_workbook.py and the equivalent
server.add_http_options({"verify": False}) pattern from extracts.py and
update_workbook_data_freshness_policy.py. A sample teaching users to
bypass TLS validation is the wrong first impression; TSC defaults to
verify=True, which is what a paved-path deployment expects. Users on
self-signed dev servers can still set the option at their own call
site.
Delete dead _shared.sign_in() helper (L9). It was not called by any
migrated sample: they all use resolve_credentials + build_auth +
`with server.auth.sign_in(auth):` for the auto-signout context
manager. The helper did not compose with `with` because it returned
a Server object rather than a context manager.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1551.
Motivation
#1551 flagged three concerns with
samples/: credentials always on thecommand line, several samples using
.get()where they meant to page,and no examples for background jobs or subscriptions. Each of the three
gets one commit for reviewability.
Behavior change
Samples only -- no library changes. Users running the sample scripts
directly will see:
Credentials.
samples/_shared.pyaddsresolve_credentials(args)which fills missing sign-in values from env vars
(
TABLEAU_SERVER,TABLEAU_TOKEN_NAME, etc.), a plain.envfile, orgetpass.getpass(), in that precedence order. CLI args continue towork for CI use. Wired into
login.py,publish_workbook.py, andpublish_datasource.pyto establish the pattern; other samples leftalone to keep the diff surgical. Stdlib only, no new deps.
Short flags on the shared helper match tabcmd's canonical set:
-s--server,-t--site,-u--username,-p--password,-l--logging-level.--token-nameand--token-valueare long-only.An earlier iteration of this PR reassigned
-pto--token-name; thatwas corrected before merge -- see 54e51f5.
Pagination fixes. Several samples called
server.<endpoint>.get()and named the resultall_workbooks, whichonly returns the first page (default 100). Replaces those with
TSC.Pager(...)so every page is walked. Where a total count wasdisplayed we still
.get()once to grabtotal_available; that meansone extra request but preserves the count line.
Also fixes an unrelated bug in
getting_started/3_hello_universe.pywhere the "workbooks" section actually queried datasources.
New samples.
list_jobs.py-- background jobs (extract refreshes, publishes,flow runs) with
.filter()queryset API +wait_for_jobmanage_subscriptions.py-- list/create/delete subscriptions withSubscriptionItem/Targetand paginated listingNot exhaustive on coverage -- data alerts, metrics, tables, databases,
virtual connections still have no dedicated sample. Left for follow-up.
Test plan
samples/ has no automated tests; each check below is manual.
python samples/login.py --helpshows the new flags with updated help textTABLEAU_TOKEN_NAME/TABLEAU_TOKEN_VALUEin env and runningpython samples/login.py -s <server>signs in with no secrets on the CLIpython samples/list_jobs.py --hours 24lists recent jobs;--wait <job_id>blocks until completionpython samples/manage_subscriptions.py listprints existing subs;create+deleteround-trips cleanlyexplore_datasource.py,explore_workbook.py,extracts.py,update_workbook_data_freshness_policy.py, andpublish_workbook.pyreturns correct behavior on a >100-item site🤖 Generated with Claude Code