feat(go/adbc): refactor logging instrumentation into OTel tracing - part 2/3 by birschick-bq · Pull Request #4659 · apache/arrow-adbc

birschick-bq · 2026-08-04T21:48:08Z

This pull request refactors the FlightSQL ADBC driver to improve error handling, tracing, and code clarity. The main focus is on replacing logger-based tracing with OpenTelemetry (OTel) tracing, enhancing context propagation, and ensuring errors are consistently handled and returned. Several methods now include OTel tracing spans, and error handling is standardized across the connection implementation.

Tracing and Observability Improvements

Replaced logger-based tracing (doGetWithLogger) with OTel-based tracing (doGetWithTracer) throughout the connection implementation, ensuring all relevant methods now emit OTel spans for better observability. ([[1]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL384-L472), [[2]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL855-R783), [[3]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1112-R1132))
Added OTel tracing spans to methods such as PrepareDriverInfo, GetObjectsCatalogs, GetObjectsDbSchemas, GetObjectsTables, and GetTableSchema, capturing start time, errors, and context for each operation. ([[1]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL821-R743), [[2]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL937-R873), [[3]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL971-R948), [[4]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1016-R984), [[5]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1096-R1081))

Error Handling and API Consistency

Standardized error handling by consistently capturing and returning errors, including context and metadata, across all public methods. This includes propagating errors from context checks and response metadata. ([[1]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL892-R832), [[2]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL955-R899), [[3]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL971-R948), [[4]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1016-R984), [[5]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1047-R1016), [[6]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1077-R1045), [[7]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1112-R1132), [[8]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1162-R1159))
Updated function signatures to use named return values and to properly set and return errors, improving clarity and reliability. ([[1]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL821-R743), [[2]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL937-R873), [[3]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL971-R948), [[4]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1016-R984), [[5]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1096-R1081), [[6]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1162-R1159))

Code Cleanup and Minor Fixes

Removed unused imports (such as log/slog) and the obsolete doGetWithLogger function. ([[1]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL26), [[2]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL384-L472))
Improved variable declarations and error assignments for better Go idiomatic style and code readability. ([[1]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL249-R250), [[2]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL842-R765), [[3]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL855-R783), [[4]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL955-R899), [[5]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1047-R1016), [[6]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1077-R1045), [[7]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1112-R1132), [[8]](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cL1162-R1159))

Test and Output Updates

Updated test assertions to match new trace output naming conventions, reflecting the refactored tracing approach. ([go/adbc/driver/flightsql/flightsql_adbc_test.goL371-R371](https://github.com/apache/arrow-adbc/pull/4659/files#diff-1147cdf8ab7a5dcb1d84bcb2b7a2185cf2fc3802b7eb3718bffadfe990ecdf56L371-R371))

Documentation and Comments

Added or updated comments to clarify the purpose of new and refactored functions, particularly around tracing and error handling. ([go/adbc/driver/flightsql/flightsql_connection.goR236-R237](https://github.com/apache/arrow-adbc/pull/4659/files#diff-5ea5525a7206089962734801d643f4aa6f553637d760a90255d54867270abc8cR236-R237))

These changes collectively improve the maintainability, observability, and robustness of the FlightSQL ADBC driver.Refactors logging instrumentation into OTel tracing

Integrate otelgrpc tracing
updates for flightsql connection
handle correct structured shutdown of database and tracing in correct order (tracingLifecycle)
ensures tracing provider not leaked on failed driver/database startup

Extends: #4655

Is part 2 of 3 changes.

… dev/birschick-bq/flight-sql-log-to-trace-4-connection

…onnection

lidavidm · 2026-08-19T01:49:53Z

+	const spanName = "FlightSQL.Database.Close"
+	startTime := time.Now()
+	var span trace.Span
+	_, span = internal.StartSpan(context.Background(), spanName, d)


Maybe a refactor for later, but I wonder if StartSpan shouldn't also return a NewEndSpanHelper with the start time pre-configured

lidavidm · 2026-08-19T01:53:32Z

 			Unary:  unaryTimeoutInterceptor,
 			Stream: streamTimeoutInterceptor,
 		},
+		{Stream: responseMetadataStreamInterceptor},


(We don't need to trace unary calls?)

lidavidm · 2026-08-19T02:40:38Z

It seems this tries to refactor things from logging.go but duplicates it instead?

It is duplicated only because the return types differ.

Ah, right. It does bother me but it's unavoidable then.

…ight-sql-log-to-trace-4-connection

lidavidm

LGTM, though there's still many instances of unnecessary assignment before return that could be reverted

birschick-bq · 2026-08-21T18:12:22Z

@lidavidm

LGTM, though there's still many instances of unnecessary assignment before return that could be reverted

I'll fix these and then give you a heads-up. Thanks.

birschick-bq · 2026-08-24T22:36:09Z

LGTM, though there's still many instances of unnecessary assignment before return that could be reverted
@lidavidm

I've reverted/refactored the unnecessary assignment of the err variable in this commit:
3035237

Please review and re-approve if okay. Thanks.

… - part 3/3 (#4661) This pull request makes significant improvements to the FlightSQL driver's tracing and logging, replacing legacy logging with structured OpenTelemetry tracing, enhancing error recording, and ensuring better test coverage for tracing-related cleanup. The changes modernize how operation metadata is captured and reported, making tracing more consistent and useful for observability and debugging. **Tracing and Logging Modernization:** * Replaced `slog`-based logging with OpenTelemetry tracing throughout `flightsql_statement.go` and `flightsql_bulk_ingest.go`, using `attribute.KeyValue` for structured event attributes and adding detailed span events for operation start and finish. This includes updating helper functions to return attributes in the new format. * Updated error handling in tracing: errors are now consistently recorded as span errors with stack traces, and operation-specific error wrapping is improved for clarity. **API and Internal Refactoring:** * Refactored method signatures and internal calls to use the new tracing and attribute formats, including changes to `SetSqlQuery`, `ExecuteQuery`, `ExecuteUpdate`, and `Prepare` methods in `statement`. * Changed the construction of the Flight client to add a unary interceptor for response metadata, improving consistency for both unary and stream calls. **Testing Improvements:** * Added a new test `TestFlightSQLTracingCleansUpAfterConstructionFailure` to ensure that tracing resources are properly cleaned up even when database construction fails, preventing resource leaks. **Dependency and Import Updates:** * Updated imports to include OpenTelemetry packages and internal helpers for tracing, and removed unused logging imports. **Helper and Utility Updates:** * Updated helper functions for generating tracing attributes and correlation metadata to use the new attribute-based approach, improving code clarity and maintainability. These changes collectively ensure that the driver emits rich, structured telemetry for all major operations, facilitates easier debugging, and aligns with modern observability practices. --- Extends: #4659 Part 3/3 --------- Co-authored-by: Bruce Irschick (Bit Quill Technologies Inc) <v-birschick@microsoft.com>

v-birschick-msft added 9 commits August 4, 2026 12:52

feat(go/adbc): extend tracing lifecycle helpers

b4f536c

feat(go/adbc): add Flight SQL tracing utilities

0214000

feat(go/adbc): trace Flight SQL record readers

0398194

feat(go/adbc): trace Flight SQL connections

d951702

fix codespell finding

7974884

Merge branch 'dev/birschick-bq/flight-sql-log-to-trace-3-reader' into…

fb5fd6c

… dev/birschick-bq/flight-sql-log-to-trace-4-connection

remove unused methods (golangci findings)

bc3dffb

remove unused methods (golangci-lint finding)

8f08d10

empty - retest

e3edd42

birschick-bq requested a review from zeroshade as a code owner August 4, 2026 21:48

birschick-bq changed the title ~~Dev/birschick bq/flight sql log to trace 4 connection~~ feat(go/adbc): refactor logging instrumentation into OTel tracing - part 2/3 Aug 4, 2026

remove unused methods (golangci-lint findings)

2e1088b

birschick-bq marked this pull request as draft August 4, 2026 21:56

remove unused type (golangci-lint finding)

cb93aa4

This was referenced Aug 4, 2026

feat(go/adbc): refactor logging instrumentation into OTel tracing - part 3/3 #4661

Merged

go/adbc:/drivers/flightsql convert slog messages to OTel trace spans and events #4589

Closed

v-birschick-msft added 8 commits August 11, 2026 13:22

improvements from code review comments

8446e81

empty - retest

d602600

Address code review comment to add documentation for EndSpanHelper

7165bc7

Merge branch 'dev/birschick-bq/flight-sql-log-to-trace-3-reader' into…

f39374b

… dev/birschick-bq/flight-sql-log-to-trace-4-connection

add unsaved changes

7809374

correct test failure

85bd804

empty - retest

1cf5b4b

Merge branch 'main' into dev/birschick-bq/flight-sql-log-to-trace-4-c…

a80a4d7

…onnection

birschick-bq marked this pull request as ready for review August 19, 2026 01:01

lidavidm reviewed Aug 19, 2026

View reviewed changes

v-birschick-msft added 3 commits August 19, 2026 17:22

code review updates

47c5660

Merge remote-tracking branch 'upstream/main' into dev/birschick-bq/fl…

d9c5c84

…ight-sql-log-to-trace-4-connection

inline single use closeTracing function

c4bf15c

lidavidm approved these changes Aug 21, 2026

View reviewed changes

Comment thread go/adbc/driver/flightsql/flightsql_connection.go

refactor and improve unnecessary err variable assignement before return

3035237

lidavidm approved these changes Aug 24, 2026

View reviewed changes

lidavidm merged commit 75f2ae9 into apache:main Aug 24, 2026
49 checks passed

Navigation Menu

Uh oh!

feat(go/adbc): refactor logging instrumentation into OTel tracing - part 2/3 - #4659

feat(go/adbc): refactor logging instrumentation into OTel tracing - part 2/3#4659
lidavidm merged 23 commits into
apache:mainfrom
birschick-bq:dev/birschick-bq/flight-sql-log-to-trace-4-connection

birschick-bq commented Aug 4, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

lidavidm Aug 19, 2026

Uh oh!

Uh oh!

lidavidm Aug 19, 2026

Uh oh!

birschick-bq Aug 20, 2026

Uh oh!

Uh oh!

Uh oh!

lidavidm Aug 19, 2026

Uh oh!

birschick-bq Aug 20, 2026

Uh oh!

lidavidm Aug 21, 2026

Uh oh!

lidavidm left a comment

Uh oh!

Uh oh!

birschick-bq commented Aug 21, 2026

Uh oh!

birschick-bq commented Aug 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

birschick-bq commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lidavidm Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lidavidm Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

birschick-bq Aug 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lidavidm Aug 19, 2026

Choose a reason for hiding this comment

Uh oh!

birschick-bq Aug 20, 2026

Choose a reason for hiding this comment

Uh oh!

lidavidm Aug 21, 2026

Choose a reason for hiding this comment

Uh oh!

lidavidm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

birschick-bq commented Aug 21, 2026

Uh oh!

birschick-bq commented Aug 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

birschick-bq commented Aug 4, 2026 •

edited

Loading