Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Docs

on:
push:
branches: [main]
workflow_dispatch:

permissions:
contents: read
pages: write
id-token: write

concurrency:
group: pages
cancel-in-progress: false

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7

- uses: actions/setup-python@v7
with:
python-version: "3.12"

- run: pip install -e ".[docs]"

- run: pdoc --output-directory site --docformat google logquill

- uses: actions/upload-pages-artifact@v3
with:
path: site

deploy:
needs: build
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- id: deployment
uses: actions/deploy-pages@v4
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,20 @@ All notable changes to this project are documented in this file.

## Unreleased

- Phase 9, docs, complete:
- Every public class and function across the package now has a docstring
explaining what it does and why you'd reach for it — not a restatement
of its name — matching the bar already set by the existing public API.
- A full API reference, generated straight from those docstrings with
[pdoc](https://pdoc.dev) (`pip install logquill[docs]`), is published to
GitHub Pages and rebuilt automatically on every push to `main` via
`.github/workflows/docs.yml`.
- README: a new "Kubernetes" section explains why `ConsoleTransport`
(stdout/stderr, captured by the node's log agent) belongs in a
container instead of `FileTransport` (writes to an ephemeral
filesystem nothing aggregates), and how to avoid losing queued records
to `SIGTERM` when `async_dispatch=True` is combined with a container's
termination grace period.
- Phase 8, CLI, complete:
- `logquill tail <file> [--level=] [--json] [-f/--follow] [-n/--lines]` — a
`logquill` console-script for tailing a JSONL log file in local dev.
Expand Down
40 changes: 40 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -781,6 +781,33 @@ the same decorator under a name that reads naturally at each platform's own
handler definition — the flush-before-return behavior is identical across
all three.

### Kubernetes

Default to `ConsoleTransport`, not `FileTransport`, for anything running in a
container. A container's filesystem is ephemeral and invisible to the rest of
the cluster — a log file written inside it disappears the moment the pod is
rescheduled, and nothing aggregates it in the meantime unless you also run a
sidecar to tail it back out. Writing to stdout/stderr instead costs nothing
extra: every major container runtime already captures both streams, and the
node-level log agent your cluster runs (Fluentd, Fluent Bit, Vector, or your
cloud provider's own) ships them to your aggregator without any code on your
side that needs to know that agent exists:

```python
from logquill import ConsoleTransport, Logger

logger = Logger("app", transports=[ConsoleTransport()])
```

If you do reach for `async_dispatch=True` in a container, make sure
`logger.close()` (or `logger.flush()`) runs before the container actually
stops — Kubernetes sends `SIGTERM` and then kills the process after
`terminationGracePeriodSeconds` (30s by default) regardless of whether it's
finished shutting down, so a queued-but-undispatched record can be lost if
nothing catches the signal. A `SIGTERM` handler or `preStop` hook that calls
`logger.close(timeout=...)` closes that gap the same way `with_lambda` closes
it for a serverless freeze.

## Context propagation, exception capture & the stdlib bridge

`bind_context()` binds request-scoped values for a `with` block — every
Expand Down Expand Up @@ -881,6 +908,19 @@ colors) when writing to a terminal; pass `--no-color` to disable that, or
that isn't valid JSON, or isn't a JSON object, is skipped with a warning on
stderr rather than aborting the whole tail.

## API reference

Every public class and function is documented with a docstring; the full
reference, generated from those docstrings with [pdoc](https://pdoc.dev), is
published at
[nikhilvdev.github.io/logquill-python](https://nikhilvdev.github.io/logquill-python/)
and rebuilt on every push to `main`. To build it locally:

```bash
pip install -e ".[docs]"
pdoc --docformat google logquill # opens a local server; add -o DIR to write static HTML instead
```

## Development

```bash
Expand Down
9 changes: 9 additions & 0 deletions logquill/adapters/autogen.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,9 @@ class AutoGenAdapter(LogQuillAdapter, logging.Handler):
"""

def __init__(self, agent_log: Logger) -> None:
"""Attaches this adapter as a `logging.Handler` on AutoGen's
`EVENT_LOGGER_NAME` logger immediately — active as soon as it's
constructed, no separate "start" call needed."""
LogQuillAdapter.__init__(self, agent_log)
logging.Handler.__init__(self)
self._event_logger = logging.getLogger(EVENT_LOGGER_NAME)
Expand All @@ -78,10 +81,16 @@ def __init__(self, agent_log: Logger) -> None:
self._event_logger.addHandler(self)

def close(self) -> None:
"""Detach this handler from AutoGen's event logger, then run the
base `logging.Handler.close()`."""
self._event_logger.removeHandler(self)
super().close()

def emit(self, record: logging.LogRecord) -> None:
"""Unpack an `autogen_core` structured event object off `record.msg`
and forward it as the matching `.action()`/`.observation()`/
`.error()` call; silently ignored if `record.msg` isn't one of
those structured event objects (e.g. an ordinary log line)."""
kwargs = getattr(record.msg, "kwargs", None)
if not isinstance(kwargs, dict):
return # not one of autogen_core.logging's structured event objects
Expand Down
2 changes: 2 additions & 0 deletions logquill/adapters/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,6 @@ class LogQuillAdapter:
"""

def __init__(self, agent_log: Logger) -> None:
"""`agent_log` is the `Logger` (typically `.child(...)` with a
`RunPlugin` attached) every translated event is forwarded onto."""
self.log = agent_log
6 changes: 6 additions & 0 deletions logquill/adapters/crewai.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,9 @@ class CrewAIAdapter(LogQuillAdapter, BaseEventListener): # type: ignore[misc]
"""

def __init__(self, agent_log: Logger) -> None:
"""Registers this listener's handlers on CrewAI's event bus
immediately (`BaseEventListener.__init__`'s own behavior) — keep a
reference alive for as long as you want it active."""
LogQuillAdapter.__init__(self, agent_log)
self._open_spans: dict[str, SpanContext] = {}
self._call_starts: dict[str, datetime] = {}
Expand Down Expand Up @@ -132,6 +135,9 @@ def _step_end(self, name: str, event: Any, *, error: str | None = None) -> None:
# `@`-applying an `Any`-typed decorator even though the wrapped method
# itself is fully annotated. A plain call sidesteps that check.
def setup_listeners(self, crewai_event_bus: Any) -> None:
"""`BaseEventListener`'s required override: subscribes every CrewAI
event this adapter translates (crew/task/agent/tool/LLM
start/end/error) to its matching handler on `crewai_event_bus`."""
crewai_event_bus.on(CrewKickoffStartedEvent)(self._on_crew_started)
crewai_event_bus.on(CrewKickoffCompletedEvent)(self._on_crew_completed)
crewai_event_bus.on(CrewKickoffFailedEvent)(self._on_crew_failed)
Expand Down
29 changes: 29 additions & 0 deletions logquill/adapters/langchain.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,8 @@ class LangChainAdapter(LogQuillAdapter, BaseCallbackHandler): # type: ignore[mi
"""

def __init__(self, agent_log: Logger) -> None:
"""Pass the resulting instance into a chain/agent's `callbacks=[...]`
— no separate registration step needed."""
LogQuillAdapter.__init__(self, agent_log)
BaseCallbackHandler.__init__(self)
self._open_spans: dict[UUID, SpanContext] = {}
Expand All @@ -91,6 +93,8 @@ def on_chain_start(
parent_run_id: UUID | None = None,
**kwargs: Any,
) -> Any:
"""Opens a `span()` for this chain run, keyed by `run_id` so the
matching `on_chain_end`/`on_chain_error` can close it."""
name = _tool_name(serialized, "chain")
span = self.log.span(
name,
Expand All @@ -108,6 +112,7 @@ def on_chain_end(
parent_run_id: UUID | None = None,
**kwargs: Any,
) -> Any:
"""Closes the `span()` opened by the matching `on_chain_start`."""
span = self._open_spans.pop(run_id, None)
if span is not None:
span.__exit__(None, None, None)
Expand All @@ -120,6 +125,9 @@ def on_chain_error(
parent_run_id: UUID | None = None,
**kwargs: Any,
) -> Any:
"""Closes the `span()` opened by the matching `on_chain_start`,
propagating `error` into it so the span's own record captures the
failure (at `ERROR`, with `meta.error` set)."""
span = self._open_spans.pop(run_id, None)
if span is not None:
span.__exit__(type(error), error, error.__traceback__)
Expand All @@ -135,6 +143,8 @@ def on_llm_start(
parent_run_id: UUID | None = None,
**kwargs: Any,
) -> Any:
"""Records the call's start time (for the matching `on_llm_end`'s
`duration_ms`) and emits `.action("llm_start")`."""
self._call_starts[run_id] = time.monotonic()
self.log.action("llm_start", **_span_ids(run_id, parent_run_id))

Expand All @@ -146,6 +156,8 @@ def on_llm_end(
parent_run_id: UUID | None = None,
**kwargs: Any,
) -> Any:
"""Emits `.observation("llm_end")` with `duration_ms` measured since
the matching `on_llm_start`."""
duration_ms = self._duration_ms(run_id)
meta = _span_ids(run_id, parent_run_id)
if duration_ms is not None:
Expand All @@ -160,6 +172,9 @@ def on_llm_error(
parent_run_id: UUID | None = None,
**kwargs: Any,
) -> Any:
"""Emits `.error("llm_error")`; discards the timing recorded by
`on_llm_start` without using it, since a failed call has no
meaningful `duration_ms` to report here."""
self._duration_ms(run_id)
self.log.error("llm_error", error=str(error), **_span_ids(run_id, parent_run_id))

Expand All @@ -181,6 +196,11 @@ def on_agent_action(
parent_run_id: UUID | None = None,
**kwargs: Any,
) -> Any:
"""Emits `.action(tool)` for the agent's chosen action; leaves
`span_id`/`parent_span_id` unset since `run_id` here is the
enclosing chain's own id, not a fresh one — the ambient-span
auto-stamp in `Logger._log` supplies the correct
`parent_span_id` instead."""
tool = getattr(action, "tool", "agent_action")
self.log.action(tool)

Expand All @@ -192,6 +212,8 @@ def on_agent_finish(
parent_run_id: UUID | None = None,
**kwargs: Any,
) -> Any:
"""Emits `.decision("agent_finish")` for the agent's concluding
decision."""
self.log.decision("agent_finish")

# -- tools: action (start) / observation (end) / error ------------------
Expand All @@ -205,6 +227,8 @@ def on_tool_start(
parent_run_id: UUID | None = None,
**kwargs: Any,
) -> Any:
"""Records the call's start time (for the matching `on_tool_end`'s
`duration_ms`) and emits `.action(name)` for the tool call."""
name = _tool_name(serialized, "tool")
self._call_starts[run_id] = time.monotonic()
self.log.action(name, **_span_ids(run_id, parent_run_id))
Expand All @@ -217,6 +241,8 @@ def on_tool_end(
parent_run_id: UUID | None = None,
**kwargs: Any,
) -> Any:
"""Emits `.observation("tool_end")` with `duration_ms` measured
since the matching `on_tool_start`."""
duration_ms = self._duration_ms(run_id)
meta = _span_ids(run_id, parent_run_id)
if duration_ms is not None:
Expand All @@ -231,5 +257,8 @@ def on_tool_error(
parent_run_id: UUID | None = None,
**kwargs: Any,
) -> Any:
"""Emits `.error("tool_error")`; discards the timing recorded by
`on_tool_start` without using it, since a failed call has no
meaningful `duration_ms` to report here."""
self._duration_ms(run_id)
self.log.error("tool_error", error=str(error), **_span_ids(run_id, parent_run_id))
5 changes: 5 additions & 0 deletions logquill/adapters/langgraph.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,14 @@ class exists for that reason alone; everything else is inherited
"""

def on_interrupt(self, event: Any) -> None:
"""Emits `.observation("graph_interrupted")` when the graph pauses on
an `interrupt()` call, carrying each pending interrupt's `id`/`value`
plus the checkpoint fields."""
meta = _checkpoint_meta(event)
meta["interrupts"] = [{"id": i.id, "value": i.value} for i in event.interrupts]
self.log.observation("graph_interrupted", **meta)

def on_resume(self, event: Any) -> None:
"""Emits `.action("graph_resumed")` when the graph resumes from a
persisted checkpoint."""
self.log.action("graph_resumed", **_checkpoint_meta(event))
13 changes: 13 additions & 0 deletions logquill/adapters/llamaindex.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,9 @@ def prepare_to_exit_span(
result: Any = None,
**kwargs: Any,
) -> Any:
"""Logs a successfully-completed span (`.info()`, with `duration_ms`
and `parent_span_id` if nested) after delegating to
`SimpleSpanHandler` for the actual span bookkeeping."""
span = super().prepare_to_exit_span(id_, bound_args, instance, result, **kwargs)
self._log_close(id_, span)
return span
Expand All @@ -71,6 +74,9 @@ def prepare_to_drop_span(
err: BaseException | None = None,
**kwargs: Any,
) -> Any:
"""Logs a span that exited via exception (`.error()`, with `err`'s
message) after delegating to `SimpleSpanHandler` for the actual span
bookkeeping."""
span = super().prepare_to_drop_span(id_, bound_args, instance, err, **kwargs)
self._log_close(id_, span, error=err)
return span
Expand Down Expand Up @@ -101,6 +107,10 @@ class _EventLogger(BaseEventHandler): # type: ignore[misc]
log: Logger

def handle(self, event: Any, **kwargs: Any) -> Any:
"""`BaseEventHandler`'s required override: classifies `event` by its
`class_name()` suffix (`*StartEvent`/`*EndEvent`/`*ErrorEvent`) and
forwards it as the matching `.action()`/`.observation()`/`.error()`
call; a few noisy progress/delta event types are skipped entirely."""
name = event.class_name()
if name.endswith(_SKIPPED_SUFFIXES):
return None
Expand Down Expand Up @@ -149,6 +159,9 @@ class LlamaIndexAdapter(LogQuillAdapter):
"""

def __init__(self, agent_log: Logger) -> None:
"""Registers a span handler and an event handler on LlamaIndex's
global instrumentation dispatcher immediately — active as soon as
it's constructed, no separate "start" call needed."""
super().__init__(agent_log)
self._span_handler = _SpanLogger(log=agent_log)
self._event_handler = _EventLogger(log=agent_log)
Expand Down
6 changes: 6 additions & 0 deletions logquill/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@


def build_parser() -> argparse.ArgumentParser:
"""Builds the `logquill` CLI's argument parser (currently just the
`tail` subcommand); split out from `main()` so tests can inspect/exercise
it without going through `sys.argv`."""
parser = argparse.ArgumentParser(
prog="logquill", description="LogQuill command-line tools for local development."
)
Expand Down Expand Up @@ -221,6 +224,9 @@ def _run_tail(


def main(argv: Sequence[str] | None = None) -> int:
"""The `logquill` console-script entry point: parses `argv` (defaulting
to `sys.argv`) and dispatches to the matching subcommand, returning the
process exit code."""
parser = build_parser()
args = parser.parse_args(argv)

Expand Down
6 changes: 5 additions & 1 deletion logquill/formatter.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,15 @@
class Formatter(Protocol):
"""`format(record) -> string`, per the transport contract shared with logquill-js."""

def format(self, record: LogRecord) -> str: ...
def format(self, record: LogRecord) -> str:
"""Render `record` to the string a transport will write."""
...


class JSONFormatter:
"""Serializes a record to the canonical JSON line shape."""

def format(self, record: LogRecord) -> str:
"""Serializes `record` to a single compact JSON line; non-JSON-native
values fall back to `str()` rather than raising."""
return json.dumps(record, separators=(",", ":"), default=str)
8 changes: 8 additions & 0 deletions logquill/handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,10 +82,18 @@ class LogQuillHandler(logging.Handler):
"""

def __init__(self, logger: Logger, level: int = logging.NOTSET) -> None:
"""`logger` is the LogQuill `Logger` every bridged stdlib record is
forwarded onto; `level` is this handler's own stdlib-level filter,
applied on top of `logger`'s own level."""
super().__init__(level)
self._logger = logger

def emit(self, record: logging.LogRecord) -> None:
"""Translates a stdlib `logging.LogRecord` into a LogQuill call:
maps its level, folds any `extra=` fields into `meta`, formats
`exc_info` if present, and routes it through the wrapped `Logger`'s
own `_log` path. Delegates to `self.handleError()` (never raises)
if translation itself fails."""
try:
meta: dict[str, Any] = {
key: value
Expand Down
Loading