Tracing¶

Recording agent runs as traces (docs/design.md §2, FR-T1..FR-T5).

Typical use:

import agentlens

@agentlens.wrap_tool
def lookup_order(order_id: str) -> dict[str, str]: ...

llm = agentlens.wrap_llm(MockLLM(["Your order shipped."]))

with agentlens.run("support-agent", input="Where is order 1234?") as trace:
    trace.output = my_agent(llm, "Where is order 1234?")

TraceSink `module-attribute` ¶

TraceSink = Callable[[Trace], None]

Receives each finished trace, e.g. store.save_trace or list.append.

DEFAULT_PATTERNS `module-attribute` ¶

DEFAULT_PATTERNS: tuple[str, ...] = ('-----BEGIN [A-Z ]*PRIVATE KEY-----[\\s\\S]*?-----END [A-Z ]*PRIVATE KEY-----', '\\bsk-[A-Za-z0-9][A-Za-z0-9_\\-]{15,}', '(?i)\\bBearer\\s+(?P<secret>[A-Za-z0-9\\-._~+/]{8,}=*)', '\\b(?:AKIA|ASIA)[0-9A-Z]{16}\\b', '\\bgh[pousr]_[A-Za-z0-9]{36,}\\b', '\\bgithub_pat_[A-Za-z0-9_]{22,}\\b', '\\bAIza[0-9A-Za-z\\-_]{35}\\b', '\\bxox[abprs]-[A-Za-z0-9\\-]{10,}', '\\beyJ[A-Za-z0-9_\\-]{5,}\\.eyJ[A-Za-z0-9_\\-]{5,}\\.[A-Za-z0-9_\\-]+', '(?i)\\b[\\w-]*(?:api[_-]?key|secret|password|passwd|token)[\\"\']?\\s*[:=]\\s*[\\"\']?(?P<secret>[^\\s\\"\',;&]{8,})')

Regular expressions for secrets inside strings. When a pattern has a group named secret only that group is masked (Bearer [REDACTED]), otherwise the whole match is.

DEFAULT_REDACTOR `module-attribute` ¶

DEFAULT_REDACTOR: Redactor = PatternRedactor()

The redactor agentlens.run uses unless told otherwise.

DEFAULT_SENSITIVE_KEYS `module-attribute` ¶

DEFAULT_SENSITIVE_KEYS: frozenset[str] = frozenset({'api_key', 'apikey', 'x_api_key', 'authorization', 'proxy_authorization', 'cookie', 'set_cookie', 'password', 'passwd', 'secret', 'client_secret', 'secret_key', 'private_key', 'access_token', 'refresh_token', 'id_token', 'auth_token', 'token', 'aws_secret_access_key', 'aws_session_token'})

Mapping keys (compared case-insensitively, - treated as _) whose values are replaced wholesale, whatever their type.

REDACTED `module-attribute` ¶

REDACTED = '[REDACTED]'

The default replacement for masked secrets.

Redactor `module-attribute` ¶

Redactor = Callable[[JsonValue], JsonValue]

Takes a JSON payload and returns it with secrets masked. Must not mutate its input.

TOOL_NAME_ATTR `module-attribute` ¶

TOOL_NAME_ATTR = '__agentlens_tool__'

Attribute set on functions returned by wrap_tool: the tool (span) name.

Recorder ¶

Recorder(name: str, *, input: object = None, metadata: Mapping[str, object] | None = None, sink: TraceSink | None = None, redactor: Redactor | None = DEFAULT_REDACTOR)

Builds one trace span by span and flushes it to a sink when finished.

finished `property` ¶

finished: bool

Whether finish has already run.

start_span ¶

start_span(kind: SpanKind, name: str, *, parent_id: str | None = None, input: object = None, attributes: Mapping[str, object] | None = None) -> Span

Create a running span and append it to the trace.

Spans are appended in start order, so a parent always precedes its children in trace.spans.

end_span ¶

end_span(span: Span, *, output: object = None, error: BaseException | None = None, usage: Usage | None = None, attributes: Mapping[str, object] | None = None) -> None

Close span with its output, or with the exception that ended it.

usage (token counts of an LLM call) and attributes known only once the work is done (e.g. the model that answered) are added as well; attributes are merged into the ones given at start.

finish ¶

finish(*, error: BaseException | None = None) -> Trace

End the run and hand the trace to the sink. Idempotent.

The sink receives a redacted copy (see redact_trace); the returned live trace is left untouched. If redaction fails the trace is not passed to the sink: dropping a trace is better than persisting a secret (NFR-5).

PatternRedactor ¶

PatternRedactor(patterns: Iterable[str | Pattern[str]] = DEFAULT_PATTERNS, *, keys: Iterable[str] = DEFAULT_SENSITIVE_KEYS, replacement: str = REDACTED)

Redactor masking sensitive mapping keys and secret-looking substrings.

It walks nested lists and dicts and returns a new value; the input is never mutated.

.. code-block:: python

redactor = PatternRedactor(
    patterns=(*DEFAULT_PATTERNS, r"\bINTERNAL-[0-9]{6}\b"),
    keys=DEFAULT_SENSITIVE_KEYS | {"session_id"},
)
agentlens.run("agent", sink=store.save_trace, redactor=redactor)

Parameters:

Name	Type	Description	Default
`patterns`	`Iterable[str \| Pattern[str]]`	Regular expressions matched inside every string. A group named `secret` limits masking to that group.	`DEFAULT_PATTERNS`
`keys`	`Iterable[str]`	Mapping keys whose values are always masked. Keys ending in `_api_key`, `_secret` or `_password` are masked as well.	`DEFAULT_SENSITIVE_KEYS`
`replacement`	`str`	The text that replaces each secret.	`REDACTED`

redact_text ¶

redact_text(text: str) -> str

Mask every pattern match in text.

is_sensitive_key ¶

is_sensitive_key(key: str) -> bool

Whether the value stored under key must be masked entirely.

TracedLLM ¶

TracedLLM(client: LLMClient, *, name: str | None = None, provider: str | None = None, include_raw: bool = False)

An LLMClient that records each call as an llm_call span.

Created by wrap_llm; see there for what is recorded. Attributes not defined here are looked up on the wrapped client, so client-specific helpers (e.g. MockLLM.requests) keep working through the wrapper.

client `instance-attribute` ¶

client = client

The wrapped client; calls go straight to it outside a run.

complete ¶

complete(request: LLMRequest) -> LLMResponse

Call the wrapped client's complete, recording an llm_call span.

acomplete `async` ¶

acomplete(request: LLMRequest) -> LLMResponse

Call the wrapped client's acomplete, recording an llm_call span.

current_span ¶

current_span() -> Span | None

The innermost open span in this context, or None.

current_trace ¶

current_trace() -> Trace | None

The trace being recorded in this context, or None outside run.

run ¶

run(name: str, *, input: object = None, metadata: Mapping[str, object] | None = None, sink: TraceSink | None = None, redactor: Redactor | None = DEFAULT_REDACTOR) -> Iterator[Trace]

Record one agent run as a Trace.

.. code-block:: python

with agentlens.run("support-agent", input=question) as trace:
    trace.output = agent(question)
print(trace.latency_ms, trace.tool_calls)

Spans created inside the block (by trace, span or the LLM wrappers) are attached to the yielded trace. When the block exits the trace gets its end_time (and error if the block raised) and a redacted copy is passed to sink; the yielded trace itself is not redacted. Exceptions from the block are re-raised after recording.

Runs may be nested; an inner run records an independent trace.

Parameters:

Name	Type	Description	Default
`name`	`str`	Trace name, e.g. the agent or suite name.	required
`input`	`object`	The agent input; converted to JSON (`repr` as a fallback).	`None`
`metadata`	`Mapping[str, object] \| None`	Run context such as git sha, case id or repetition.	`None`
`sink`	`TraceSink \| None`	Called once with the finished trace (e.g. `store.save_trace`). Errors raised by the sink are logged, not propagated.	`None`
`redactor`	`Redactor \| None`	Masks secrets in the copy given to `sink` (NFR-5); defaults to `DEFAULT_REDACTOR`. `None` disables redaction. If it raises, the error is logged and the trace is not passed to the sink.	`DEFAULT_REDACTOR`

span ¶

span(name: str, *, kind: SpanKind | str = CUSTOM, input: object = None, attributes: Mapping[str, object] | None = None) -> Iterator[Span | None]

Record a block of code as a span.

Yields the running Span (set span.output to record a result), or None when no run is active.

.. code-block:: python

with agentlens.span("search", kind="retrieval", input=query) as s:
    docs = index.search(query)
    if s is not None:
        s.output = [d.id for d in docs]

trace ¶

trace(func: Callable[P, R]) -> Callable[P, R]

trace(*, kind: SpanKind | str = ..., name: str | None = ...) -> Callable[[Callable[P, R]], Callable[P, R]]

trace(func: Callable[P, R] | None = None, /, *, kind: SpanKind | str = CUSTOM, name: str | None = None) -> Callable[P, R] | Callable[[Callable[P, R]], Callable[P, R]]

Decorator recording each call of a sync or async function as a span.

The span input is the call's arguments by parameter name (without self/cls), the output is the return value, and an exception is stored in span.error before being re-raised.

.. code-block:: python

@agentlens.trace(kind="tool_call")
def lookup_order(order_id: str) -> dict: ...

@agentlens.trace                      # kind=custom, name=function name
async def plan(question: str) -> list[str]: ...

Parameters:

Name	Type	Description	Default
`kind`	`SpanKind \| str`	The `SpanKind` (or its string value).	`CUSTOM`
`name`	`str \| None`	Span name; defaults to the function's `__qualname__`.	`None`

Raises:

Type	Description
`TypeError`	if applied to a generator function (not supported yet).
`ValueError`	if `kind` is not a valid span kind.

redact_trace ¶

redact_trace(trace: Trace, redactor: Redactor | None = None) -> Trace

Return a copy of trace with every payload passed through redactor.

Redacted fields: the trace's input, output, metadata and error and each span's input, output, attributes and error. Ids, names, timings and usage are kept as they are. The original trace is not modified.

Parameters:

Name	Type	Description	Default
`trace`	`Trace`	The trace to redact.	required
`redactor`	`Redactor \| None`	Defaults to `DEFAULT_REDACTOR`.	`None`

Raises:

Type	Description
`Exception`	whatever `redactor` raises; `agentlens.run` catches it and drops the trace rather than persisting unredacted data.

wrap_llm ¶

wrap_llm(client: LLMClient, *, name: str | None = None, provider: str | None = None, include_raw: bool = False) -> TracedLLM

Wrap an LLM client so every call inside run is recorded.

Each complete/acomplete call becomes an llm_call span:

name: name if given, else the requested model, else the client's model attribute, else the client class name;
input: the LLMRequest as JSON (fields left at their defaults omitted);
output: the LLMResponse as JSON, without usage (stored in span.usage) and without the provider's raw payload unless include_raw=True;
usage: the response's token counts and cost (None if unreported);
attributes: client (class path), provider (if known), model, temperature, max_tokens, n_messages, n_tools, tool_names, response_format, request_hash (cassette key, see ADR 0004) and, once answered, response_model and n_tool_calls.

Errors raised by the client are stored in span.error and re-raised. Outside a run the wrapper calls straight through. Wrapping an already wrapped client returns it unchanged.

.. code-block:: python

llm = agentlens.wrap_llm(MockLLM(["Hello!"]))
with agentlens.run("agent") as trace:
    llm.complete(LLMRequest(messages="Hi"))
assert trace.spans[0].usage is not None

Parameters:

Name	Type	Description	Default
`client`	`LLMClient`	Any object implementing `LLMClient`.	required
`name`	`str \| None`	Fixed span name instead of the model name.	`None`
`provider`	`str \| None`	`provider` attribute; defaults to the client's own `provider` attribute when it is a string.	`None`
`include_raw`	`bool`	Also record `LLMResponse.raw` in the span output.	`False`

wrap_tool ¶

wrap_tool(fn: Callable[P, R], /, *, name: str | None = ...) -> Callable[P, R]

wrap_tool(*, name: str | None = ...) -> Callable[[Callable[P, R]], Callable[P, R]]

wrap_tool(fn: Callable[P, R] | None = None, /, *, name: str | None = None) -> Callable[P, R] | Callable[[Callable[P, R]], Callable[P, R]]

Record each call of a tool (sync or async callable) as a tool_call span.

Use it as a plain wrapper or as a decorator, with or without a name:

.. code-block:: python

search = agentlens.wrap_tool(index.search, name="search_docs")

@agentlens.wrap_tool
def lookup_order(order_id: str) -> dict[str, str]: ...

@agentlens.wrap_tool(name="refund")
async def issue_refund(order_id: str, amount: float) -> bool: ...

The span name is the tool name (name, else the function's __name__), which is what Trace.tool_calls and the tool_sequence evaluator compare. The input is the call's arguments by parameter name (self/cls dropped), the output the return value; an exception is recorded in span.error and re-raised. Outside a run the tool is called directly. The returned function exposes the original as __wrapped__ and the tool name as __agentlens_tool__; wrapping it again returns it unchanged, or re-wraps the original under a new name (never two nested spans).

Raises:

Type	Description
`TypeError`	if `fn` is a generator function (not supported yet).

Tracing¶

TraceSink module-attribute ¶

DEFAULT_PATTERNS module-attribute ¶

DEFAULT_REDACTOR module-attribute ¶

DEFAULT_SENSITIVE_KEYS module-attribute ¶

REDACTED module-attribute ¶

Redactor module-attribute ¶

TOOL_NAME_ATTR module-attribute ¶

Recorder ¶

finished property ¶

start_span ¶

end_span ¶

finish ¶

PatternRedactor ¶

redact_text ¶

is_sensitive_key ¶

TracedLLM ¶

client instance-attribute ¶

complete ¶

acomplete async ¶

current_span ¶

current_trace ¶

run ¶

span ¶

trace ¶

redact_trace ¶

wrap_llm ¶

wrap_tool ¶

TraceSink `module-attribute` ¶

DEFAULT_PATTERNS `module-attribute` ¶

DEFAULT_REDACTOR `module-attribute` ¶

DEFAULT_SENSITIVE_KEYS `module-attribute` ¶

REDACTED `module-attribute` ¶

Redactor `module-attribute` ¶

TOOL_NAME_ATTR `module-attribute` ¶

finished `property` ¶

client `instance-attribute` ¶

acomplete `async` ¶