A linter for agent runs. It reads the execution trace — what your agent actually did — and flags structural bugs deterministically. Hard defects fail CI; events and candidates are shown for review, never asserted — no second model ever judges the trace.
tracelint reads a finished agent trace and flags structural defects, each with the exact step as evidence — no second model judges the trace. Every finding carries a confidence tier that decides whether it fails CI.
In the tables below, the Findings column shows the rules that fired (coloured chips) plus how many checks were suppressed because the trace lacked a field they needed — a suppression is never a clean pass.
One agent trajectory linted end to end: the steps the agent actually took, then exactly what tracelint found and the step it points to — the same output as tracelint check your-trace.json.
Task: Cancel order 0000. · exit 0
get_order_status(order_id='0000')cancel_order(order_id='0000', reason='customer_request')| Tier | Rule | Finding | Steps |
|---|---|---|---|
| hard_event | R2a tool_error_event | 'get_order_status' returned an error (http 404) | 4 |
| hard_event | R2a tool_error_event | 'cancel_order' returned an error (http 404) | 7 |
| candidate | R2b error_mishandled | 'get_order_status' returned a structured error that was not retried before the agent proceeded (acknowledgement cannot be verified deterministically) | 4 |
| candidate | R2b error_mishandled | 'cancel_order' returned a structured error that was not retried before the agent proceeded (acknowledgement cannot be verified deterministically) | 7 |
One planted instance of each defect, clean controls, and legitimate-but-suspicious cases (a real retry, a value transform, a generated key). Each case asserts an expected outcome; the linter must match it.
| Result | Case | Scenario | Expected | Findings |
|---|---|---|---|---|
| PASS | r1_schema_violationplanted | A tool call whose arguments violate the declared JSON Schema (int, not string). | R1 hard_defect | R1+5 suppressed |
| PASS | r2a_tool_errorplanted | A tool returns a structured error (HTTP 500) — a hard event. | R2a hard_event | R2aR2bR3+4 suppressed |
| PASS | r2b_error_consumedplanted | A value from an errored result is reused as an argument to a side-effecting call. | R2b hard_defect | R2aR2b+2 suppressed |
| PASS | r3_hallucination_candidateplanted | An argument absent from provenance, with no schema annotation → candidate. | R3 candidate (not hard) | R2aR2bR3+2 suppressed |
| PASS | r3_hallucination_hardplanted | The same absent argument, but the schema declares the field 'provided' → hard_defect. | R3 hard_defect | R2aR2bR3+2 suppressed |
| PASS | r4_loopplanted | The agent repeats an identical failing call with no progress → loop candidate. | R4 loop candidate | R2aR2aR2aR2bR4R5 |
| PASS | r5_redundantplanted | An identical read repeated after unrelated work, with no mutation between. | R5 redundant candidate | R5+2 suppressed |
| PASS | clean_controlcontrol | A correct order-cancellation run — the linter must be silent. | no active findings | clean+1 suppressed |
| PASS | suspicious_retrysuspicious | A transient error followed by a successful retry — legitimate, not a loop. | no loop, no hard_defect | R2a+3 suppressed |
| PASS | suspicious_transformsuspicious | A legitimate reformat of a user-provided value (1,234.56 becomes 1234.56). | no R3 | clean+6 suppressed |
| PASS | suspicious_generated_keysuspicious | An idempotency key absent from context but declared 'generated' → not flagged. | no R3 | clean+5 suppressed |
| PASS | r6_malformed_argumentsplanted | A tool call whose emitted arguments are not valid JSON (a broken function call). | R6 hard_defect | R6+6 suppressed |
| PASS | r7_unknown_toolplanted | The agent calls a tool that isn't in the declared toolset (a likely hallucination). | R7 candidate (not hard) | R7+6 suppressed |
Each agent is run against injected faults; recovery is scored against a deterministic success oracle, with a 95% Wilson CI.
cancel-if-not-shipped-robustcancel-if-not-shipped-buggy