Simplified traces, detections & alerts

Traces you don’t
have to read

Q: Do detections affect my application?

No. Detections are read-only labels applied to recorded spans. They help people and downstream analysis find relevant behavior without changing how the application runs.

Q: Can neatlogs catch failures that do not produce errors?

Yes. Classifiers and application-specific detections can flag behavior such as irrelevant retrieval, user frustration, missing actions, policy violations, and plausible but incorrect outcomes—even when the workflow completed without an exception.

Q: Do I need to replace my existing tracing platform?

No. You can instrument applications with neatlogs or import traces from supported platforms. This lets you adopt neatlogs for monitoring and investigations without replacing your existing stack first.

Q: What’s the difference between a detection and an alert?

A detection labels an event inside a trace. An alert rule determines when one event, or a pattern across events, deserves a notification.

Q: Can I create detections for my own application?

Yes. You can define detections around the workflows, spans, attributes, patterns, thresholds, and behaviors that matter to your product.

Q: How do detections relate to evals?

Detections surface events and behaviors continuously. Evals apply an explicit rubric to determine whether an agent’s behavior met a quality standard. A failed eval may become a signal worth monitoring; a detection may identify the cases that should be evaluated.

Q: What happens after a detection fires?

You can inspect the triggering span, filter for related runs, monitor its frequency, create an alert, or open the issue in Investigations to determine the cause and prepare a fix.

neatlogs cleans every run as it arrives and labels important events. You start with the handful of spans that explain the outcome instead of the hundreds that don’t. So does every agent, evaluator and teammate downstream.

Start free Read the docs

Production trace overloadNeatlogs finds the signal

checkout_agent.run286 spans · 24.8 secondsWORKFLOWcheckout_agent.run

Slack · #support47 messages: checkout failingAGENT · 3.6striage_agent

lookup_customerretry storm · 9 callsTOOL · 84μslookup_customer

gpt-4.1slow response · 374 tokensLLM · 3.6sgpt-4.1

knowledge_agent12 overlapping operationsAGENT · 7.5sknowledge_agent

POST /v1/checkout504 gateway timeoutERROR · 24.8sexecution failed

Slack · #supportCheckout is failing for customersPager · checkout-apip95 latency crossed 8 secondsEval · refund policyExpected refund, got escalationFAILEDAgent · resolutionTool call retried 9 timesretry 09/09

workflowagenttoolllmtoolerrorworkflowagenttoolllmtoolerror

workflowagentllmagenttooltoolerrorworkflowagentllmagenttooltoolerror

workflowtoolagenttoolllmagentworkflowtoolagenttoolllmagent

checkout_agent.run24.8s

👩🏽‍💻Engineering

👩🏻‍💼Product

👩🏾‍🎧Customer support

Everything happened at once.Only the evidence remains.

Traces

WorkflowDetectionsEvalsLatency＋ More filters

Aug 17Aug 19Aug 21Aug 24Aug 26Aug 28Aug 30

TraceStatusDetectionsEvaluationInput → output

checkout_agent.run12:57 PM · Aug 21Execution failed+6Needs reviewComplete checkout for cart…Gateway timeout

support_agent.run12:56:41 PM · Aug 17Refund misrouted+3PassedCustomer requested a refund…Refund approved

support_agent.run12:55:19 PM · Aug 17Repeated tool calls+4Needs reviewWhere is my shipment?Delivery estimate found

renewal_agent.run12:53:58 PM · Aug 17Escalation opened+2PassedRenew annual subscription…Renewal confirmed

triage_agent.run12:52:14 PM · Aug 17User frustration+3PassedAccount access issue…Routed to identity

21st Aug, 12:57 PM

support-automation-demo14

Input

Support ticket

Preset: refund_request
Customer: Maya Chen
Email: maya.chen@example.com
Order: NL-10482

Message

Hi Northline — I bought the Cascade Softshell Jacket two weeks ago and the zipper failed on my first overnight. I’d like a full refund to the original card. Thanks, Maya.

Customer attachment

images.unsplash.com

SHOW LESS

Detections⌄　14 STEPS · 24.8SEXPAND ALL

Policy mismatch
Order loaded
PII leaked · Email
Repeated tool calls
KB consulted
Execution failed
Gateway timeout

checkout_agent.runWORKFLOW24.8s
route_requestHTTP22.3s
triage_agentAGENT3.0s
lookup_customerTOOL84ms
search_knowledge_base × 9TOOL4.4s
gpt-4.1LLM　374 tok · $0.00493.4s
retrieve_recovery_policyTOOL420ms
decide_resolutionAGENT2.9s

Customer SupportCustomer Support+4

21st Aug, 12:57 PM

support-automation-demo14

Input

Support ticket

Preset: refund_request
Customer: Maya Chen
Email: maya.chen@example.com
Order: NL-10482

⌄　14 STEPS · 24.8SEXPAND ALL

checkout_agent.runWORKFLOW24.8s
route_requestHTTP22.3s
triage_agentAGENT3.0s
lookup_customerTOOL84ms
search_knowledge_base × 9TOOL4.4s
gpt-4.1LLM　374 tok · $0.00493.4s
retrieve_recovery_policyTOOL420ms
decide_resolutionAGENT2.9s

▪ GRAPH

northline_support_run

triage_agent

knowledge_agent

resolution_agent

lookup_customer

gpt-5.5-2026-04…

get_order

search_kb

render_policy_brief

gpt-5.5-2026-04…

case_media_board

create_refund

21st Aug, 12:57 PM

support-automation-demo14

Input

Support ticket

Preset: refund_request
Customer: Maya Chen
Email: maya.chen@example.com
Order: NL-10482

⌄　14 STEPS · 24.8SEXPAND ALL

checkout_agent.runWORKFLOW24.8s
route_requestHTTP22.3s
triage_agentAGENT3.0s
lookup_customerTOOL84ms
search_knowledge_base × 9TOOL4.4s
gpt-4.1LLM　374 tok · $0.00493.4s
retrieve_recovery_policyTOOL420ms
decide_resolutionAGENT2.9s

▪ GRAPH

northline_support_run

triage_agent

knowledge_agent

resolution_agent

lookup_customer

gpt-5.5-2026-04…

get_order

search_kb

render_policy_brief

gpt-5.5-2026-04…

case_media_board

create_refund

▪ TIMELINE

checkout_agent.run

route_request

triage_agent

lookup_customer

search_knowledge_base × 9

gpt-4.1

retrieve_recovery_policy

decide_resolution

Make traces useful to people and AI

Raw production traces are too noisy for people and too large for models. Sending the whole thing to a model burns tokens and weakens its reasoning.

neatlogs turns every trace into structured evidence before anyone touches it.

Deduplicate

A run is one complete agent execution, from the customer’s input to the final output. Group repeated and related runs before investigation.

Clean

A span is one operation inside a run—an LLM call, tool execution, or retrieval. Pull its useful inputs and outputs out of framework noise.

Label

Add human labels to runs and spans: correct or incorrect, quality ratings, what failed, and what a better outcome should be.

Simplify

neatlogs searches through labeled evidence to find the spans that affected an outcome. You get more reliable root cause analyses.

One investigation

47 related runs

refund_agent.runrefund request

support_agent.runbilling question

checkout_agent.runcheckout failure

250+ spans

lookup_policyretrieval

create_refundtool

gpt-5.5 responseLLM

19 labeled spans

Incorrect responseincorrect

Tool failurefailure

Better outcomesuggestion

6 relevant spans

lookup_policyevidence

create_refundevidence

gpt-5.5 responseevidence

Search all of
production
like one trace

Find a customer, tool call, error, workflow, output, or behavior across every run.

Ask in plain English, and neatlogs returns the answer with the evidence attached.

Ask in Slack

Neatlogs# agent-support•••

Mark9:41 AM

@neatlogs which northline checkout failures need my attention?

neatlogsAPP9:41 AM

12 Northline runs match. Four ended in payment errors after payments_api returned HTTP 503.

The affected runs share the same payment failure and retry path.

northline_checkout.run4 failures

payments_apiHTTP 503

refund_fallback3 retries

Ask in the app

Find high-latency checkout traces

Analysis

3 traces match. checkout_agent.run is the only run with significant latency.

Evidence

checkout_agent.run17.59s

payments_apiHTTP 503

create_refund11.8s

Ask anything…All workflows

Traces tell you what happened.
Detections tell you
where to look.

Automatic, read-only labels attached to the exact span where a condition appeared. Jump directly to the evidence.

neatlogs detects automatically

When you bring in a trace, neatlogs suggests detections for the patterns it finds. Accept, edit or ignore.

Traces SearchLast 14 days

All detections in this trace

PaymentsAPI Refund Outage 2PII leaked · EmailResolution Agent ErrorsOrder Loaded 2KB Consulted

Select a trace to see exactly where each signal appeared.

checkout_agent.run14 steps · 24.8s

Detection “Resolution Agent Errors” · 1 affected span

northline_support_run24.8s

triage_agent3.6s

lookup_customer84μs

gpt-5.5-2026-043.6s

resolution_agent6.4s

ERROR

PaymentsAPI 503 — refund service unavailable

create_refund573μs

Add your own detections

Start with exact conditions and regex. Add classifiers when a failure depends on semantic meaning or business-specific judgment.

Draft with AI

What do you want to detect?

Flag PII in agent outputs

Detection createdPII in Outputs

Active

Finds personal information in agent responses.

Type: PII classifier
Scope: All output spans
Sentiment: Negative

Send alerts in Slack or email

Notify the right people when a condition occurs. Set severity, aggregation, evaluation window and cooldown, with affected runs and evidence attached.

Slacknow

Neatlogs · #agent-alerts

Critical: PII detected in agent outputs

10 events · 5 min evaluation · 30 min cooldown

Mailnow

Neatlogs alert: PII detected

Critical severity · 10 events across 7 runs

5 min evaluation window · 30 min cooldown

Monitor detections

Track how often each signal appears, where it appears, whether it is rising, and how it overlaps with failed evals and known issues.

Detection

PII in Outputs

Personal information found in agent outputs.

Status: Active
Type: PII
Scope: All spans

41Flagged spans33PeakLast 3 months

JunJulAug

shipping_briefCustomer attachment included an email address

ToolPII in Outputs

resolution_agentResponse exposed a customer phone number

AgentPII in Outputs

get_orderOrder payload contained an IP address

ToolPII in Outputs

High PII rateFires when the threshold is reached

Severity: Critical
Aggregation: 10 matches
Evaluation: 5 minutes
Cooldown: 30 minutes

Name: PII in Outputs
Sentiment: Negative
Type: PII classifier
Scope: All output spans
Entities: Email, phone, credit card, IP, URL

Detections find. Evals judge.Investigations diagnose.

Each layer answers a different question about production behavior.

CapabilityThe question it answers

01Trace

What happened during this run?

02Detection

Where did a known condition or behavior appear?

03Investigation

Why did it happen, and what should change?

04Eval

Did the behavior meet an explicit standard?

Product questions and answers

The practical details about traces, detections, alerts, and bringing neatlogs into your current stack.

Selected question01 / 08

What’s the difference between a trace and a span?

A trace is the complete record of one agent task. A span is one operation inside that trace, such as a model call, retrieval, tool invocation, agent handoff, or deterministic function.

What’s the difference between a trace and a span?01

A trace is the complete record of one agent task. A span is one operation inside that trace, such as a model call, retrieval, tool invocation, agent handoff, or deterministic function.

Do detections affect my application?02

Can neatlogs catch failures that do not produce errors?03

Do I need to replace my existing tracing platform?04

What’s the difference between a detection and an alert?05

Can I create detections for my own application?06

How do detections relate to evals?07

What happens after a detection fires?08

Support ticket

Message

Customer attachment

Support ticket

Support ticket

Deduplicate

Clean

Label

Simplify

Evidence

neatlogs detects automatically

Add your own detections

What do you want to detect?

Send alerts in Slack or email

Neatlogs · #agent-alerts

Neatlogs alert: PII detected

Monitor detections

Bring the traces you already have

Product questions and answers

What’s the difference between a trace and a span?