# neatlogs

> Neatlogs is a collaborative debugging workspace for AI agent teams. It is the shared context layer where engineers and domain experts trace agent failures, investigate root causes with an AI assistant, and ship fixes together — all in one place. Setup takes a few lines of Python or TypeScript, and works with every major agent framework.

## What Neatlogs Does

Neatlogs replaces the fragmented workflow of debugging AI agents across logs, Slack threads, and spreadsheets. When an agent fails in production, your team opens the trace in Neatlogs, sees exactly what happened step by step (every LLM call, tool use, retrieval, and decision), discusses the failure directly on the span that broke, and ships an AI-suggested fix — without switching tools or waiting for an engineer to translate what happened.

The core insight: most observability tools tell engineers *what* happened. Neatlogs tells the whole team — engineers, PMs, domain experts — and then helps everyone decide *what to do about it*.

## Who It's For

- **AI/ML Engineers** building agents with CrewAI, LangGraph, LangChain, the OpenAI Agents SDK, Pydantic AI, Agno, DSPy, Google ADK, Strands, or the Claude Agent SDK
- **Product Managers** who need to understand why an agent failed without reading raw logs
- **Domain Experts** (support leads, content reviewers, compliance) who evaluate agent output quality
- **Teams shipping AI agents to production** and maintaining them over time

## Key Features

- **Traces** — The full execution timeline of any agent run. Every LLM call, tool invocation, retrieval step, and decision branch rendered as a structured span tree. Inspect inputs, outputs, metadata, and cost at each step.
- **Comments** — Team collaboration directly on traces. Comment on any span, @mention teammates, create threads, and resolve discussions without leaving the run.
- **Investigate** — Point Neat AI at any failing trace. It runs a multi-step investigation across your runs, links the evidence it finds, and turns real problems into fixes. Threaded and stateful, so follow-ups drill in without restating context.
- **Fixes** — AI-generated fix suggestions from failing and flagged traces, with root-cause reasoning. Review a suggestion, then dispatch it to a coding agent that opens a pull request — without leaving the dashboard.
- **Detections** — Automatic monitoring for quality regressions, latency spikes, cost overruns, and unusual agent behavior. Catch issues before users notice.
- **Session Replay** — Play back an agent session step by step — its structure, timeline, and decisions — as a guided replay.
- **Analytics** — Build dashboards tracking the signals that matter: failure rate, latency percentiles, cost per run, token usage, and quality scores.
- **Experiments** — A/B test prompt versions with real traffic. Split runs between variants, measure quality/cost/latency differences, and promote the winner.
- **Human Evals** — Define what "good" looks like with custom rubrics. Assign traces to human reviewers, manage deadlines, and aggregate results to track quality over time.
- **Alerts** — Configurable alerting on any metric or pattern: thresholds, anomalies, or custom conditions, delivered to Slack and more.
- **Integrations** — Connect your whole workflow: GitHub for dispatching fixes as pull requests, Slack for alerts, plus the MCP server so your coding agent can read investigations and act on fixes from your editor.

## Supported Frameworks & Providers

Instrumentation is automatic — `neatlogs.init()` captures supported libraries with no per-call changes.

### LLM Providers (auto-instrumented)
- OpenAI
- Anthropic (Claude)
- Google Gemini (google-genai)
- Azure OpenAI
- Amazon Bedrock
- Google Vertex AI
- OpenRouter

### Agent Frameworks (auto-instrumented)
- CrewAI — multi-agent orchestration with delegation and role-based agents
- LangChain — chains, agents, and retrieval pipelines
- LangGraph — stateful, cyclic agent graphs
- OpenAI Agents SDK — tool-calling agents with the official OpenAI SDK
- Pydantic AI — type-safe agents with structured outputs
- Agno — lightweight agent framework
- DSPy — programmatic prompt optimization
- Google ADK — the Google Agent Development Kit
- Strands — AWS's agent framework
- Claude Agent SDK — Anthropic's agent SDK (AGENT/LLM/TOOL spans from streamed messages)

### Coding Agents & Tools
- Claude Code
- Cursor (dispatch fixes that open pull requests)
- opencode
- Hermes (zero-code CLI tracing)
- MCP server — expose traces, detections, and fixes to any MCP-compatible client

### Team Tool Integrations
- GitHub — dispatch fixes as pull requests; link traces to PRs
- Slack — alerts, notifications, and investigation threads

## MCP Interface (for AI agents)

Neatlogs exposes a Model Context Protocol (MCP) server so AI agents and IDEs can query traces, read detections, and manage fixes directly.

- **Endpoint:** `https://ingest.neatlogs.com/mcp` (the `/mcp` path on the same host as your SDK endpoint; use your own host if you self-host)
- **Transport:** Streamable HTTP
- **Authentication:** project API key as a bearer token — `Authorization: Bearer YOUR_PROJECT_KEY`
- **Setup:** `npx @neatlogs/wizard mcp --api-key YOUR_PROJECT_KEY` wires up your client config automatically
- **Reference:** https://docs.neatlogs.com/integrations/mcp-tools

### MCP Tools

- **Connection:** `ping`, `whoami` — verify connectivity and authentication
- **Observability:** `search_traces`, `get_trace_context`, `list_detections`, `get_detection_trend` — query production data and trends
- **Logging:** `log_trace` — push traces from agents into Neatlogs
- **Fixes:** `triage_list`, `triage_get`, `triage_accept`, `triage_dismiss`, `triage_update_status`, `triage_update`, `triage_get_investigation`, `triage_list_steps` — read and manage code fixes

### Span Kinds

Traces are built from typed spans: LLM, TOOL, AGENT, CHAIN, WORKFLOW, TASK, RETRIEVER, RERANKER, EMBEDDING, EVALUATOR, GUARDRAIL, MCP, HTTP, API, LOG. See https://docs.neatlogs.com/sdk/span-kinds.

### Detections

Detections flag problem spans automatically. Mechanisms: **Regex** (pattern-match span text), **Condition** (numeric/boolean attribute rules, e.g. `latency_ms > 5000`, `cost_usd > 0.10`), **PII detection**, and **Classifier** (model-based). See https://docs.neatlogs.com/docs/features/detections.

### MCP Limits

- Rate limit: 60 requests/minute (per API key, all sessions combined)
- Max request payload: 1 MB · Max response payload: 10 MB
- Search results per request: 100 (paginate for more)
- Trace context max depth: 1000 spans
- Session timeout: 1 hour of inactivity

## Documentation

- Introduction: https://docs.neatlogs.com/docs/quickstart/introduction — Start here. Overview of Neatlogs, what it solves, and how the platform is organized.
- Instrument with the Wizard: https://docs.neatlogs.com/docs/quickstart/wizard — Interactive setup. Install the SDK, configure your project, connect your framework, and send your first trace.
- Explore the Dashboard: https://docs.neatlogs.com/docs/quickstart/explore-dashboard — Guided walkthrough after your first traces land.
- Import Traces: https://docs.neatlogs.com/docs/quickstart/import — Migrate from Langfuse, LangSmith, or OpenTelemetry exports.
- Traces: https://docs.neatlogs.com/docs/features/traces — Anatomy of a span, timeline navigation, input/output inspection, cost breakdown, and multi-agent execution trees.
- Comments: https://docs.neatlogs.com/docs/features/comments — Comment on any span, @mention teammates, create threads, and resolve discussions.
- Detections: https://docs.neatlogs.com/docs/features/detections — Detection types, configuration, severity levels, and suppression rules.
- Investigate: https://docs.neatlogs.com/docs/features/ai-search — Multi-step AI investigation over your trace data that turns findings into fixes.
- Session Replay: https://docs.neatlogs.com/docs/features/session-replay — Step through an agent session as a guided replay.
- Analytics: https://docs.neatlogs.com/docs/features/analytics — Dashboard builder: chart types, metrics, time ranges, grouping, and filtering.
- Fixes: https://docs.neatlogs.com/docs/features/triage — Review AI-generated fix suggestions, then dispatch them to a coding agent that opens a pull request.
- Experiments: https://docs.neatlogs.com/docs/features/experiments — Controlled prompt experiments: traffic splitting, metric comparison, and promoting winners.
- Human Evals: https://docs.neatlogs.com/docs/features/human-evals — Eval forms, reviewer assignment, rubric design, deadlines, and aggregated results.
- Alerts: https://docs.neatlogs.com/docs/features/alerts — Threshold-, anomaly-, and pattern-based alerts, plus delivery channels.
- Integrations: https://docs.neatlogs.com/docs/features/integrations — Connect Neatlogs to GitHub, Slack, and more.
- FAQ: https://docs.neatlogs.com/docs/faq — Setup, data retention, security, privacy, supported frameworks, and account management.
- Guides: https://docs.neatlogs.com/guides — In-depth how-to guides: multi-agent (LangGraph), multi-framework, RAG pipelines, long-running sessions, chatbot sessions, custom orchestration, and MCP integration.
- SDK Reference: https://docs.neatlogs.com/sdk — Complete SDK reference: Python, TypeScript, Go, decorators, span kinds, sessions, tags, prompt templates, PII redaction, and more.
- Integrations Directory: https://docs.neatlogs.com/integrations — Full catalog of supported frameworks and tools with per-integration quickstarts.

## SDK

- Python package: `pip install neatlogs`
- PyPI: https://pypi.org/project/neatlogs
- GitHub: https://github.com/neatlogs/neatlogs
- License: MIT

### Minimal Setup

```python
import neatlogs
neatlogs.init()
# That's it. Supported LLM and framework calls are now traced automatically.
```

### With Decorators

```python
import neatlogs
neatlogs.init()

@neatlogs.trace
def my_agent_workflow(query: str):
    # Your agent logic here.
    # Every LLM call, tool use, and retrieval step is captured automatically.
    pass
```

## Links

- Website: https://neatlogs.com
- App: https://app.neatlogs.com
- Documentation: https://docs.neatlogs.com
- GitHub: https://github.com/neatlogs/neatlogs
- PyPI: https://pypi.org/project/neatlogs
- Book a Demo: https://neatlogs.com/book-demo
- Changelog: https://neatlogs.com/changelog
- Agent Observability Glossary: https://neatlogs.com/glossary — definitions of the core agent-observability vocabulary (traces, spans, evals, metrics, guardrails, human-in-the-loop)
- LinkedIn: https://www.linkedin.com/company/neatlogs
- X/Twitter: https://x.com/neatlogs

## Use Cases

1. **Debug agent failures in production** — An agent hallucinates or loops in production. Open the trace, see the exact span where it went wrong, investigate the root cause with Neat AI, and dispatch a fix as a pull request. From alert to fix in minutes, not hours.

2. **Collaborate across roles on agent quality** — A PM notices the support agent giving wrong answers, comments on the span with the bad output, and tags the engineer — who sees the full context immediately, with no back-and-forth.

3. **Catch prompt regressions before users do** — You ship a prompt update; Detections flag a quality drop. You compare versions in an Experiment and promote the better one.

4. **Onboard new team members to agent debugging** — Instead of a stale architecture doc, point new engineers at recent traces and the team's comments on past failures. They learn the system from real execution data.

5. **Evaluate agent output at scale** — Create a Human Eval form, assign traces to reviewers, set a deadline, and let Neatlogs aggregate scores to show where quality is weakest.