agent-observability-eval-pipeline
agent-observability-eval-pipeline is an LLM evaluation & observability skill for AI agents, published by datadog-labs in the agent-skills repository on GitHub. Skillspace rates agent-observability-eval-pipeline F (45/100) and the safety scan rated it risky; the rating is based on static analysis only.
Risky. This skill has patterns associated with unsafe behaviour. Read the source carefully before running it.
End-to-end Agent Observability pipeline for an instrumented ml_app — classify production traces, root-cause failures, bootstrap evaluators, then (optionally) sample + publish a dataset, generate + run an experiment, and analyze results. Six narrated phases with a standardized banner and a "continue" checkpoint between each. Pure orchestration over the agent-observability sub-skills (`agent-observability-session-classify`, `agent-observability-trace-rca`, `agent-observability-eval-bootstrap`, `agent-observability-experiment-py-bootstrap`, `agent-observability-experiment-analyzer`). Use when user says "run the eval pipeline", "go from traces to evals", "bootstrap evals end to end", "classify then RCA then bootstrap", "build an eval set from scratch", "onboard me to datasets and experiments", "walk me through experiments", "I have an ml_app, now what", "Agent Observability onboarding", "guided experiment setup", "from traces to experiments", or wants a deterministic, narrated tour from…
- Grade
- F
- Quality
- 45 / 100
- Safety
- Risky
- GitHub stars
- 142
Unproven: rated by static analysis only, not yet compared head-to-head.
Our two independent classifiers differ on this skill's category, so treat the placement as provisional.
This rating describes the version of the repository we crawled; it may have changed since.
Our review
- Architecture
- 70 / 100
- Maturity
- Structured: ships references or scripts
- Security scan
- High-risk patterns
- License
- MIT
- Adoption
- Adopted by 0 other repositories, in 1 collections
What the scanner flagged
- references sensitive credential env
- mentions secrets in text
Not checked yet
- Not judged on the rubric yet (not a contender in its category)
- No blind task eval yet
- Scope (portable / vendor / repo-internal) not assessed; only the top of each category is read for scope
- Sibling files (scripts, references) not fetched; security scan covers SKILL.md only
Skillspace indexes and rates this skill; it does not host or endorse it. Run third-party skills at your own risk.