langsmith-evaluator
langsmith-evaluator is an LLM evaluation & observability skill for AI agents, published by langchain-ai in the skills-benchmarks repository on GitHub. Skillspace rates langsmith-evaluator F (15/100) and the safety scan rated it unsafe; the rating is based on static analysis only.
Unsafe. This skill has patterns associated with malicious or dangerous behaviour. We do not recommend running it.
INVOKE THIS SKILL when building evaluation pipelines for LangSmith. Covers three core components: (1) Creating Evaluators - LLM-as-Judge, custom code; (2) Defining Run Functions - how to capture outputs and trajectories from your agent; (3) Running Evaluations - locally with evaluate() or auto-run via LangSmith. Uses the langsmith CLI tool.
- Grade
- F
- Quality
- 15 / 100
- Safety
- Unsafe
- GitHub stars
- 109
Unproven: rated by static analysis only, not yet compared head-to-head.
Our two independent classifiers differ on this skill's category, so treat the placement as provisional.
This rating describes the version of the repository we crawled; it may have changed since.
Our review
- Architecture
- 96 / 100
- Maturity
- Basic: a single SKILL.md
- Security scan
- Critical-risk patterns
- License
- MIT
- Adoption
- Adopted by 0 other repositories, in 0 collections
What the scanner flagged
- pipe remote script to shell
- references sensitive credential env
- mentions secrets in text
Not checked yet
- Not judged on the rubric yet (not a contender in its category)
- No blind task eval yet
- Scope (portable / vendor / repo-internal) not assessed; only the top of each category is read for scope
- Sibling files (scripts, references) not fetched; security scan covers SKILL.md only
Skillspace indexes and rates this skill; it does not host or endorse it. Run third-party skills at your own risk.