ai-eval-failure-analysis

ai-eval-failure-analysis is an LLM evaluation & observability skill for AI agents, published by microsoft in the BCTech repository on GitHub. Skillspace rates ai-eval-failure-analysis F (45/100) and the safety scan rated it risky; the rating is based on static analysis only.

Analyze Business Central AI Test Toolkit (AIT) evaluation results to explain WHY an eval run failed. Use when the user has an AIT run (a version/suite, or a saved aitTestLogEntries JSON export) and wants failure analysis, a confusion matrix, precision/recall/F1, specificity, MatchRate, accuracy, sibling-confusion, model comparison, or a breakdown of failing rows (misses, spurious matches, wrong accounts, crashes). Triggers on phrases like "analyze my eval", "why did the AIT fail", "compare these model runs", "precision/recall for this run", "AI test failure analysis", "analyze the latest <suite> run".

Grade
F
Quality
45 / 100
Safety
Risky
GitHub stars
735

Unproven: rated by static analysis only, not yet compared head-to-head.

Our two independent classifiers differ on this skill's category, so treat the placement as provisional.

This rating describes the version of the repository we crawled; it may have changed since.

View source on GitHubmicrosoft/BCTech

Our review

Architecture
84 / 100
Maturity
Structured: ships references or scripts
Security scan
High-risk patterns
License
MIT
Adoption
Adopted by 0 other repositories, in 0 collections

What the scanner flagged

  • prompt-injection: ignore-previous
  • mentions secrets in text

Not checked yet

  • Not judged on the rubric yet (not a contender in its category)
  • No blind task eval yet
  • Scope (portable / vendor / repo-internal) not assessed; only the top of each category is read for scope
  • Sibling files (scripts, references) not fetched; security scan covers SKILL.md only

Better-rated skills in LLM evaluation & observability

Skillspace indexes and rates this skill; it does not host or endorse it. Run third-party skills at your own risk.