auto-arena
auto-arena is an LLM evaluation & observability skill for AI agents, published by agentscope-ai in the OpenJudge repository on GitHub. Skillspace rates auto-arena F (45/100) and the safety scan rated it risky; the rating is based on static analysis only.
Risky. This skill has patterns associated with unsafe behaviour. Read the source carefully before running it.
Automatically evaluate and compare multiple AI models or agents without pre-existing test data. Generates test queries from a task description, collects responses from all target endpoints, auto-generates evaluation rubrics, runs pairwise comparisons via a judge model, and produces win-rate rankings with reports and charts. Supports checkpoint resume, incremental endpoint addition, and judge model hot-swap. Use when the user asks to compare, benchmark, or rank multiple models or agents on a custom task, or run an arena-style evaluation.
- Grade
- F
- Quality
- 45 / 100
- Safety
- Risky
- GitHub stars
- 732
Unproven: rated by static analysis only, not yet compared head-to-head.
Our two independent classifiers differ on this skill's category, so treat the placement as provisional.
This rating describes the version of the repository we crawled; it may have changed since.
Our review
- Architecture
- 79 / 100
- Maturity
- Basic: a single SKILL.md
- Security scan
- High-risk patterns
- License
- Apache-2.0
- Adoption
- Adopted by 0 other repositories, in 2 collections
What the scanner flagged
- references sensitive credential env
- mentions secrets in text
Not checked yet
- Not judged on the rubric yet (not a contender in its category)
- No blind task eval yet
- Scope (portable / vendor / repo-internal) not assessed; only the top of each category is read for scope
- Sibling files (scripts, references) not fetched; security scan covers SKILL.md only
Skillspace indexes and rates this skill; it does not host or endorse it. Run third-party skills at your own risk.