quality-flywheel
quality-flywheel is an LLM evaluation & observability skill for AI agents, published by GoogleCloudPlatform in the vertex-ai-samples repository on GitHub. Skillspace rates quality-flywheel D (69/100) and it passed the safety scan; the rating is based on static analysis only.
Evaluate and improve GenAI models and agents using the Google GenAI Evaluation SDK. Creates eval datasets (from session traces or synthetic generation), selects and configures metrics (RubricMetric, LLMMetric, CodeExecutionMetric), executes evals via client.evals.evaluate(), and analyzes results to suggest concrete fixes. Supports both single-turn model evaluation and multi-turn agent trajectory evaluation. Use when asked to "evaluate my agent", "evaluate my model", "create eval dataset", "run evals", "analyze eval results", "which metrics should I use", "generate test data", or "improve quality".
- Grade
- D
- Quality
- 69 / 100
- Safety
- Passed
- GitHub stars
- 764
Unproven: rated by static analysis only, not yet compared head-to-head.
Our two independent classifiers differ on this skill's category, so treat the placement as provisional.
This rating describes the version of the repository we crawled; it may have changed since, so go by its current files.
Our review
- Architecture
- 78 / 100
- Maturity
- Structured: ships references or scripts
- Security scan
- No issues found
- License
- Apache-2.0
- Adoption
- Adopted by 1 other repositories, in 0 collections
Not checked yet
- Not judged on the rubric yet (not a contender in its category)
- No blind task eval yet
- Scope (portable / vendor / repo-internal) not assessed; only the top of each category is read for scope
- Sibling files (scripts, references) not fetched; security scan covers SKILL.md only
Skillspace indexes and rates this skill; it does not host or endorse it. Run third-party skills at your own risk.