quark-torch-llm-eval
quark-torch-llm-eval is an LLM evaluation & observability skill for AI agents, published by amd in the Quark repository on GitHub. Skillspace rates quark-torch-llm-eval D (62/100) and it passed the safety scan; the rating is based on static analysis only.
End-to-end LLM accuracy evaluation on AMD ROCm (ROCm-only) — container setup, vLLM/SGLang/ATOM serving, lm-eval / lighteval / evalscope benchmarks. Use when the user wants to evaluate, benchmark, or compare an LLM's accuracy. Trigger for "evaluate this model", "run gsm8k/mmlu/mmlu_pro/aime/gpqa/hellaswag/arc", "test accuracy", "measure perplexity", "compare quantized model accuracy", "does this mxfp4 model lose accuracy". For evaluating Quark Agent Skills themselves, use quark-torch-eval-runner instead.
- Grade
- D
- Quality
- 62 / 100
- Safety
- Passed
- GitHub stars
- 148
Unproven: rated by static analysis only, not yet compared head-to-head.
This rating describes the version of the repository we crawled; it may have changed since, so go by its current files.
Our review
- Architecture
- 84 / 100
- Maturity
- Structured: ships references or scripts
- Security scan
- Low-risk patterns
- License
- MIT
- Adoption
- Adopted by 1 other repositories, in 0 collections
What the scanner flagged
- mentions secrets in text
Not checked yet
- Not judged on the rubric yet (not a contender in its category)
- No blind task eval yet
- Scope (portable / vendor / repo-internal) not assessed; only the top of each category is read for scope
- Sibling files (scripts, references) not fetched; security scan covers SKILL.md only
Skillspace indexes and rates this skill; it does not host or endorse it. Run third-party skills at your own risk.