setup-benchmark-inputs
setup-benchmark-inputs is a Model training & evaluation skill for AI agents, published by mlc-ai in the pith-train repository on GitHub. Skillspace rates setup-benchmark-inputs C (74/100) and it passed the safety scan; the rating is based on static analysis only.
Set up the minimal set of artifacts (tokenized DCLM corpus shard + released HuggingFace checkpoint converted to DCP) required to benchmark, profile, or regression-test a MoE model in PithTrain. Use when the user asks to "prepare benchmark inputs", "set up the benchmark workspace", "download the DCLM shard", "fetch and convert the released checkpoint", "tokenize DCLM for DeepSeek/Qwen3", or when a downstream skill (capture-nsys-profile, validate-correctness, or any short canonical run) needs its workspace pre-populated. Produce `workspace/datasets/dclm-baseline/toktxt/<model>` and `workspace/checkpoints/<model>/torch-dcp/step-00000000`. Idempotent and safe to re-run.
- Grade
- C
- Quality
- 74 / 100
- Safety
- Passed
- GitHub stars
- 267
Unproven: rated by static analysis only, not yet compared head-to-head.
This rating describes the version of the repository we crawled; it may have changed since, so go by its current files.
Our review
- Architecture
- 100 / 100
- Maturity
- Structured: ships references or scripts
- Security scan
- No issues found
- License
- Apache-2.0
- Adoption
- Adopted by 0 other repositories, in 0 collections
Not checked yet
- Not judged on the rubric yet (not a contender in its category)
- No blind task eval yet
- Scope (portable / vendor / repo-internal) not assessed; only the top of each category is read for scope
- Sibling files (scripts, references) not fetched; security scan covers SKILL.md only
Skillspace indexes and rates this skill; it does not host or endorse it. Run third-party skills at your own risk.