benchmark-model

benchmark-model is a Model training & evaluation skill for AI agents, published by autogluon in the tabarena repository on GitHub. Skillspace rates benchmark-model B (80/100) and it passed the safety scan; the rating is based on static analysis only.

Scaffold the setup + eval scripts to run one model on the TabArena benchmark cluster. Use this skill whenever a maintainer wants to benchmark an already-integrated model — e.g. "benchmark TabM", "run Nori on the cluster", "create a setup/eval script for DenseLight", "I want to launch <model> on TabArena and evaluate it". Generates a single `tmp_scripts/run_<model>.py` with `setup` and `eval` subcommands that share one benchmark_name + paths (so they cannot drift), auto-filling GPU/CPU, eval subsets, the install reminder, foundation-model prefetch from the model's registry `info.py`, and — mandatory for GPU models — the VRAM budget (`fake_memory_for_estimates`; asks the user when the partition's VRAM is not inferable). Complements `add-model` (integrate a model) and `upload-method` (publish its results).

Grade

B

Quality

80 / 100

Safety

Passed

GitHub stars

271

Unproven: rated by static analysis only, not yet compared head-to-head.

Our two independent classifiers differ on this skill's category, so treat the placement as provisional.

This rating describes the version of the repository we crawled; it may have changed since, so go by its current files.

Skillspace indexes and rates this skill; it does not host or endorse it. Run third-party skills at your own risk.