annotator-input-parity-check

annotator-input-parity-check is a Fine-tuning & training data skill for AI agents, published by kennethkhoocy in the applied-micro-skills repository on GitHub. Skillspace rates annotator-input-parity-check D (66/100) and it passed the safety scan; the rating is based on static analysis only.

Before designing, training, or auditing ANY model that replicates human-annotated labels, audit the annotation protocol's INPUT — the exact document/evidence the human labelers consulted — and give the model that same input. Use when: (1) designing a classifier/LLM extractor whose target is a hand-coded label set, (2) a label-replication model shows low recall concentrated in a label subset and the diagnosis on offer is "the label's information is not in the features", (3) reviewers propose construct splits (e.g. "designation vs record-evident"), adjudication sittings, or per-domain stop rules to explain residual disagreement with gold, (4) validating an extraction pipeline against labels transcribed from a source document. Symptom of the underlying failure: elaborate theory accumulates to explain why gold is "partially unpredictable" when the model was simply never shown the document the annotators read.

Grade

D

Quality

66 / 100

Safety

Passed

GitHub stars

28

Unproven: rated by static analysis only, not yet compared head-to-head.

Our two independent classifiers differ on this skill's category, so treat the placement as provisional.

This rating describes the version of the repository we crawled; it may have changed since, so go by its current files.

Skillspace indexes and rates this skill; it does not host or endorse it. Run third-party skills at your own risk.