diagnosing-experiment-results

diagnosing-experiment-results is an Experimentation & statistics skill for AI agents, published by PostHog in the ai-plugin repository on GitHub. Skillspace rates diagnosing-experiment-results D (68/100) and it passed the safety scan; the rating is based on static analysis only.

Diagnoses bias, anomalies, and strange-looking results on a specific PostHog experiment. Covers empty / 0-exposure experiments, sample ratio mismatch, identity fragmentation, multi-variant exposure, uneven-split exclusion bias, significance traps (peeking, A/A, Bayesian vs Frequentist), PostHog-vs-SQL discrepancies, and surprises after mid-run edits. Symptom-driven dispatch to the right diagnostic. TRIGGER when: user asks 'is my experiment biased?' or 'why 0 exposures?', references the bias banner, says a variant looks strange / wrong / off, sees significance flipping, notices PostHog numbers disagreeing with their SQL, sees an A/A test showing significance, or reports surprises after mid-run edits. DO NOT TRIGGER when: creating a new experiment (use creating-experiments), only configuring rollout (use configuring-experiment-rollout) or metrics (use configuring-experiment-analytics), or only asking lifecycle questions (use managing-experiment-lifecycle).

Grade

D

Quality

68 / 100

Safety

Passed

GitHub stars

59

Unproven: rated by static analysis only, not yet compared head-to-head.

This rating describes the version of the repository we crawled; it may have changed since, so go by its current files.

Skillspace indexes and rates this skill; it does not host or endorse it. Run third-party skills at your own risk.