writing-evals

writing-evals is an LLM evaluation & observability skill for AI agents, published by PostHog in the posthog repository on GitHub. Skillspace rates writing-evals F (45/100) and the safety scan rated it risky; the rating is based on static analysis only.

Risky. This skill has patterns associated with unsafe behaviour. Read the source carefully before running it.

Teaches how to write and run evals on the `products/posthog_ai/eval_harness/` harness — sandboxed agent suites that execute the real coding agent in a Docker or Modal sandbox against a seeded Hedgebox project, and one-shot suites that score a single in-process model invocation per case. Use when adding or changing eval suites, cases, scorers, seeders, or synthesizers under `products/posthog_ai/evals/` or `products/*/evals/`, when touching the harness under `products/posthog_ai/eval_harness/`, or when running or debugging those evals (`hogli evals`). Covers suite kinds and discovery, case anatomy, the seeder/synthesizer split, the one-branch scorer patterns, and how to read results. Not for `ee/hogai/eval/ci/` pytest evals, and not for the LLM Analytics product's evaluation features.

Grade

F

Quality

45 / 100

Safety

Risky

GitHub stars

35,560

Unproven: rated by static analysis only, not yet compared head-to-head.

Our two independent classifiers differ on this skill's category, so treat the placement as provisional.

This rating describes the version of the repository we crawled; it may have changed since.

Skillspace indexes and rates this skill; it does not host or endorse it. Run third-party skills at your own risk.