harbor-f2p-p2p-deep-dive

harbor-f2p-p2p-deep-dive is an LLM evaluation & observability skill for AI agents, published by NVIDIA-NeMo in the Switchyard repository on GitHub. Skillspace rates harbor-f2p-p2p-deep-dive D (62/100) and it passed the safety scan; the rating is based on static analysis only.

Deep dive on Harbor trial results for tasks that use SWE-Bench-style F2P (FAIL_TO_PASS) and P2P (PASS_TO_PASS) reference tests. Diagnoses why an agent failed and audits whether a failing task is genuinely hard or unfair (instruction-vs-verifier mismatch). Use when the user asks to analyze, debug, investigate, or deep dive on a Harbor run, trial, or job directory; when the user says "harbor deep dive", "f2p deep dive", or "p2p deep dive"; when a trial scores lower than expected; or when the user wants to assess task fairness, instruction quality, verifier correctness, or whether an F2P or P2P test is reasonable for the instruction given. Works with any Harbor task whose verifier reports F2P/P2P-style results, and with any supported agent (claude-code, opencode, codex). Produces a per-failure root-cause verdict (capability gap / verifier issue / instruction issue) by triangulating verifier output, the agent's implementation, the reference tests, and the instruction.

Grade
D
Quality
62 / 100
Safety
Passed
GitHub stars
3,008

Unproven: rated by static analysis only, not yet compared head-to-head.

Our two independent classifiers differ on this skill's category, so treat the placement as provisional.

This rating describes the version of the repository we crawled; it may have changed since, so go by its current files.

View source on GitHubNVIDIA-NeMo/Switchyard

Our review

Architecture
96 / 100
Maturity
Basic: a single SKILL.md
Security scan
Low-risk patterns
License
Not declared. Check the repository before reusing it.

What the scanner flagged

  • mentions secrets in text

Not checked yet

  • Category is a classifier prediction; no model has read this skill yet
  • Not judged on the rubric yet (not a contender in its category)
  • No blind task eval yet
  • Scope (portable / vendor / repo-internal) not assessed; only the top of each category is read for scope
  • Sibling files (scripts, references) not fetched; security scan covers SKILL.md only

Better-rated skills in LLM evaluation & observability

Skillspace indexes and rates this skill; it does not host or endorse it. Run third-party skills at your own risk.