9.0 KiB
Universal Scanner Output Schema
All quality scanners — both LLM-based and deterministic lint scripts — MUST produce output conforming to this schema. No exceptions.
Top-Level Structure
{
"scanner": "scanner-name",
"skill_path": "{path}",
"findings": [],
"assessments": {},
"summary": {
"total_findings": 0,
"by_severity": {},
"assessment": "1-2 sentence overall assessment"
}
}
| Key | Type | Required | Description |
|---|---|---|---|
scanner |
string | yes | Scanner identifier (e.g., "workflow-integrity", "prompt-craft") |
skill_path |
string | yes | Absolute path to the skill being scanned |
findings |
array | yes | ALL items — issues, strengths, suggestions, opportunities. Always an array, never an object |
assessments |
object | yes | Scanner-specific structured analysis (cohesion tables, health metrics, user journeys, etc.). Free-form per scanner |
summary |
object | yes | Aggregate counts and brief overall assessment |
Finding Schema (7 fields)
Every item in findings[] has exactly these 7 fields:
{
"file": "SKILL.md",
"line": 42,
"severity": "high",
"category": "frontmatter",
"title": "Brief headline of the finding",
"detail": "Full context — rationale, what was observed, why it matters",
"action": "What to do about it — fix, suggestion, or script to create"
}
| Field | Type | Required | Description |
|---|---|---|---|
file |
string | yes | Relative path to the affected file (e.g., "SKILL.md", "scripts/build.py"). Empty string if not file-specific |
line |
int|null | no | Line number (1-based). null or 0 if not line-specific |
severity |
string | yes | One of the severity values below |
category |
string | yes | Scanner-specific category (e.g., "frontmatter", "token-waste", "lint") |
title |
string | yes | Brief headline (1 sentence). This is the primary display text |
detail |
string | yes | Full context — fold rationale, observation, impact, nuance into one narrative. Empty string if title is self-explanatory |
action |
string | yes | What to do — fix instruction, suggestion, or script to create. Empty string for strengths/notes |
Severity Values (complete enum)
critical | high | medium | low | high-opportunity | medium-opportunity | low-opportunity | suggestion | strength | note
Routing rules:
critical,high→ "Truly Broken" section in reportmedium,low→ category-specific findings sectionshigh-opportunity,medium-opportunity,low-opportunity→ enhancement/creative sectionssuggestion→ creative suggestions sectionstrength→ strengths section (positive observations worth preserving)note→ informational observations, also routed to strengths
Assessment Sub-Structure Contracts
The assessments object is free-form per scanner, but the HTML report renderer expects specific shapes for specific keys. These are the canonical formats.
user_journeys (enhancement-opportunities scanner)
Always an array of objects. Never an object keyed by persona.
"user_journeys": [
{
"archetype": "first-timer",
"summary": "Brief narrative of this user's experience",
"friction_points": ["moment 1", "moment 2"],
"bright_spots": ["what works well"]
}
]
autonomous_assessment (enhancement-opportunities scanner)
"autonomous_assessment": {
"potential": "headless-ready|easily-adaptable|partially-adaptable|fundamentally-interactive",
"hitl_points": 3,
"auto_resolvable": 2,
"needs_input": 1,
"notes": "Brief assessment"
}
top_insights (enhancement-opportunities scanner)
Always an array of objects with title/detail/action (same shape as findings but without file/line/severity/category).
"top_insights": [
{
"title": "The key observation",
"detail": "Why it matters",
"action": "What to do about it"
}
]
cohesion_analysis (skill-cohesion / agent-cohesion scanner)
"cohesion_analysis": {
"dimension_name": { "score": "strong|moderate|weak", "notes": "explanation" }
}
Dimension names are scanner-specific (e.g., stage_flow_coherence, persona_alignment). The report renderer iterates all keys and renders a table row per dimension.
skill_identity / agent_identity (cohesion scanners)
"skill_identity": {
"name": "skill-name",
"purpose_summary": "Brief characterization",
"primary_outcome": "What this skill produces"
}
skillmd_assessment (prompt-craft scanner)
"skillmd_assessment": {
"overview_quality": "appropriate|excessive|missing",
"progressive_disclosure": "good|needs-extraction|monolithic",
"notes": "brief assessment"
}
Agent variant adds "persona_context": "appropriate|excessive|missing".
prompt_health (prompt-craft scanner)
"prompt_health": {
"total_prompts": 3,
"with_config_header": 2,
"with_progression": 1,
"self_contained": 3
}
skill_understanding (enhancement-opportunities scanner)
"skill_understanding": {
"purpose": "what this skill does",
"primary_user": "who it's for",
"assumptions": ["assumption 1", "assumption 2"]
}
stage_summary (workflow-integrity scanner)
"stage_summary": {
"total_stages": 0,
"missing_stages": [],
"orphaned_stages": [],
"stages_without_progression": [],
"stages_without_config_header": []
}
metadata (structure scanner)
Free-form key-value pairs. Rendered as a metadata block.
script_summary (scripts lint)
"script_summary": {
"total_scripts": 5,
"by_type": {"python": 3, "shell": 2},
"missing_tests": ["script1.py"]
}
existing_scripts (script-opportunities scanner)
Array of strings (script paths that already exist).
Complete Example
{
"scanner": "workflow-integrity",
"skill_path": "/path/to/skill",
"findings": [
{
"file": "SKILL.md",
"line": 12,
"severity": "high",
"category": "frontmatter",
"title": "Missing required 'version' field in frontmatter",
"detail": "The SKILL.md frontmatter is missing the version field. This prevents the manifest generator from producing correct output and breaks version-aware consumers.",
"action": "Add 'version: 1.0.0' to the YAML frontmatter block"
},
{
"file": "build-process.md",
"line": null,
"severity": "strength",
"category": "design",
"title": "Excellent progressive disclosure pattern in build stages",
"detail": "Each stage provides exactly the context needed without front-loading information. This reduces token waste and improves LLM comprehension.",
"action": ""
},
{
"file": "SKILL.md",
"line": 45,
"severity": "medium-opportunity",
"category": "experience-gap",
"title": "No guidance for first-time users unfamiliar with build workflows",
"detail": "A user encountering this skill for the first time has no onboarding path. The skill assumes familiarity with stage-based workflows, which creates friction for newcomers.",
"action": "Add a 'Getting Started' section or link to onboarding documentation"
}
],
"assessments": {
"stage_summary": {
"total_stages": 7,
"missing_stages": [],
"orphaned_stages": ["cleanup"]
}
},
"summary": {
"total_findings": 3,
"by_severity": {"high": 1, "medium-opportunity": 1, "strength": 1},
"assessment": "Well-structured skill with one critical frontmatter gap. Progressive disclosure is a notable strength."
}
}
DO NOT
- DO NOT rename fields. Use exactly:
file,line,severity,category,title,detail,action - DO NOT use
issuesinstead offindings— the array is always calledfindings - DO NOT add fields to findings beyond the 7 defined above. Put scanner-specific structured data in
assessments - DO NOT use separate arrays for strengths, suggestions, or opportunities — they go in
findingswith appropriate severity values - DO NOT change
user_journeysfrom an array to an object keyed by persona name - DO NOT restructure assessment sub-objects — use the shapes defined above
- DO NOT put free-form narrative data into
assessments— that belongs indetailfields of findings or insummary.assessment
Self-Check Before Output
Before writing your JSON output, verify:
- Is your array called
findings(notissues, notopportunities)? - Does every item in
findingshave all 7 fields:file,line,severity,category,title,detail,action? - Are strengths in
findingswithseverity: "strength"(not in a separatestrengthsarray)? - Are suggestions in
findingswithseverity: "suggestion"(not in a separatecreative_suggestionsarray)? - Is
assessmentsan object containing structured analysis data (not items that belong in findings)? - Is
user_journeysan array of objects (not an object keyed by persona)? - Do
top_insightsitems usetitle/detail/action(notinsight/suggestion/why_it_matters)?