skill-validator

Skill Validator

Validate any skill against production-level quality criteria.

Validation Workflow

Phase 1: Gather Context

1. Read the skill's SKILL.md completely
2. Identify skill type from frontmatter description:
   • Builder skill (creates artifacts)
   • Guide skill (provides instructions)
   • Automation skill (executes workflows)
   • Analyzer skill (extracts insights)
   • Validator skill (enforces quality)
   • Hybrid skill (combination of above)
3. Read all reference files in references/ directory
4. Check for assets/scripts directories
5. Note frontmatter fields (name, description, allowed-tools, model)

Phase 2: Apply Criteria

Evaluate against 9 criteria categories. Each criterion scores 0-3:

• 0: Missing/Absent
• 1: Present but inadequate
• 2: Adequate implementation
• 3: Excellent implementation

Criteria Categories

1. Structure & Anatomy (Weight: 12%)

Fail condition: Missing SKILL.md or >800 lines = automatic fail

2. Content Quality (Weight: 15%)

3. User Interaction (Weight: 12%)

Key pattern to look for:

## Required Clarifications
1. Question about X
2. Question about Y

## Optional Clarifications
3. Question about Z (if relevant)

Note: Avoid asking too many questions in a single message.

4. Documentation & References (Weight: 10%)

Key pattern to look for:

| Resource | URL | Use For |
|----------|-----|---------|
| Official Docs | https://... | Complex cases |

5. Domain Standards (Weight: 10%)

Key pattern to look for:

### Must Follow
- [ ] Requirement 1
- [ ] Requirement 2

### Must Avoid
- Antipattern 1
- Antipattern 2

6. Technical Robustness (Weight: 8%)

7. Maintainability (Weight: 8%)

8. Zero-Shot Implementation (Weight: 12%)

Skills should enable single-interaction implementation with embedded expertise.

Key pattern to look for:

## Before Implementation

Gather context to ensure successful implementation:

| Source | Gather |
|--------|--------|
| **Codebase** | Existing structure, patterns, conventions |
| **Conversation** | User's specific requirements |
| **Skill References** | Domain patterns from `references/` |
| **User Guidelines** | Project-specific conventions |

Red flag: Skill instructs to "research" or "discover" domain knowledge at runtime instead of embedding it.

9. Reusability (Weight: 13%)

Skills should handle variations, not single requirements.

Good example:

"Create visualizations - adaptable to data shape, chart type, library"

Bad example (too specific):

"Create bar chart with sales data using Recharts"

Key check: Does the skill work for multiple use cases within its domain?

Type-Specific Validation

After scoring general criteria, verify type-specific requirements:

Scoring: Deduct 10 points if type-specific requirements missing for identified type.

Scoring Guide

Category Scores

Calculate each category score:

Category Score = (Sum of criterion scores) / (Max possible) * 100

Overall Score

Overall = Σ(Category Score × Weight)

Rating Thresholds

Output Format

Generate validation report:

# Skill Validation Report: [skill-name]

**Rating**: [Production/Good/Adequate/Developing/Incomplete]
**Overall Score**: [X]/100

## Summary
[2-3 sentence assessment]

## Category Scores

| Category | Score | Weight | Weighted |
|----------|-------|--------|----------|
| Structure & Anatomy | X/100 | 12% | X |
| Content Quality | X/100 | 15% | X |
| User Interaction | X/100 | 12% | X |
| Documentation | X/100 | 10% | X |
| Domain Standards | X/100 | 10% | X |
| Technical Robustness | X/100 | 8% | X |
| Maintainability | X/100 | 8% | X |
| Zero-Shot Implementation | X/100 | 12% | X |
| Reusability | X/100 | 13% | X |
| **Type-Specific Deduction** | -X | - | -X |

## Critical Issues (if any)
- [Issue requiring immediate fix]

## Improvement Recommendations
1. **High Priority**: [Specific action]
2. **Medium Priority**: [Specific action]
3. **Low Priority**: [Specific action]

## Strengths
- [What skill does well]

Quick Validation Checklist

For rapid assessment, check these critical items:

Structure & Frontmatter

• SKILL.md <500 lines
• Frontmatter: name (≤64 chars, lowercase, hyphens) + description (≤1024 chars)
• Description uses third-person style ("This skill should be used when...")
• No README.md/CHANGELOG.md in skill directory

Content & Interaction

• Has clarification questions (Required vs Optional)
• Has output specification
• Has official documentation links

Zero-Shot & Reusability

• Has "Before Implementation" section (context gathering)
• Domain expertise embedded in references/ (not runtime discovery)
• Handles variations (not requirement-specific)

Type-Specific (check based on skill type)

• Builder: Clarifications + Output Spec + Standards + Checklist
• Guide: Workflow + Examples + Docs
• Automation: Scripts + Dependencies + Error Handling
• Analyzer: Scope + Criteria + Output Format
• Validator: Criteria + Scoring + Thresholds + Remediation

If 10+ checked: Likely Production (90+) If 7-9 checked: Likely Good (75-89) If 5-6 checked: Likely Adequate (60-74) If <5 checked: Needs significant work

Reference Files

Usage Examples

Validate a skill

Validate the chatgpt-widget-creator skill against production criteria

Quick audit

Quick validation check on mcp-builder skill

Focused review

Check if skill-creator skill has proper user interaction patterns