research-git

Repo Research

Scan public GitHub repos for agent skills (SKILL.md ecosystem), dev practices (git/PR/CI workflows from real teams), or code patterns (framework idioms, config layouts) — then merge validated insights into the local catalog with full attribution.

Manual extraction is unsustainable. The agent-skill ecosystem alone has 1,400+ repos as of April 2026, and that's before counting the repos worth scanning for their CI, testing, or i18n setups. This skill makes the scan repeatable and the merge auditable.

Navigation

• Quick Reference — entry-point table by need
• Three Modes — skill / practice / code targeting
• When to Use and When NOT to Use
• Default Workflow — discover → triage → fetch → diff → merge
• Output Contract — per-mode scan report shape
• references/discovery-protocol.md — discovery commands and triage (includes velocity/dependency signals and beyond-GitHub hosts)
• references/code-pattern-mining.md — Mode C extraction
• references/attribution-rules.md — licensing
• references/code-search-syntax.md — GitHub Blackbird qualifiers, gh search code CLI, rate-limit strategy, example queries per mode
• references/graphql-triage.md — single-repo health query, batch-alias pattern, repo discovery via search connection
• references/signal-quality.md — fake-star/astroturf detection: fork ratio, GH Archive spike queries, contributor account-age checks
• references/git-history-forensics.md — single-repo git-history verification: pickaxe -S/-G, blame -w -C -M, bisect run, range-diff, when git log lies
• data/sources.json — registries, authors, hot lists, ecosystem analytics, cross-host registries

Quick Reference

Four Modes

Mode A — Skill Discovery

Target: repos containing SKILL.md + references/ (the agent-skill ecosystem). Fetch: SKILL.md, references/, optionally scripts/. Output: research pack → feeds existing software-*, data-*, ai-*, ops-* skills in your catalog. Use when: enriching an existing skill, or auditing what already exists before building one.

Mode B — Practice Scan

Target: real production repos (not skill repos) with strong process signals. Fetch: .github/ (workflows, PR/issue templates, CODEOWNERS), CONTRIBUTING.md, SECURITY.md, release notes cadence, docs/adr/ (architecture decisions). Output: research pack → feeds dev-git-workflow, qa-*, ops-* skills. Use when: redesigning team policy (branching, PR review, CI gates, release cadence) and you want evidence from real teams, not just framework docs.

Mode C — Code Pattern Extraction

Target: high-signal OSS repos in a specific language/framework. Fetch: configs (tsconfig, biome, eslint, ruff, cargo), representative source modules, test layouts, scripts/ or Makefile. Output: patterns → feeds software-* skills. Use when: a local skill covers a domain where mature OSS implementations exist and the team's patterns are better than anything in docs (think: React Query's cache patterns, tRPC's type-safety tricks, Turborepo's build graph).

Mode D — Killer-Feature Mining

Target: OSS clones of a specific commercial product (e.g., supabase/supabase clones Firebase, plausible/analytics clones Google Analytics). Fetch: README.md, CHANGELOG.md, docs/, landing pages — the marketing surface that reveals which features the OSS author chose to replicate (and which they explicitly didn't). Output: rows on the shared pay-trigger-ledger.tsv with signal_type=oss_clone_focus → contributes to the bundle's Killer-Feature Convergence Protocol owned by research-review-mining. Use when: the bundle is hunting a killer feature for a commercial product, OR you want to know what the OSS world considers the load-bearing feature(s) of a category leader. Premise: OSS authors only reimplement what they think matters. That choice is revealed preference under cost — a strong proxy for monetizable core. Reference: references/killer-feature-mining.md — full extraction protocol + LLM prompts.

When to Use

• Enriching a skill: you have software-ios-native and want to steal what other operators learned
• Pre-build audit: you're about to author a new skill — has the work already been done?
• Policy redesign: your team's PR workflow is breaking — scan how 5 leading OSS repos handle it
• Framework adoption: you're committing to a new framework — pull idioms from the repos that stress-test it
• Periodic refresh: quarterly re-scan of a domain to catch new patterns from active maintainers
• Bundle handoff — killer-feature scan: research-review-mining Killer-Feature Mode asks Mode D for the OSS clone signal on a target commercial product

When NOT to Use

• Web articles, papers, blog posts → ai-deep-research
• Library/package selection or upgrade path → dev-dependency-management
• Cross-repo code context for your own portfolio → dev-context-multi-repo
• Per-commit message generation or commit-policy implementation → dev-git-commit-message
• Branching-model design in isolation (no evidence-gathering needed) → dev-git-workflow
• One-off lookup of a specific file → plain WebFetch
• Cloning to fork → plain git clone
• Validated Q&A answers or known-error solutions → the Stack Overflow corpus (community MCP or the emerging Stack Overflow for Agents exchange), via qa-debugging — not repo mining

Default Workflow

ASCII Flow

public repo research request
  -> Choose mode: skill, practice, or code
  -> Check prior packs, cached raw extracts, and target sources.json
  -> Discover and shortlist 3-5 high-signal repos
  -> Fetch only mode-specific assets and pin source commit SHAs
  -> Diff external material against the local target
  -> Mine novel patterns and write an attributed research pack
  -> Wait for explicit approval before merging changes

Phase 0 — Context Check (always run first)

Before fetching anything from GitHub:

1. Prior research packs: ls docs/research/*-scan.md — if a recent pack covers this domain + mode, read it first
2. Cached extractions: ls docs/research/*/raw/<owner>__<repo>/ — if a repo was extracted in the last 30 days, reuse unless HEAD advanced
3. Target skill's data/sources.json: if a source is already tracked, compare its commit_sha to the current repo to decide refresh vs reuse
4. Existing pack as Level 1 input — only re-fetch the delta

Mirrors the context-first protocol: use prepared artifacts before raw fetches.

Phase 1–9 — Active Research

1. Frame the goal: "Enrich software-ios-native with novel patterns from the iOS skill ecosystem" or "Redesign release workflow using practices from 3 active monorepos" or "Improve React i18n patterns in software-localisation"
2. Discover: scripts/search_repos.sh --kind <mode> <domain> → ranked shortlist
3. Triage: pick 3–5 repos using signals in references/discovery-protocol.md — applies to all modes
4. Extract (only what's missing or stale): scripts/fetch_repo_assets.sh <owner>/<repo> docs/research/<scan-id>/raw/ --kind <mode>
5. Diff: scripts/diff_against_local.sh docs/research/<scan-id>/raw/<repo>/ <target-local-skill>/
6. Mine insights: follow mode-specific guidance
   • Mode A → references/insight-mining.md
   • Mode B → references/practice-scan-targets.md
   • Mode C → references/code-pattern-mining.md
   • Mode D → references/killer-feature-mining.md (output appends to shared bundle ledger, not a local skill)
7. Synthesize: research pack at docs/research/<scan-id>.md
8. User reviews: present the pack, wait for approval on what to merge
9. Apply (opt-in): follow references/apply-protocol.md

Output Contract

Research pack at docs/research/YYYY-MM-DD-<mode>-<domain>-scan.md:

# <Mode> Scan: <Domain> — <Date>

## Mode
skill | practice | code | killer-feature

## Sources Reviewed
| Repo | Stars | Last commit | License | Scorecard | Quality | Action |
|------|-------|-------------|---------|-----------|---------|--------|

## Insights Extracted
For each insight:
- Source: <repo URL + commit SHA>
- Mode: skill | practice | code
- Pattern: <name and 1-line description>
- Why it matters: <evidence from the source>
- Where it goes: <target skill + reference file>
- Novel vs local: <new / extends existing / duplicates existing>
- Confidence: <high / medium / low + rationale>

## Recommended Merges
| Pattern | Target skill | Action | Approved? |

## Skipped
<insights reviewed and rejected, with reason>

## Attribution Pack
<full source list with URLs, commit SHAs, licenses, extraction dates>

Attribution Rules

Mandatory before any merge:

1. Check the source repo's LICENSE — MIT / Apache-2.0 / BSD / CC-BY-4.0 permit derived work with attribution
2. Never copy SKILL.md, reference files, or source files verbatim — extract patterns, rewrite in local voice
3. Cite source URL + commit SHA + extraction date + license on every merged insight
4. Add the source to the target skill's data/sources.json
5. Pin to commit SHA, never main — supply-chain drift is real

Full rules: references/attribution-rules.md

Patterns

Anti-Patterns

Known Issues (July 2026)

Scenarios

Scenario 1 — Skill Discovery (Mode A)

Goal: enrich software-kafka with patterns from the agent-skill ecosystem.

scripts/search_repos.sh --kind skill kafka
# → ranked shortlist: confluentinc/kafka-agent-skill, redpanda-data/skills, ...

# Triage: keep 3, drop LLM-generated candidates
scripts/fetch_repo_assets.sh confluentinc/kafka-agent-skill \
  docs/research/2026-04-23-skill-kafka/raw/ --kind skill

scripts/diff_against_local.sh \
  docs/research/2026-04-23-skill-kafka/raw/confluentinc__kafka-agent-skill/ \
  frameworks/shared-skills/skills/software-kafka/
# → diff shows 2 new reference files, 1 new quick-reference row

# Mine, synthesize, present research pack, apply with attribution

Scenario 2 — Practice Scan (Mode B)

Goal: redesign the team's PR workflow; harvest practices from 3 active monorepos.

scripts/search_repos.sh --kind practice monorepo
# → vercel/next.js, microsoft/vscode, nrwl/nx

scripts/fetch_repo_assets.sh vercel/next.js \
  docs/research/2026-04-23-practice-pr/raw/ --kind practice
# fetches .github/workflows/, CONTRIBUTING.md, CODEOWNERS, PR template, release-please config

# Mine via practice-scan-targets.md rubric: merge queue config, required checks,
# auto-assignment rules, review SLA signals

# Output feeds dev-git-workflow, not this skill

Scenario 4 — Killer-Feature Mining (Mode D)

Goal: contribute the OSS clone signal to the bundle's killer-feature hunt for Firebase.

scripts/search_repos.sh --kind killer-feature firebase
# → supabase/supabase, appwrite/appwrite, nhost/nhost, pocketbase/pocketbase

# Triage: keep 3 with distinct owners; reject any that ship a paid hosted tier
# (their feature choices are biased by what they want to monetize)

scripts/fetch_repo_assets.sh supabase/supabase \
  docs/research/2026-05-26-killer-feature-firebase/raw/ --kind killer-feature
# fetches README, CHANGELOG, docs/, website/, package.json

# Feed README + landing pages to LLM prompt §1 in references/killer-feature-mining.md
# → JSONL of replicated features with monetization framing

# Feed "Limitations vs Firebase" section to LLM prompt §2
# → JSONL of explicitly-omitted features (inverse signal — these are the
#   parts of Firebase that OSS authors think aren't paid for)

# Append rows to ../research-review-mining/assets/pay-trigger-ledger.tsv
#   signal_type = oss_clone_focus
# Run ../research-review-mining/scripts/converge_killer_features.py
# Convergence Rule decides which feature_ids cross the 3-of-6 threshold

Scenario 3 — Code Pattern Extraction (Mode C)

Goal: improve software-localisation with real React i18n patterns.

scripts/search_repos.sh --kind code react i18n
# → lingui-js/js-lingui, formatjs/formatjs, i18next/i18next

scripts/fetch_repo_assets.sh lingui-js/js-lingui \
  docs/research/2026-04-23-code-react-i18n/raw/ --kind code
# fetches: package.json, tsconfig, representative source modules, test layout

# Mine via code-pattern-mining.md: ICU plurals handling, runtime vs build-time,
# type-safe message catalogs

# Output feeds software-localisation

Resources

Workflow references:

• references/discovery-protocol.md — finding repos via gh CLI + awesome lists (all modes); velocity/dependency signals; beyond-GitHub hosts
• references/code-search-syntax.md — Blackbird qualifiers, gh search code, rate-limit budget, example queries for Modes A/B/C
• references/graphql-triage.md — single-repo health query, batch-alias for 5-10 repos, search connection for discovery
• references/signal-quality.md — fake-star detection: fork ratio, GH Archive spike query, contributor account-age, issue/star floor
• references/git-history-forensics.md — verify practice/pattern claims against real git history: pickaxe -S/-G, blame -w -C -M --ignore-revs-file, bisect run, range-diff, when git log lies
• references/extraction-protocol.md — fetching assets without cloning
• references/insight-mining.md — Mode A (skills) mining rubric
• references/practice-scan-targets.md — Mode B (practices) mining rubric
• references/code-pattern-mining.md — Mode C (code) mining rubric
• references/killer-feature-mining.md — Mode D (oss_clone_focus signal) mining rubric + LLM prompts
• references/attribution-rules.md — license compliance + citation format
• references/apply-protocol.md — merging insights into target skills
• references/claude-code-ecosystem-catalog.md — seed list of high-signal Claude Code / coding-agent repos for scan input

Scripts:

• scripts/search_repos.sh — gh CLI wrapper, mode-aware
• scripts/fetch_repo_assets.sh — mode-aware asset fetcher
• scripts/diff_against_local.sh — compare external vs local

Sources:

• data/sources.json — registries, high-signal authors, per-mode targets

Related Skills

• ../ai-deep-research/SKILL.md — web/paper research (this skill is repo research)
• ../dev-context-multi-repo/SKILL.md — code context across your own repos (this skill is external-repo pattern extraction)
• ../dev-dependency-management/SKILL.md — library/package selection (this skill is pattern extraction)
• ../dev-git-workflow/SKILL.md — target consumer of Mode B output
• ../dev-git-commit-message/SKILL.md — per-commit message generation (disjoint scope)
• ../agents-skills/SKILL.md — authoring your own skills (this skill enriches them)

Fact-Checking

Verify repo activity, license, and Scorecard against current GitHub before citing findings. Repo research drifts fast — re-check before merging insights. Always confirm the repo is not LLM-generated (commit history, issue activity, real contributors) before trusting any pattern from it.

GitHub API rate limits and git-version facts cited in this skill (Known Issues table, references/code-search-syntax.md, references/git-history-forensics.md) were verified against docs.github.com and git-scm.com on 2026-07-11 — re-verify at those sources before citing a specific number or version in a research pack, since both move faster than this skill's revision cadence.

Learnings Loop

Before applying this skill on a non-trivial task, read learnings.consolidated.md in this directory (and learnings.md if present).

After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to learnings.md via agents-skills-feedback-loop/scripts/append_learning.py. Do not modify SKILL.md itself.