Source profileQuality 66/100

github/awesome-copilot/skills/phoenix-evals/SKILL.md

phoenix-evals

Build and run evaluators for AI/LLM applications using Phoenix.

Source repository stars: 37,126
Declared platforms: 0
Static risk flags: 0
Last source update: 2026-07-28
Source checked: 2026-07-28

Decision Compatibility Install Workflow Permissions Evidence Source

Decision brief

What it does—and where it fits

Build evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans.

Best for

Not for

Tasks that require unconfirmed production actions or broad system permissions.
Environments where the pinned source and install steps cannot be inspected.

Compatibility matrix

Platform support, with evidence labels

Platform	Status	Evidence	What to check
Codex	Not declared	No explicit evidence	Portability before use
Claude Code	Not declared	No explicit evidence	Portability before use
Cursor	Not declared	No explicit evidence	Portability before use
Gemini CLI	Not declared	No explicit evidence	Portability before use

Open the compatibility checker

Installation

Inspect first. Install second.

The source command is displayed only when detected. A safe inspection prompt is always available so your agent can explain every action before execution.

Source-detected install commandSource

npx skills add https://github.com/github/awesome-copilot --skill "skills/phoenix-evals"

Safe inspection promptEditorial

Inspect the Agent Skill "phoenix-evals" from https://github.com/github/awesome-copilot/blob/9933dcad5be5caeb288cebcd370eeeb2fc2f1685/skills/phoenix-evals/SKILL.md at commit 9933dcad5be5caeb288cebcd370eeeb2fc2f1685. List every install step, command, network request, credential, file read/write, external action, and rollback step. Explain whether it fits my task. Do not install or execute anything until I approve.

Workflow

What the source asks the agent to do

01
Quick Reference
Review the “Quick Reference” section in the pinned source before continuing.
Review and apply the “Quick Reference” source section.
02
Workflows
Starting Fresh: observe-tracing-setup → error-analysis → axial-coding → evaluators-overview
Starting Fresh: observe-tracing-setup → error-analysis → axial-coding → evaluators-overviewBuilding Evaluator: fundamentals → common-mistakes-python → evaluators-{code|llm}-{python|typescript} → validation-evaluators-{python|typescript}RAG Systems: evaluators-rag → evaluators-code- (retrieval) → evaluators-llm- (faithfulness)
03
Reference Categories
Review the “Reference Categories” section in the pinned source before continuing.
Review and apply the “Reference Categories” source section.
04
Key Principles
Review the “Key Principles” section in the pinned source before continuing.
Review and apply the “Key Principles” source section.

Permission review

Static risk signals and limitations

No configured static risk pattern was detected

This is not proof of safety. Runtime behavior, indirect dependencies, and hidden external systems are outside the static scan.

Evidence record

Why each signal appears

EvidenceSourceComputedTestedEditorial

Signal	Value	Evidence type	Meaning
Quality score	66/100	Computed	Documentation, specificity, maintenance, and trust rules
Repository stars	37,126	Source	Repository attention, not individual Skill quality
Compatibility	0 platforms	Source	Declared in the catalog source record
Usage guide	automated source guide	Editorial	Generated or reviewed according to the visible evidence level

Pinned source

Provenance and original SKILL.md

Repository: github/awesome-copilot
Skill path: skills/phoenix-evals/SKILL.md
Commit: 9933dcad5be5caeb288cebcd370eeeb2fc2f1685
License: MIT
Collected: 2026-07-28
Default branch: main

View the original SKILL.md

Phoenix Evals

Build evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans.

Quick Reference

Task	Files
Setup	setup-python, setup-typescript
Decide what to evaluate	evaluators-overview
Choose a judge model	fundamentals-model-selection
Use pre-built evaluators	evaluators-pre-built
Build code evaluator	evaluators-code-python, evaluators-code-typescript
Build LLM evaluator	evaluators-llm-python, evaluators-llm-typescript, evaluators-custom-templates
Batch evaluate DataFrame	evaluate-dataframe-python
Run experiment	experiments-running-python, experiments-running-typescript
Create dataset	experiments-datasets-python, experiments-datasets-typescript
Generate synthetic data	experiments-synthetic-python, experiments-synthetic-typescript
Validate evaluator accuracy	validation, validation-evaluators-python, validation-evaluators-typescript
Sample traces for review	observe-sampling-python, observe-sampling-typescript
Analyze errors	error-analysis, error-analysis-multi-turn, axial-coding
RAG evals	evaluators-rag
Avoid common mistakes	common-mistakes-python, fundamentals-anti-patterns
Production	production-overview, production-guardrails, production-continuous

Workflows

Starting Fresh: observe-tracing-setup → error-analysis → axial-coding → evaluators-overview

Building Evaluator: fundamentals → common-mistakes-python → evaluators-{code|llm}-{python|typescript} → validation-evaluators-{python|typescript}

RAG Systems: evaluators-rag → evaluators-code-* (retrieval) → evaluators-llm-* (faithfulness)

Production: production-overview → production-guardrails → production-continuous

Reference Categories

Prefix	Description
`fundamentals-*`	Types, scores, anti-patterns
`observe-*`	Tracing, sampling
`error-analysis-*`	Finding failures
`axial-coding-*`	Categorizing failures
`evaluators-*`	Code, LLM, RAG evaluators
`experiments-*`	Datasets, running experiments
`validation-*`	Validating evaluator accuracy against human labels
`production-*`	CI/CD, monitoring

Key Principles

Principle	Action
Error analysis first	Can't automate what you haven't observed
Custom > generic	Build from your failures
Code first	Deterministic before LLM
Validate judges	>80% TPR/TNR
Binary > Likert	Pass/fail, not 1-5

Alternatives

Compare before choosing

Computed 9831,966

K-Dense-AI/scientific-agent-skills

dask

Distributed computing for larger-than-RAM pandas/NumPy workflows. Use when you need to scale existing pandas/NumPy code beyond memory or across clusters. Best for parallel file processing, distributed ML, integration with existing pandas code. For out-of-core analytics on single machine use vaex; for in-memory speed use polars.

Computed 9831,966

K-Dense-AI/scientific-agent-skills

medchem

Medicinal chemistry filters for compound triage. Apply drug-likeness rules (Lipinski, Veber, CNS), structural alert catalogs (PAINS, NIBR, ChEMBL), complexity metrics, and the medchem query language for library filtering.

Computed 9831,966

K-Dense-AI/scientific-agent-skills

neurokit2

Use NeuroKit2 to build or audit reproducible research workflows for physiological time-series preprocessing, event/interval analysis, multimodal alignment, variability, and complexity. Trigger when code imports neurokit2 or needs its current APIs, schemas, and method-aware validation—not for diagnosis or device validation.

Computed 97234,327

affaan-m/ECC

plan-orchestrate

Read a plan document, decompose it into steps, design a per-step agent chain from the ECC catalogue, and emit ready-to-paste /orchestrate custom prompts. Generative only — never invokes /orchestrate itself. Use when the user has a multi-step plan and wants to drive it through orchestrate without composing chains by hand.