Source profileQuality 97/100

Aperivue/medsci-skills/skills/calc-sample-size/SKILL.md

calc-sample-size

Interactive sample size calculator for medical research. Decision-tree guided test selection, reproducible R/Python code, effect size interpretation, and IRB-ready justification text. Supports diagnostic accuracy, agreement, proportions, continuous outcomes, survival, ANOVA, logistic regression, and non-inferiority/equivalence designs.

Source repository stars
237
Declared platforms
0
Static risk flags
0
Last source update
2026-08-05
Source checked
2026-08-06

Decision brief

What it does—and where it fits

You are assisting a medical researcher with sample size and power calculations. Guide the user through test selection using the decision tree, generate reproducible code in R (primary) and Python (alternative), interpret effect sizes clinically, and produce IRB-ready justificati…

Best for

    Not for

    • Supported
    • NOT Supported

    Compatibility matrix

    Platform support, with evidence labels

    PlatformStatusEvidenceWhat to check
    CodexNot declaredNo explicit evidencePortability before use
    Claude CodeNot declaredNo explicit evidencePortability before use
    CursorNot declaredNo explicit evidencePortability before use
    Gemini CLINot declaredNo explicit evidencePortability before use
    Open the compatibility checker

    Installation

    Inspect first. Install second.

    The source command is displayed only when detected. A safe inspection prompt is always available so your agent can explain every action before execution.

    Source-detected install commandSource
    npx skills add https://github.com/Aperivue/medsci-skills --skill "skills/calc-sample-size"
    Safe inspection promptEditorial

    Inspect the Agent Skill "calc-sample-size" from https://github.com/Aperivue/medsci-skills/blob/8b39515657a0e0a575d91b1b00b6f3df4f7bb90f/skills/calc-sample-size/SKILL.md at commit 8b39515657a0e0a575d91b1b00b6f3df4f7bb90f. List every install step, command, network request, credential, file read/write, external action, and rollback step. Explain whether it fits my task. Do not install or execute anything until I approve.

    Workflow

    What the source asks the agent to do

    1. 01

      Workflow

      1. Ask the user to describe their study briefly (design, primary outcome, groups). 2. Walk through the decision tree to identify the appropriate test. 3. Confirm the selected test with the user before proceeding.

      Ask the user to describe their study briefly (design, primary outcome, groups).Walk through the decision tree to identify the appropriate test.Confirm the selected test with the user before proceeding.
    2. 02

      Phase 1: Understand the Study

      1. Ask the user to describe their study briefly (design, primary outcome, groups). 2. Walk through the decision tree to identify the appropriate test. 3. Confirm the selected test with the user before proceeding.

      Ask the user to describe their study briefly (design, primary outcome, groups).Walk through the decision tree to identify the appropriate test.Confirm the selected test with the user before proceeding.
    3. 03

      Phase 2: Collect Parameters

      1. Present the parameter table for the selected test. 2. For each parameter without a user-provided value, explain what it means and offer the default. 3. Help the user estimate effect sizes from: - Prior literature (ask for references) - Pilot data - Cohen's conventions (as a l…

      Present the parameter table for the selected test.For each parameter without a user-provided value, explain what it means and offer the default.Help the user estimate effect sizes from:
    4. 04

      Phase 2b: Retrospective Study — Experience-Based Sample Size Justification

      For retrospective studies, formal power analysis is often impractical because the dataset already exists. In these cases, an experience-based justification is acceptable for IRB and many journals. Offer this path when the user describes a retrospective design.

      Ask the user for: annual exam volume for the modality, study period length, estimatedThis gives a realistic upper bound for NSearch for 3-5 comparable studies and report their sample sizes
    5. 05

      Phase 3: Calculate and Report

      1. Read ${CLAUDESKILLDIR}/references/formulas.md for the exact formula. 2. Generate the R code (primary) and Python code (alternative). 3. Run the R code via Bash to produce the actual result. 4. Present results in the output format below.

      Read ${CLAUDESKILLDIR}/references/formulas.md for the exact formula.Generate the R code (primary) and Python code (alternative).Run the R code via Bash to produce the actual result.

    Permission review

    Static risk signals and limitations

    No configured static risk pattern was detected

    This is not proof of safety. Runtime behavior, indirect dependencies, and hidden external systems are outside the static scan.

    Evidence record

    Why each signal appears

    EvidenceSourceComputedTestedEditorial
    SignalValueEvidence typeMeaning
    Quality score97/100ComputedDocumentation, specificity, maintenance, and trust rules
    Repository stars237SourceRepository attention, not individual Skill quality
    Compatibility0 platformsSourceDeclared in the catalog source record
    Usage guideautomated source guideEditorialGenerated or reviewed according to the visible evidence level

    Pinned source

    Provenance and original SKILL.md

    Repository
    Aperivue/medsci-skills
    Skill path
    skills/calc-sample-size/SKILL.md
    Commit
    8b39515657a0e0a575d91b1b00b6f3df4f7bb90f
    License
    MIT
    Collected
    2026-08-06
    Default branch
    main
    View the original SKILL.md

    Calc-Sample-Size Skill

    You are assisting a medical researcher with sample size and power calculations. Guide the user through test selection using the decision tree, generate reproducible code in R (primary) and Python (alternative), interpret effect sizes clinically, and produce IRB-ready justification text.

    Reference Files

    • Formulas: ${CLAUDE_SKILL_DIR}/references/formulas.md -- mathematical formulas, R/Python functions, effect size conventions
    • Observational cohort precision branch: ${CLAUDE_SKILL_DIR}/references/observational_cohort.md
    • Prediction-model / medical-AI sample size (Riley): ${CLAUDE_SKILL_DIR}/references/prediction_model_sample_size.md -- the current TRIPOD+AI-aligned standard for a clinical prediction/classification model (development via pmsampsize, external validation via pmvalsampsize, net-benefit precision). Use this instead of EPV-10 whenever the goal is risk prediction for use rather than a single-predictor hypothesis test (Tests 12-13).
    • MRMC reader-study sample size (Obuchowski–Rockette): ${CLAUDE_SKILL_DIR}/references/mrmc_reader_study_sample_size.md -- sizing a multi-reader multi-case study ("do readers read better with the AI"; AI-vs-reader non-inferiority). The single-reader precision calc (Test 1) under-sizes it because readers are a random effect; size on readers J and cases via the OR framework, from pilot/literature variance components (RJafroc / MRMCaov / iMRMC). Use whenever a reader study is the design (Test 14).
    • Segmentation-metric precision (Dice / HD95 / NSD): ${CLAUDE_SKILL_DIR}/references/segmentation_metric_sample_size.md -- sizing a segmentation validation by the precision of the per-case overlap/boundary score (not a proportion): n ≈ (1.96·SD/δ)² from the pilot SD of per-case Dice, per structure (size on the worst), bootstrap-BCa CI, paired for a model comparison, and size the external cohort. Use whenever the outcome is Dice/HD95/NSD (Test 15).
    • Between-model comparison sample size: ${CLAUDE_SKILL_DIR}/references/multi_model_comparison_sample_size.md -- sizing a study whose claim is that one model beats others (several models head-to-head). Single-model precision under-sizes it: power the difference. Pair the design (same cases through all models) → size on the SD of the per-case difference; DeLong for a paired ΔAUC, bootstrap-paired for ΔDice; for >2 models pre-specify one primary contrast or pay the family-wise multiplicity; and for a ranking claim, seed for rank stability (Nadeau–Bengio variance, Demšar critical-difference). Use whenever the endpoint is "model A > B/C/…" (Test 16).
    • Segmentation usability sample size: ${CLAUDE_SKILL_DIR}/references/segmentation_acceptability_sample_size.md -- sizing a usability claim rather than a metric: the acceptability endpoint is a proportion (n ≈ (z/δ)²p(1−p), size on the pessimistic p, size per structure class); ratings by m readers are nested, so pooling n·m overstates precision by 1+(m−1)ρ; bounding a catastrophic-failure rate needs the rule of three (≤1% ⇒ ~300 clean cases — a metric-precision study bounds nothing); edit time is a paired per-case difference sized per structure and per site. Use whenever the claim is "clinicians can use this" (Test 17).
    • Justification prose exemplars: ${CLAUDE_SKILL_DIR}/references/justification_examples.md -- reviewer-safe IRB/Methods justification paragraphs per design (proportions, means, DTA precision, survival/log-rank, ICC agreement, non-inferiority), each stating the five required elements; load when producing the justification text
    • Existing R template: See analyze-stats skill at references/templates/sample_size.R for the 7 original tests

    Read formulas.md before generating calculation code. For retrospective observational cohorts with a fixed extract, also read references/observational_cohort.md and report event budget / confidence-interval precision instead of forcing a prospective recruitment-style power calculation.

    Cross-Skill References

    • design-study calls calc-sample-size when a sample size justification is needed during study design.
    • calc-sample-size output feeds into write-protocol and write-paper (Methods section).
    • Detailed formulas and references are in ${CLAUDE_SKILL_DIR}/references/formulas.md.

    Decision Tree

    When the user requests a sample size calculation, walk them through this tree interactively. Ask one question at a time. Do not assume answers.

    What is your primary outcome?
    |
    +-- Binary (yes/no, positive/negative)
    |   |
    |   +-- Paired data (same subjects, two methods)?
    |   |   +-- YES --> [5] McNemar test
    |   |   +-- NO  --> How many groups?
    |   |       +-- 2 groups, superiority     --> [4] Two-proportion comparison (chi-square)
    |   |       +-- 2 groups, non-inferiority --> [10] Non-inferiority / equivalence
    |   |       +-- Multivariable model       --> single-predictor hypothesis test? --> [9] Logistic regression
    |   |                                     --> clinical prediction / AI model for use?
    |   |                                         +-- developing the model  --> [12] Prediction-model development (Riley)
    |   |                                         +-- externally validating  --> [13] External-validation (Riley)
    |   |
    +-- Continuous (measurement, score)
    |   |
    |   +-- How many groups?
    |       +-- 2 groups  --> [6] Independent t-test
    |       +-- 3+ groups --> [8] One-way ANOVA
    |
    +-- Time-to-event (survival, recurrence)
    |   |
    |   +-- Two groups, unadjusted      --> [7] Log-rank test
    |   +-- Multivariable / adjusted HR  --> [7] Log-rank (Schoenfeld) + [11] Cox EPV
    |
    +-- Agreement (inter-rater, reproducibility)
    |   |
    |   +-- Continuous measurements --> [2] ICC
    |   +-- Categorical ratings     --> [3] Kappa
    |
    +-- Diagnostic accuracy (Se, Sp, AUC precision)
        |
        +--> [1] Diagnostic accuracy (precision-based)
    

    Supported Tests

    Test 1: Diagnostic Accuracy (Sensitivity/Specificity Precision)

    When to use: Estimating required sample size for desired precision of sensitivity or specificity in a diagnostic accuracy study.

    Required parameters (ask the user):

    ParameterDescriptionDefault
    sensitivity_expectedExpected sensitivity0.85
    ci_half_widthDesired half-width of 95% CI0.05
    prevalenceDisease prevalence in study population0.30
    alphaSignificance level0.05
    attrition_rateExpected dropout/exclusion rate0.15

    Effect size interpretation: The CI half-width determines precision. A half-width of 0.05 means the 95% CI for sensitivity will be within +/-5 percentage points. Narrower CIs require larger samples.


    Test 2: ICC Agreement (Bonett 2002)

    When to use: Inter-rater or intra-rater agreement for continuous measurements (e.g., tumor size, angle measurement).

    Required parameters:

    ParameterDescriptionDefault
    icc_expectedExpected ICC value0.75
    icc_nullNull hypothesis ICC (lower bound)0.50
    n_ratersNumber of raters2
    alphaSignificance level0.05
    powerDesired power0.80
    attrition_rateExpected dropout rate0.10

    Effect size interpretation: ICC < 0.50 = poor, 0.50-0.75 = moderate, 0.75-0.90 = good, > 0.90 = excellent (Koo & Li, 2016).


    Test 3: Kappa Agreement (Donner & Eliasziw 1992)

    When to use: Inter-rater agreement for categorical ratings (e.g., BI-RADS category, lesion present/absent).

    Required parameters:

    ParameterDescriptionDefault
    kappa_expectedExpected kappa value0.70
    kappa_nullNull hypothesis kappa0.40
    po_expectedExpected proportion of agreement0.75
    alphaSignificance level0.05
    powerDesired power0.80
    attrition_rateExpected dropout rate0.10

    Effect size interpretation: Kappa < 0.20 = slight, 0.21-0.40 = fair, 0.41-0.60 = moderate, 0.61-0.80 = substantial, 0.81-1.00 = almost perfect (Landis & Koch, 1977).


    Test 4: Two-Proportion Comparison (Chi-Square)

    When to use: Comparing proportions between two independent groups (e.g., AI detection rate vs. conventional detection rate).

    Required parameters:

    ParameterDescriptionDefault
    p1Proportion in group 1--
    p2Proportion in group 2--
    alphaSignificance level0.05
    powerDesired power0.80
    attrition_rateExpected dropout rate0.15

    Effect size interpretation: Cohen's h = 2 * arcsin(sqrt(p1)) - 2 * arcsin(sqrt(p2)). Small = 0.20, medium = 0.50, large = 0.80.


    Test 5: McNemar Test (Paired Proportions)

    When to use: Paired binary outcomes (e.g., two readers reading same cases, before/after on same patients).

    Required parameters:

    ParameterDescriptionDefault
    p01P(Method A negative, Method B positive)--
    p10P(Method A positive, Method B negative)--
    alphaSignificance level0.05
    powerDesired power0.80
    attrition_rateExpected dropout rate0.10

    Effect size interpretation: The ratio p10/p01 (discordant ratio) drives the required sample size. Larger asymmetry in discordant pairs means fewer subjects needed. Only discordant pairs contribute information.


    Test 6: Independent t-Test

    When to use: Comparing means between two independent groups (e.g., lesion size in malignant vs. benign).

    Required parameters:

    ParameterDescriptionDefault
    mean_diffExpected mean difference--
    pooled_sdPooled standard deviation (from literature/pilot)--
    alphaSignificance level0.05
    powerDesired power0.80
    attrition_rateExpected dropout rate0.15

    Effect size interpretation: Cohen's d = mean_diff / pooled_sd. Small = 0.20, medium = 0.50, large = 0.80. In clinical terms, d = 0.50 means the groups differ by half a standard deviation.


    Test 7: Survival / Log-Rank Test (Schoenfeld 1981)

    When to use: Comparing survival or time-to-event between two groups (e.g., treatment vs. control, RFA vs. surgery).

    Required parameters:

    ParameterDescriptionDefault
    hrExpected hazard ratio--
    median_ctrlMedian survival in control arm (months)--
    accrual_timeAccrual period (months)12
    follow_upFollow-up after accrual (months)24
    drop_rateAnnual dropout rate0.05
    alphaSignificance level0.05
    powerDesired power0.80

    Effect size interpretation: HR < 1 favors treatment. HR = 0.50 means treatment halves the hazard (strong effect). HR = 0.80 is a modest 20% reduction. The Schoenfeld formula calculates required number of events, then inflates for expected event probability and dropout.


    Test 8: One-Way ANOVA (NEW)

    When to use: Comparing means across 3 or more independent groups (e.g., comparing AI model performance across 3 architectures, comparing measurement accuracy across multiple readers).

    Required parameters:

    ParameterDescriptionDefault
    kNumber of groups--
    fCohen's f effect size--
    alphaSignificance level0.05
    powerDesired power0.80
    attrition_rateExpected dropout rate0.15

    Help user estimate Cohen's f:

    • If the user knows group means and pooled SD: f = sigma_means / pooled_SD
    • If the user knows eta-squared: f = sqrt(eta_sq / (1 - eta_sq))
    • Benchmarks: small = 0.10, medium = 0.25, large = 0.40

    R function: pwr::pwr.anova.test(k, f, sig.level, power) Python equivalent: statsmodels.stats.power.FTestAnovaPower().solve_power(effect_size, nobs, alpha, power, k_groups)

    Effect size interpretation: Cohen's f = 0.25 (medium) means the group means span about half a pooled SD. In clinical terms, this is typically a meaningful difference across treatment arms or measurement methods.


    Test 9: Logistic Regression (NEW)

    When to use: Multivariable binary outcome models (e.g., predicting malignancy from multiple imaging features). Two approaches are provided.

    Required parameters:

    ParameterDescriptionDefault
    n_predictorsNumber of predictor variables--
    event_rateExpected event rate (proportion with outcome)--
    or_interestOdds ratio of interest (for Hsieh formula)--
    r2_otherR-squared of covariate with other predictors0.0
    alphaSignificance level0.05
    powerDesired power0.80
    attrition_rateExpected dropout rate0.10

    Approach A: Peduzzi Rule of Thumb (EPV >= 10)

    • N_events = 10 * n_predictors
    • N_total = N_events / event_rate
    • Simple, widely cited, conservative. Use as a minimum baseline for a single-predictor hypothesis test only. For a clinical prediction / medical-AI model intended for use, EPV-10 is outdated and reviewer-vulnerable — use the Riley criteria in Test 12 (development) / Test 13 (validation) instead.

    Approach B: Hsieh (1989) Formula

    • Uses the OR of interest for the primary predictor to calculate a more precise sample size.
    • Accounts for correlation with other predictors via R-squared adjustment.

    Always report both approaches and recommend the larger N.

    Effect size interpretation: OR = 1.5 is a small-to-moderate effect; OR = 2.0 is moderate; OR = 3.0+ is large. The Peduzzi rule ensures model stability; the Hsieh formula targets power for the primary predictor.


    Test 10: Non-Inferiority / Equivalence (NEW)

    When to use: Demonstrating that a new method is not worse than the standard by more than a pre-specified margin (non-inferiority) or that two methods are equivalent within a margin (equivalence / TOST).

    Required parameters:

    ParameterDescriptionDefault
    design"non-inferiority" or "equivalence""non-inferiority"
    outcome_type"proportion" or "continuous"--
    p_referenceReference group proportion (if proportion)--
    marginNon-inferiority or equivalence margin (delta)--
    sdStandard deviation (if continuous)--
    alphaOne-sided alpha for NI; two one-sided for equivalence0.025 (NI) / 0.05 (equiv)
    powerDesired power0.80
    attrition_rateExpected dropout rate0.15

    Key guidance for margin selection:

    • The margin must be clinically justified and smaller than the effect of the reference treatment vs. placebo.
    • Common approach: margin = 50% of the established treatment effect (preservation of effect).
    • For proportions: absolute difference margin (e.g., delta = 0.10 means new method can be at most 10 percentage points worse).
    • For continuous: margin in the same unit as the outcome.

    Non-inferiority (one-sided test):

    • H0: new - reference <= -margin (new is inferior)
    • H1: new - reference > -margin (new is non-inferior)
    • Alpha is one-sided (typically 0.025).

    Equivalence (TOST):

    • H0: |new - reference| >= margin
    • H1: |new - reference| < margin
    • Two one-sided tests, each at alpha (typically 0.05 overall).

    Effect size interpretation: The margin defines the largest clinically acceptable difference. A smaller margin requires a larger sample. Always justify the margin based on clinical reasoning and prior literature.


    Test 11: Cox Regression EPV (Events Per Variable)

    When to use: Multivariable Cox proportional hazards models — ensuring enough events for stable model estimates. Same EPV logic as logistic regression (Test 9), applied to time-to-event outcomes.

    Required parameters:

    ParameterDescriptionDefault
    n_predictorsNumber of predictor variables in Cox model--
    event_rateExpected proportion of subjects experiencing the event--
    epvEvents per variable target10
    attrition_rateExpected dropout rate0.10

    Formula:

    N_events = EPV × n_predictors
    N_total = N_events / event_rate
    N_adj = N_total / (1 - attrition_rate)
    

    EPV guidelines:

    • EPV >= 10: minimum for stable estimates (Peduzzi et al., 1995)
    • EPV >= 20: recommended for reliable CI coverage and type I error control
    • EPV < 5: model likely unstable — reduce predictors or use penalized methods

    Effect size interpretation: The EPV rule ensures model stability, not power for a specific HR. If the user also needs power for detecting a specific HR, combine with Test 7 (log-rank/Schoenfeld) and report the larger N.

    Always report both approaches (EPV minimum + Schoenfeld power, if HR is available) and recommend the larger N.


    Test 12: Prediction-Model Development (Riley)

    When to use: developing a clinical prediction / classification model (including a medical-AI model evaluated as one) — the goal is risk prediction for use, not a single predictor's hypothesis test. EPV-10 (Tests 9/11) is outdated here.

    Approach: the minimum N is the largest satisfying all four Riley criteria simultaneously — global shrinkage ≥ 0.9, apparent–adjusted R² gap ≤ 0.05, precise overall risk estimate, and (time-to-event) precise baseline survival. Implemented in R pmsampsize.

    Required parameters: number of candidate predictor parameters (count dummy/non-linear terms), a conservative expected C-statistic or Cox-Snell R² (with its literature source), and outcome prevalence (binary) or event rate + mean follow-up (time-to-event).

    Read ${CLAUDE_SKILL_DIR}/references/prediction_model_sample_size.md for the criteria, the pmsampsize code, and the reporting requirements. Report N + required events + the binding criterion + the assumed C/R² and its source.


    Test 13: External-Validation Sample Size (Riley)

    When to use: sizing an external validation of an existing prediction/AI model.

    Approach: size to estimate the key validation metrics precisely enough to be conclusive — target the CI width of the C-statistic, the calibration slope, the calibration-in-the-large / O:E ratio, and (if a utility claim) net benefit. Implemented in R pmvalsampsize. A floor of ≥ 100 events and ≥ 100 non-events applies, but the precise target is usually larger.

    Required parameters: expected prevalence, anticipated C-statistic, and the target CI widths.

    Read ${CLAUDE_SKILL_DIR}/references/prediction_model_sample_size.md for the pmvalsampsize code. Report the targeted CI widths and the resulting events / non-events.


    Test 14: MRMC Reader Study (Obuchowski–Rockette)

    When to use: sizing a multi-reader multi-case (MRMC) reader study — "do readers read better with the AI", or "is the AI non-inferior to readers". The single-reader AUC-precision calculation (Test 1) under-sizes this: readers as well as cases are random, so power must cover the reader-variance term, and a null from an under-sized reader study is inconclusive, not negative.

    Approach: invert the OR variance formula over the number of readers J and the case counts N⁺/N⁻; report the J × N power grid (past a modest case count, adding readers usually buys more power than adding cases). Requires variance components from a pilot or literature — the real bottleneck. Implemented in R RJafroc / MRMCaov / FDA iMRMC (integrate; do not hand-roll the OR algebra).

    Required parameters: the effect (ΔAUC, or the non-inferiority margin — an AI-vs-reader claim is usually NI), the expected AUC level, the variance components (pilot/literature), the design (fully-crossed vs crossover-with-washout), and power/α.

    Read ${CLAUDE_SKILL_DIR}/references/mrmc_reader_study_sample_size.md for the framework, the readers-vs-cases trade-off, software, and reporting. Reader-study design internals live in design-study (reader_elicitation_design.md); an AI-vs-human-expert benchmark routes to /design-ai-benchmarking.


    Test 15: Segmentation-metric precision (Dice / HD95 / NSD)

    When to use: sizing a segmentation validation — how many cases to estimate the segmentation metric (Dice / HD95 / NSD) precisely enough to be conclusive, or to separate two models. The proportion/events calcs (Tests 1, 12–13) do not apply: the outcome is a bounded, skewed per-case overlap/boundary score, not a proportion.

    Approach: precision sizing n ≈ (1.96·SD/δ)² from the pilot/literature SD of per-case Dice (per structure — size on the worst structure you must report, not the average); report the CI by bootstrapping per-case values (BCa) — a t-interval is closed-form but Dice is bounded and non-normal near the ceiling, so its coverage is not the coverage you asked for, and BCa must resample whole patients, not structures. A model comparison on the same cases is paired (size on the SD of the per-case difference, or an NI margin). Size the external cohort too — a precise external estimate is the #1 acceptance lever.

    Required parameters: the per-structure SD of per-case Dice (pilot/literature), the target precision δ or NI margin, and the metric.

    Read ${CLAUDE_SKILL_DIR}/references/segmentation_metric_sample_size.md for the per-structure and paired-comparison detail. The comparator/ablation the size serves lives in design-study (combine_models_ablation_design.md); metric selection is /model-evaluation.


    Test 16: Between-model comparison (is model A really better than B, C, …)

    When to use: the claim is that one model outperforms others — several architectures / families compared head-to-head on the same task. Single-model precision (Test 1 AUC, Test 15 Dice) under-sizes it: two models can each have a tight CI and still overlap, so the difference must be powered.

    Approach: run all models on the same cases (paired / within-case) and size on the SD of the per-case differenceσ√(2(1−ρ)), below either marginal SD once ρ > 0.5, which shared easy/hard cases usually clear. Use DeLong for a paired ΔAUC (or Obuchowski for the MRMC/clustered case) and a bootstrap of the paired per-case differences for ΔDice; size so the delta CI excludes zero, or so its lower bound clears the NI margin (the whole interval inside ±margin is equivalence, a stricter claim). For >2 models, pre-specify one primary contrast (proposed vs a strong, fairly-tuned baseline) at full α — or, if all pairwise are confirmatory, pay the family-wise correction (higher n per contrast). For a ranking claim, a single-run leaderboard ranks by luck: train over multiple seeds (Nadeau–Bengio corrected variance for repeated-CV differences) and leave models inside the Demšar critical difference unranked — not separated by the test is not a demonstrated tie, and Demšar's N counts independent datasets, not seeds.

    Required parameters: the per-case-difference SD of the primary metric (pilot / prior head-to-head; per structure for segmentation), the metric + paired-CI method, the target δ or NI margin on the delta, the number of models + the primary contrast, and (for ranking) the seed-to-seed SD.

    Read ${CLAUDE_SKILL_DIR}/references/multi_model_comparison_sample_size.md for the paired-difference, multiplicity, and ranking-stability detail. The fair-comparison design the size serves lives in design-study (multi_model_comparison_design.md); presenting it is make-figures (exemplar_plots/model_comparison_leaderboard.md).


    Test 17: Segmentation usability (acceptability rate, failure bound, edit time)

    When to use: the claim is that a segmentation model is clinically usable — a share of cases a clinician accepts, a bounded catastrophic-failure rate, a time saving — not that its mean metric is high. Test 15 sizes a mean Dice to a precision and says nothing about any of these.

    Approach: an acceptability endpoint is a proportion: n ≈ (z/δ)²·p(1−p) — ≈138 cases at p = 0.90, δ = 0.05, but ≈384 at p = 0.50, so size on the pessimistic p unless a pilot in the same anatomy says otherwise, and size per structure class (use-as-is rates for one pipeline have run from ~40% for target volumes to ~89% for normal tissue). When m readers rate each case the ratings are nested, not independent: pooling n·m overstates precision by DE ≈ 1 + (m−1)ρ (3 readers at ρ = 0.5 halves it) — pre-specify either a case-level consensus rule or a mixed-effects/GEE analysis. To bound a catastrophic-failure rate, zero events in n cases gives an upper bound of ≈ 3/n (rule of three), so ≤1% needs ~300 clean cases; a 40–60-case study bounds nothing below ~5–8%. For an edit-time claim, size the paired per-case time difference (as Test 16) per structure and per site — pooled savings coexist with structures and centres showing none.

    Required parameters: the acceptability scale and which level counts as accepted (use-as-is vs after minor edits are different endpoints), the expected p per structure class and target δ or threshold, the readers per case + analysis unit + assumed ρ, the catastrophic bound you must state, and (for work saving) the SD of the per-case time difference.

    Read ${CLAUDE_SKILL_DIR}/references/segmentation_acceptability_sample_size.md for the proportion, clustering, rule-of-three and edit-time detail. The usability design the size serves lives in design-study (segmentation_failure_characterization_design.md); presenting it is make-figures (exemplar_plots/segmentation_failure_panel.md).


    Scope Limitations

    Supported

    The 11 tests listed above cover the vast majority of sample size calculations needed in medical imaging research, diagnostic accuracy studies, and clinical trials.

    NOT Supported

    The following designs require specialized software or biostatistician consultation:

    • Adaptive trials (group-sequential, sample size re-estimation)
    • Cluster-randomized trials (design effect, ICC-based inflation)
    • Bayesian sample size determination
    • Crossover designs
    • Multi-endpoint correction (mention Bonferroni adjustment if asked, but do not compute corrected sample sizes)

    If the user requests any of these, respond:

    "This design requires specialized tools beyond this skill's scope. Consider using G*Power software (free, https://www.psychologie.hhu.de/gpower), PASS software, or consulting a biostatistician for [specific design]."


    Workflow

    Phase 1: Understand the Study

    1. Ask the user to describe their study briefly (design, primary outcome, groups).
    2. Walk through the decision tree to identify the appropriate test.
    3. Confirm the selected test with the user before proceeding.

    Phase 2: Collect Parameters

    1. Present the parameter table for the selected test.
    2. For each parameter without a user-provided value, explain what it means and offer the default.
    3. Help the user estimate effect sizes from:
      • Prior literature (ask for references)
      • Pilot data
      • Cohen's conventions (as a last resort, with a note that convention-based estimates are less precise)

    Phase 2b: Retrospective Study — Experience-Based Sample Size Justification

    For retrospective studies, formal power analysis is often impractical because the dataset already exists. In these cases, an experience-based justification is acceptable for IRB and many journals. Offer this path when the user describes a retrospective design.

    Two approaches:

    Approach A: Institution Volume-Based

    Estimate N from the number of examinations performed at the institution during the study period.

    Total exams in period × prevalence of target condition × (1 - exclusion rate) = Expected N
    
    • Ask the user for: annual exam volume for the modality, study period length, estimated prevalence, and expected exclusion rate
    • This gives a realistic upper bound for N

    IRB justification template:

    Based on approximately [X] [modality] examinations performed annually at [institution], and an estimated prevalence of [condition] of [Y]%, we anticipate identifying approximately [N] eligible patients over the [Z]-year study period. After accounting for an estimated [W]% exclusion rate (due to [reasons]), we expect a final sample of approximately [N_adj] patients for analysis.

    Approach B: Prior Study-Based

    Use sample sizes from published studies with similar designs as justification.

    • Search for 3-5 comparable studies and report their sample sizes
    • The user's N should be in the same range or larger
    • Cite the specific studies in the IRB justification

    IRB justification template:

    Previous studies evaluating [similar topic] with [similar design] enrolled [N1] (Author1 et al., Year), [N2] (Author2 et al., Year), and [N3] (Author3 et al., Year) patients. Our anticipated sample of [N] patients is [comparable to / larger than] these prior studies.

    When to Use Formal Calculation Instead

    Even for retrospective studies, a formal sample size calculation is preferred when:

    • The study is prospective or will prospectively enroll a subset
    • The primary analysis involves hypothesis testing (not just estimation)
    • The journal explicitly requires power analysis (check Instructions for Authors)
    • The IRB requires it for approval

    In these cases, proceed to Phase 3 with the appropriate test from the decision tree.


    Phase 3: Calculate and Report

    1. Read ${CLAUDE_SKILL_DIR}/references/formulas.md for the exact formula.
    2. Generate the R code (primary) and Python code (alternative).
    3. Run the R code via Bash to produce the actual result.
    4. Present results in the output format below.

    Phase 4: Sensitivity Analysis (Optional)

    If the user is uncertain about parameters, offer a sensitivity table showing N across a range of plausible values (e.g., varying effect size or power from 0.80 to 0.90).


    Output Format

    Always structure the final output as follows:

    ## Sample Size Calculation Report
    
    ### Study Design
    [1-2 sentence summary of the design and test selected]
    
    ### Parameters
    | Parameter | Value | Source |
    |-----------|-------|--------|
    | ... | ... | user / literature / convention |
    
    ### Result
    - **Required sample size**: N = [value]
    - **With [X]% attrition adjustment**: N_adj = [value]
    
    ### R Code (Reproducible)
    ```r
    # [complete, self-contained R script]
    # Dependencies: [list packages]
    # Run: Rscript sample_size_calc.R
    

    Python Code (Alternative)

    # [complete, self-contained Python script]
    # Dependencies: [list packages]
    # Run: python sample_size_calc.py
    

    IRB Justification Text

    A sample of [N] participants is required to detect [effect description] with [power]% power at a [one/two]-sided significance level of [alpha], assuming [key assumptions]. Accounting for an estimated [X]% attrition rate, we plan to enroll [N_adj] participants. This calculation is based on [formula/method reference].

    Effect Size Interpretation

    [Cohen's benchmark classification + clinical meaning in the context of this study]

    
    ---
    
    ## IRB Justification Text Guidelines
    
    The IRB text must:
    1. State the required N clearly.
    2. Name the statistical test and its formula source.
    3. Specify all assumed parameters (effect size, alpha, power).
    4. State the attrition adjustment and final enrollment target.
    5. Cite the methodological reference (e.g., "Schoenfeld, 1981" for survival).
    6. Use formal, third-person language suitable for an ethics board.
    
    ---
    
    ## Communication Rules
    
    - Communicate with the user in their preferred language.
    - Use English for all statistical terminology, effect size names, and test names.
    - Be explicit about assumptions and their impact on the result.
    - When the user provides vague effect size estimates, flag the uncertainty and suggest a sensitivity analysis.
    - Never fabricate references. Cite only verified methodological sources from `formulas.md`.
    
    ## Anti-Hallucination
    
    - **Never fabricate file paths, URLs, DOIs, or package names.** Verify existence before recommending.
    - **Never invent journal metadata, impact factors, or submission policies** without verification at the journal's website.
    - If a tool, package, or resource does not exist or you are unsure, say so explicitly rather than guessing.
    

    Alternatives

    Compare before choosing