neural-decoding-analysis

Neural Population Decoding Analysis

Purpose

This skill encodes expert methodological knowledge for multivariate neural decoding analyses in systems neuroscience. It covers cross-validated classification (MVPA), representational similarity analysis (RSA), temporal generalization, and encoding models. The skill provides domain-specific decision logic, parameter recommendations, and pitfall warnings that a machine-learning engineer without neuroscience training would not know.

When to Use This Skill

• Determining whether stimulus or task information is represented in neural population activity
• Comparing the representational geometry of brain regions to computational models
• Characterizing the temporal dynamics of neural representations from EEG/MEG
• Building encoding models to predict neural responses from stimulus features
• Designing a decoding analysis pipeline and choosing appropriate methods and parameters

Research Planning Protocol

Before executing the domain-specific steps below, you MUST:

1. State the research question — What specific representational or informational question is this decoding analysis addressing?
2. Justify the method choice — Why decoding/RSA (not univariate analysis, connectivity, etc.)? What alternatives were considered?
3. Declare expected outcomes — What decoding accuracy or representational structure would support vs. refute the hypothesis?
4. Note assumptions and limitations — What does this method assume? Where could it mislead (e.g., confounds, leakage)?
5. Present the plan to the user and WAIT for confirmation before proceeding.

For detailed methodology guidance, see the research-literacy skill.

⚠️ Verification Notice

This skill was generated by AI from academic literature. All parameters, thresholds, and citations require independent verification before use in research. If you find errors, please open an issue.

When to Use Decoding vs. Univariate Analysis

Univariate analysis tests whether the mean activity level differs across conditions in a region. Decoding tests whether spatial patterns of activity carry information, even when mean activity is identical across conditions (Haynes, 2015). Use decoding when:

• You expect information to be encoded in distributed patterns, not mean amplitude
• The signal-to-noise ratio per voxel/channel is low but the population carries information
• You want to compare neural representations to computational model predictions (use RSA)
• You want to track when information emerges and transforms over time (use temporal generalization)

Domain judgment: High decoding accuracy does NOT mean the decoded region is the source of the representation. It means the information is accessible from that region's patterns. A downstream region receiving a copy of the signal will also decode well (Haynes, 2015).

Method Selection Decision Tree

What is your research question?
|
+-- "Is stimulus/task information present in this brain region's patterns?"
| --> Cross-validated classification (MVPA)
| Output: classification accuracy or d-prime
|
+-- "How are representations organized? Does the geometry match a model?"
| --> Representational Similarity Analysis (RSA)
| Output: model-RDM correlation, noise ceiling
|
+-- "When does information emerge and how does it transform over time?"
| --> Temporal Generalization (time x time decoding)
| Output: temporal generalization matrix
| Best for: EEG, MEG, intracranial recordings
|
+-- "What stimulus features drive neural responses across the feature space?"
 --> Encoding Models (voxelwise/channel-wise prediction)
 Output: prediction accuracy (R^2), feature tuning maps

Cross-Validated Classification (MVPA)

Classifier Selection

Domain judgment: Linear classifiers are strongly preferred in neuroimaging because (1) fMRI/EEG patterns are high-dimensional relative to sample size, making nonlinear methods prone to overfitting, and (2) linear weights are more interpretable neurally, though see Haufe et al. (2014) on the distinction between classifier weights and activation patterns.

Cross-Validation Strategy

CRITICAL -- Information leakage: Feature selection, normalization, and dimensionality reduction MUST be performed WITHIN each cross-validation fold, using ONLY training data. Fitting a PCA or z-scoring across all data before splitting inflates accuracy by leaking test-set statistics into training (Kriegeskorte et al., 2009; Varoquaux et al., 2017).

Chance Level and Statistical Testing

• Theoretical chance: 1/n_classes for balanced designs (e.g., 50% for 2-class)
• CRITICAL: Theoretical chance is only valid with infinite samples. With small samples, empirical accuracy on random data can substantially exceed 1/n_classes (Combrisson & Jerbi, 2015)
• Use permutation testing: Shuffle labels 1000+ times, decode each permutation, compute p-value as the proportion of permuted accuracies >= observed accuracy (Combrisson & Jerbi, 2015)
• Binomial test: Acceptable quick alternative for large trial counts, but permutation testing is preferred (Combrisson & Jerbi, 2015)
• For group-level inference: test accuracy against chance across subjects using a one-sample t-test or Wilcoxon signed-rank test on subject-level accuracies

Minimum Data Requirements

• Trials per class: Minimum 20-30 trials per class for reliable within-subject decoding (Varoquaux, 2018; Grootswagers et al., 2017)
• Cross-validation error bars: With ~100 samples, expect confidence intervals of approximately +/-10% on accuracy estimates (Varoquaux, 2018)
• Trial averaging: Averaging 5-10 trials before classification improves SNR but reduces effective sample size; balance based on total trial count (Grootswagers et al., 2017)

Representational Similarity Analysis (RSA)

RSA abstracts from activity patterns to a condition-by-condition dissimilarity matrix (RDM), enabling comparison across brain regions, species, and computational models (Kriegeskorte et al., 2008).

RDM Construction

Domain judgment: The crossnobis estimator is unbiased -- its expected value is zero when two conditions have identical representations, unlike correlation distance or Euclidean distance which are positively biased by noise. This means crossnobis values can be negative (not a true distance), but this property makes it valid for statistical inference without bias correction (Walther et al., 2016).

Model Comparison

• Pearson/Spearman correlation: Correlate model RDM with brain RDM (use Spearman for robustness to outliers; Nili et al., 2014)
• Partial correlation: Control for one model while testing another (essential when models are correlated)
• Regression on RDMs: Fit multiple model RDMs simultaneously; use weighted least squares or component models (Kriegeskorte & Diedrichsen, 2019)

Statistical Inference

• Noise ceiling: Upper and lower bounds on the best achievable model fit given between-subject variability. Upper bound: average correlation of each subject's RDM with the group mean RDM. Lower bound: same but computed with leave-one-subject-out (Nili et al., 2014)
• Stimulus-label randomization: Permute condition labels to construct a null distribution for RDM correlation (Nili et al., 2014)
• Bootstrap confidence intervals: Resample subjects with replacement to estimate confidence intervals on model-RDM correlations

Domain judgment: If a model falls within the noise ceiling, it explains as much variance as is explainable given the noise in the data. A model below the lower bound leaves systematic variance unexplained. This is NOT the same as a significance test -- a model can be significantly correlated with brain RDMs yet still fall below the noise ceiling (Nili et al., 2014).

See references/rsa-guide.md for a complete step-by-step RSA workflow.

Temporal Generalization (EEG/MEG)

Train a classifier at each time point t, test it at every time point t'. The resulting time x time matrix reveals the dynamics of neural representations (King & Dehaene, 2014).

Interpreting the Temporal Generalization Matrix

(King & Dehaene, 2014; Grootswagers et al., 2017)

Sliding Window Parameters

Statistical Testing for Temporal Generalization

• Cluster-based permutation test on the time x time matrix (Maris & Oostenveld, 2007)
• Cluster-forming threshold: p < 0.05 (two-tailed) at the individual time-point level; correct at cluster level with 1000+ permutations
• Caution: Cluster tests control family-wise error rate but do NOT localize the effect to specific time points (Maris & Oostenveld, 2007)

Encoding Models

Encoding models predict neural responses from stimulus features, complementing decoding (which predicts stimuli from neural responses).

When to Use Encoding Over Decoding

• When the feature space is continuous or high-dimensional (e.g., image pixels, spectrograms)
• When you want to characterize what features a region encodes, not just whether it encodes information
• Voxelwise encoding models: fit a regularized regression (ridge) from features to each voxel's response (Kriegeskorte & Diedrichsen, 2019)

Key Considerations

• Regularization: Ridge regression (L2) is standard; prevents overfitting when features > samples
• Cross-validation: Leave-one-run-out; evaluate with Pearson correlation between predicted and actual responses
• Feature spaces: Gabor wavelets (V1), DNN layers (ventral stream), semantic embeddings (language regions)
• Relationship to RSA: When using a zero-mean isotropic Gaussian weight prior (ridge regression), encoding models, RSA, and pattern component models test equivalent hypotheses captured by the second moment matrix G (Kriegeskorte & Diedrichsen, 2019)

Common Pitfalls

1. Information Leakage (The Most Common Error)

Feature selection, z-scoring, PCA, or any data-driven preprocessing on the full dataset before cross-validation splitting will leak information from test folds into training, inflating accuracy. ALL such steps must occur WITHIN each fold (Kriegeskorte et al., 2009; Varoquaux et al., 2017).

2. Confounds Driving Decoding

Decoding "success" may reflect confounds rather than neural representations:

• Eye movements: Systematic gaze differences between conditions produce decodable EOG artifacts in EEG and BOLD signal differences near the eyes in fMRI
• Reaction time differences: Conditions with different RTs produce different motor preparation signals
• Head motion: Differential motion across conditions (e.g., speech vs. rest) introduces confounded spatial patterns
• Run order: If conditions are blocked or ordered within runs, slow signal drifts can drive classification

3. Interpreting Accuracy Magnitude

• Decoding accuracy reflects information accessible to the classifier, not the amount of neural information (Haynes, 2015)
• Low accuracy does not mean low information (features may be nonlinearly coded)
• High accuracy does not imply the region is the origin of the representation
• Never compare accuracy magnitudes across regions with different voxel counts, SNR, or dimensionality without controlling for these factors

4. Double-Dipping in Searchlight Analysis

Selecting an ROI based on significant searchlight clusters and then performing additional analyses on those clusters is circular (Kriegeskorte et al., 2009; Etzel et al., 2013). Use independent data or pre-registered ROIs for follow-up analyses.

5. Classifier Weights Are Not Activation Patterns

Raw SVM or regression weights do NOT indicate which voxels/channels are most activated by a condition. They indicate which features are most useful for discrimination, which can include suppressing noise. To obtain neurophysiologically interpretable maps, transform weights into activation patterns using the method of Haufe et al. (2014).

6. Imbalanced Classes

Unequal trial counts across classes bias accuracy toward the majority class. Solutions:

• Undersample the majority class
• Use balanced accuracy (average of per-class recall)
• Stratify cross-validation folds

Minimum Reporting Checklist

Based on Haynes (2015), Varoquaux et al. (2017), and Grootswagers et al. (2017):

• Classifier type and hyperparameters (e.g., linear SVM, C=1)
• Cross-validation scheme (e.g., leave-one-run-out, 5-fold stratified)
• Feature space (number of voxels/channels, ROI definition or searchlight radius)
• Preprocessing steps performed WITHIN vs. OUTSIDE cross-validation folds
• Number of trials/samples per class per subject
• Statistical test for significance (permutation test with N permutations, or parametric test)
• Effect size or confidence intervals on accuracy
• For RSA: distance metric, model RDMs tested, noise ceiling
• For temporal generalization: window width, step size, cluster-correction parameters
• Software and version used

Key References

• Combrisson, E., & Jerbi, K. (2015). Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy. Journal of Neuroscience Methods, 250, 126-136.
• Etzel, J. A., Zacks, J. M., & Braver, T. S. (2013). Searchlight analysis: Promise, pitfalls, and potential. NeuroImage, 78, 261-269.
• Grootswagers, T., Wardle, S. G., & Carlson, T. A. (2017). Decoding dynamic brain patterns from evoked responses: A tutorial on multivariate pattern analysis applied to time series neuroimaging data. Journal of Cognitive Neuroscience, 29(4), 677-697.
• Haufe, S., et al. (2014). On the interpretation of weight vectors of linear models in multivariate neuroimaging. NeuroImage, 87, 96-110.
• Haynes, J.-D. (2015). A primer on pattern-based approaches to fMRI: Principles, pitfalls, and perspectives. Neuron, 87(2), 257-270.
• King, J.-R., & Dehaene, S. (2014). Characterizing the dynamics of mental representations: The temporal generalization method. Trends in Cognitive Sciences, 18(4), 203-210.
• Kriegeskorte, N., Goebel, R., & Bandettini, P. (2006). Information-based functional brain mapping. Proceedings of the National Academy of Sciences, 103(10), 3863-3868.
• Kriegeskorte, N., Mur, M., & Bandettini, P. (2008). Representational similarity analysis -- Connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 4.
• Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S., & Baker, C. I. (2009). Circular analysis in systems neuroscience: The dangers of double dipping. Nature Neuroscience, 12(5), 535-540.
• Kriegeskorte, N., & Diedrichsen, J. (2019). Peeling the onion of brain representations. Annual Review of Neuroscience, 42, 407-432.
• Maris, E., & Oostenveld, R. (2007). Nonparametric statistical testing of EEG- and MEG-data. Journal of Neuroscience Methods, 164(1), 177-190.
• Misaki, M., Kim, Y., Bandettini, P. A., & Kriegeskorte, N. (2010). Comparison of multivariate classifiers and response normalizations for pattern-information fMRI. NeuroImage, 53(1), 103-118.
• Nili, H., et al. (2014). A toolbox for representational similarity analysis. PLoS Computational Biology, 10(4), e1003553.
• Varoquaux, G., et al. (2017). Assessing and tuning brain decoders: Cross-validation, caveats, and guidelines. NeuroImage, 145, 166-179.
• Varoquaux, G. (2018). Cross-validation failure: Small sample sizes lead to large error bars. NeuroImage, 180, 68-77.
• Walther, A., et al. (2016). Reliability of dissimilarity measures for multi-voxel pattern analysis. NeuroImage, 137, 188-200.

See references/decoding-methods.md for detailed classifier comparisons, searchlight parameters, and software tools. See references/rsa-guide.md for a complete step-by-step RSA analysis workflow.