# Instructor guide: Blinded residual and search-penalty analysis

Course: Experimental design, inference, and reproducibility

Suggested time: 55–75 minutes

## Learning target

Learners treat blinding, correlated uncertainty, injection recovery, and multiplicity correction as one evidence pipeline rather than optional post hoc cautions.

## Prepare

- Define the analysis decision before revealing the modeled result.
- Prepare one failed-injection case.
- Distinguish local from family-wise probability.

## Facilitation moves

- Ask which choices were made before unblinding.
- Do not allow trial count to erase shared systematics.
- Require a decision statement even for an interesting residual.

## Misconception checks

- **Enough repeated trials eliminate every uncertainty.** Independent noise averages down; correlated bias and model error require separate controls or bounds.
- **A small local p-value proves the preferred mechanism.** Search multiplicity changes surprise, and mechanism attribution requires discriminating predictions and controls.

## Accessibility and participation

- Translate probabilities into frequencies without overstating certainty.
- Provide the decision tree in text and diagram form.
- Allow spreadsheet or calculator support for uncertainty combination.

## Evidence of learning

- A frozen analysis decision
- A correct correlated-error explanation
- An injection and multiplicity-aware conclusion

## Extension

Design a preregistered two-laboratory replication with a shared injection protocol and independent analysis teams.

## Evidence boundary

Assess the learner's reasoning only within the declared model and recorded observations. Do not upgrade a simulation result into a claim about an unmodeled physical system.