# Independent reproduction and discrepancy court

Make artifacts portable, reruns independent, and disagreements diagnosable before declaring replication or failure.

## Instructor edition

## 1. Reconstruct the dependency chain

**Task type:** derivation

Starting from a claimed result, derive the full dependency graph from raw observation through calibration, preprocessing, model, statistic, and figure. Identify every parameter that could change the conclusion.

### Deliverables

- Executable-style dependency graph
- Parameter and version manifest
- Recomputed result with checksum trail

### Scoring criteria

- Graph is complete and ordered: 8 points
- Versions and parameters are recoverable: 7 points
- Recalculation matches or explains mismatch: 5 points

### Solution outline

- Trace the headline result backward to raw inputs without skipping manual transformations.
- Record software, calibration, random seeds, and inclusion rules.
- Classify any mismatch before attributing it to scientific disagreement.

## 2. Quantify the discrepancy

**Task type:** analysis

Compare original and independent results using uncertainty-aware effect estimates and a structured discrepancy model. Evaluate protocol, apparatus, environment, data, and analysis explanations rather than treating replication as a binary vote.

### Deliverables

- Comparable effect-size table
- Discrepancy likelihood or decision matrix
- Sensitivity to reasonable analysis variants

### Scoring criteria

- Comparability and uncertainties: 7 points
- Competing discrepancy explanations: 7 points
- Robustness and calibrated conclusion: 6 points

### Solution outline

- Harmonize estimands before comparing numerical results.
- Test prespecified discrepancy categories with available evidence.
- Use replicated, not replicated, contradicted, and indeterminate only under explicit criteria.

## 3. Stage a cross-lab swap and adjudication

**Task type:** design

Design a two-laboratory replication in which teams exchange data, code, and where practical apparatus or calibration artifacts. Preserve independent analysis, then use a frozen adjudication process for disagreements.

### Deliverables

- Swap matrix and contamination safeguards
- Independent-analysis and reveal sequence
- Discrepancy remediation and public release plan

### Scoring criteria

- Independence survives the swap: 7 points
- Reveal sequence localizes discrepancies: 7 points
- Release plan includes negative and indeterminate outcomes: 6 points

### Solution outline

- Run own-data/own-code, swapped-data, swapped-code, and shared-calibration cells where feasible.
- Freeze primary outputs before cross-team discussion.
- Publish the matrix of outcomes so readers can see where agreement changes.

---

Evidence rule: distinguish calculation, model-dependent inference, experimental observation, and unresolved claim in every response.