The same checklist used by epidemiologists to catch fatal design flaws before they reach peer review. Covers RCTs, cohort, case-control, and quasi-experimental designs.
Enter your email to unlock the full checklist instantly.
No spam. Just research methodology resources.
✓ Unlocked! Scroll down.
Before touching data or running a single test.
Population, Intervention/Exposure, Comparator, Outcome — each unambiguous.
ATE, ATT, CATE, or marginal effect? This determines your entire analysis strategy.
Are you making a causal claim? If yes, you need an identification strategy. If no, don't use causal language.
Even for observational studies — what RCT would you run if you could? This exposes design gaps.
Data availability, sample size, timeline, ethics approval, budget. Kill unfeasible designs early.
The architecture of your study.
RCT, cohort, case-control, cross-sectional, quasi-experimental — and why this one over alternatives.
When does follow-up begin? Misalignment = immortal time bias.
Consistency assumption: same label → same intervention? "Statin use" vs "atorvastatin 40mg for ≥6 months" are different studies.
Active comparator vs. no treatment vs. standard of care — each answers a different question.
List confounders, mediators, colliders. If you haven't drawn a DAG, you don't know what to adjust for.
Every covariate stratum has both treated and untreated. Violations break propensity-based methods.
Name the confounders you can't measure. Plan E-value, bias analysis, or bounding approach.
Garbage in, garbage out — but methodically.
ICD codes? Self-report? Lab values? What's the sensitivity/specificity of your outcome definition?
Detection bias: knowing the outcome shouldn't change how you measure the exposure (and vice versa).
MCAR, MAR, MNAR? This determines whether multiple imputation, IPW, or sensitivity analysis is appropriate.
Based on clinically meaningful effect size, not "what we can detect with our data."
Long enough to observe outcome? Short enough to maintain retention? Time horizon matches the question.
Claims ≠ clinical reality. EHR ≠ complete medical history. Be explicit about what your data can and cannot capture.
Statistics serve the design, not the other way around.
Why this method? How does it handle the specific threats you identified? Don't default to logistic regression.
If patients can die before the event, Kaplan-Meier overestimates cumulative incidence. Use CIF or cause-specific hazards.
At minimum: different confounder sets, alternate outcome definitions, subgroup analyses, E-value for unmeasured confounding.
Pre-specify primary outcome. Secondary outcomes are hypothesis-generating. Don't p-hack — it shows.
Subgroup analysis ≠ stratified adjustment. Are you looking for who benefits differently, or trying to remove bias?
CONSORT, STROBE, RECORD, PRISMA — choose before writing, not after. Reviewers check.
What could still go wrong?
Selection bias, information bias, confounding — specific to YOUR design, not a generic list.
Who does your finding apply to? Single-center academic hospital ≠ community practice.
What will Reviewer 2 attack? Prepare defenses now. If you can't defend the design, redesign it.
SchemaForge runs this entire analysis on your research aim in under 20 minutes. Power calculations, DAGs, and reviewer rebuttals included.
Try It Free →