How to read a paper

A practical, evidence-based guide for early-career trainees — useful well beyond this club. Reading primary literature is a skill you build deliberately, not a talent you either have or lack. Here is how experienced readers actually do it, why it feels hard at first, and how to leave annotations that help the next reader and the COST Action.

⛔ No AI

Read and annotate the paper yourself. The point is your learning and your judgement — precisely what AI cannot do for you. For Journal Club reviews, a no-AI declaration is required and AI-written reviews will not score. See How to review for the full rules.

Why this feels hard at first (and that's normal)

If the Methods and Results sections are the parts you find hardest, you are not doing it wrong — you are exactly where most trainees are. A survey of 260 students and researchers found that skill in reading the Results section develops slowly across an entire academic career, that inexperienced readers find Methods and Results the most difficult, and that they tend to undervalue the Results and the critical interpretation of data.[3] Experienced readers, by contrast, do not read front-to-back — they move around a paper strategically. The good news: this is a trainable skill, and structured practice is what closes the gap.[3][4]

Read in passes, not linearly

The single most useful habit is to stop reading top-to-bottom. A widely used approach is the three-pass method:[1]

Pass 1 — triage (~5–30 min). Read in this order: title → abstract → figures → methods overview → conclusions. Ask the four questions: what problem, what data, what processing pipeline, what claim? You are mapping the skeleton before annotating details — and deciding whether the paper is worth a deeper pass.
Pass 2 — grasp the content (~1 h). Read carefully but do not get stuck on every derivation. Study the figures and tables — this is where results actually live. Note what you understand and, just as importantly, what you don't.
Pass 3 — read like you'd re-run it. Reconstruct the work in your head: could you reproduce the pipeline from what's written? Challenge each assumption. This is the pass that turns reading into critical appraisal.

Read the methods as a pipeline

Most papers in this pool describe or evaluate a preprocessing pipeline. Read the methods as exactly that — a sequence of stages — and for each stage ask what it does, what it assumes, and how it was evaluated:

Inputs: modality, acquisition, sample, data standard (BIDS?).
Steps: each preprocessing stage — what it does, why, what it assumes.
Parameters & defaults: what was chosen, what was left at default, what was justified.
Outputs & evaluation: derivatives produced; how quality/validity was assessed.

Reading figures, results & statistics

The Results are the part trainees skip and experts scrutinise first.[3] Don't take a figure's caption on trust — interrogate it: What is actually plotted? What are the axes, the units, the sample size? Do the figures support the claim in the text, or a weaker version of it? What is not shown? Treating each figure as a small hypothesis test — asking what result would have changed the authors' conclusion — is the core of figure-focused reading pedagogies such as the CREATE method.[2]

Check reproducibility

For a preprocessing paper, reproducibility is the science. Ask: is code, containerisation, or versioning available? Could you re-run it? Does it use or extend community standards such as BIDS? Transparency and reproducibility are increasingly recognised as first-class criteria when appraising neuroimaging work,[5] and they sit at the centre of the INDoS mission — so a paper's reproducibility is always worth an explicit annotation.

Mark what matters

The goal is not to summarise a paper — it is to read it critically and leave annotations that help the next reader. Good readers don't stop at "what did they do"; they ask "what would I do next?"[2] As you read, mark:

What you did not understand. Located confusion ("why is the field map applied before, not after, motion correction?") is far more valuable — to you and to us — than a vague "confusing". Honest "I didn't get this" notes are a strength, not a weakness: they are the single most useful signal for building training materials.
What has been contested, corrected, or superseded since publication — and the newer work, if you know it.
What matters for INDoS — the data-sharing, standardization, and reproducibility angle.
Your own judgement — strengths worth carrying forward, weaknesses, and what you would do differently. Your voice counts.

A one-glance checklist

Triaged first: title → abstract → figures → methods → conclusions.
Identified the problem, the data, the pipeline, and the central claim.
Read the methods as a pipeline (inputs, steps, parameters, evaluation).
Interrogated the figures and results — not just the caption.
Checked reproducibility: code, containers, versioning, BIDS — could you re-run it?
Annotated across the whole paper, anchored to exact passages.
Flagged what you didn't understand, and what's been contested or superseded.
Noted the INDoS angle and your own judgement.
Did it all yourself — no AI.

References

[1] Keshav S. (2007). How to read a paper. ACM SIGCOMM Computer Communication Review 37(3):83–84. https://doi.org/10.1145/1273445.1273458
[2] Hoskins SG, Stevens LM, Nehm RH. (2007). Selective use of the primary literature transforms the classroom into a virtual laboratory. Genetics 176(3):1381–1389. https://doi.org/10.1534/genetics.107.071183
[3] Hubbard KE, Dunbar SD. (2017). Perceptions of scientific research literature and strategies for reading papers depend on academic career stage. PLoS ONE 12(12):e0189753. https://doi.org/10.1371/journal.pone.0189753
[4] Goudsouzian LK, Hsu JL. (2023). Reading primary scientific literature: approaches for teaching students in the undergraduate STEM classroom. CBE—Life Sciences Education 22(3):es4. https://doi.org/10.1187/cbe.22-10-0211
[5] Klapwijk ET, van den Bos W, Tamnes CK, Raschle NM, Mills KL. (2021). Opportunities for increased reproducibility and replicability of developmental neuroimaging. Developmental Cognitive Neuroscience 47:100902. https://doi.org/10.1016/j.dcn.2020.100902