Sampling and sample size
Sampling Methods, Sample Size, Power and Type I and II Errors
Random, systematic, stratified, cluster and non-probability sampling, sample size, drop-out adjustment, power, and type I and type II errors. Explained with the FCPS scenarios that test each method, plus intention-to-treat analysis and the link between sample size and standard error.
Key terms
- Population: the whole group of interest. The target population is who you want to draw conclusions about; the study (accessible) population is who you can actually reach.
- Sample: the subset studied. Sampling frame: the list of all units from which the sample is drawn (electoral roll, household register, clinic list). Sampling unit: individual, household, village or school.
- Parameter (true population value, such as μ) versus statistic (sample value, such as x̄).
- Sampling error: chance difference between a sample statistic and the population parameter; it shrinks as the sample grows. Sampling bias is systematic and does not shrink with size.
Probability (random) sampling
- Every unit has a known, non-zero chance of selection, so results can be generalised and sampling error calculated.
| Method | How it works | When and why |
|---|---|---|
| Simple random | Every unit has an equal chance (random numbers, computer, lottery) | Needs a complete frame; may by chance miss small subgroups |
| Systematic | Every kth unit after a random start (k = N ÷ n), such as every 10th patient | Easy in clinics and streets; biased if the list has a repeating pattern |
| Stratified | Split the population into homogeneous strata (age, sex, urban/rural), then sample randomly within each | Guarantees each subgroup is represented; more precise |
| Cluster | Randomly choose groups (villages, schools), then study all or some members | Cheap and practical when no list of individuals exists; larger sampling error (design effect) |
| Multistage | Random selection at several levels (districts, then villages, then households) | National surveys such as DHS |
| Multiphase | Basic data from everyone, more detail from a subsample | Screening surveys |
- The WHO EPI 30-cluster survey (30 clusters × 7 children) estimates immunisation coverage quickly.
- Stratified versus cluster: strata are internally similar and all strata are sampled; clusters are internally mixed (each a mini-population) and only some clusters are chosen.
Exam trap Stratified sampling is chosen for representation of subgroups; cluster sampling for convenience and cost when no list of individuals exists.
Non-probability sampling
| Method | Description |
|---|---|
| Convenience | Whoever is available (patients on today's ward); prone to selection bias |
| Consecutive | Every eligible patient over a set period; better than convenience |
| Quota | Fixed numbers from subgroups, chosen non-randomly within them |
| Purposive (judgemental) | Researcher picks information-rich cases; used in qualitative research |
| Snowball | Participants recruit others; reaches hidden populations (drug users, sex workers) |
- Results from non-probability samples cannot be generalised with confidence because selection chances are unknown.
Hypothesis testing errors
- The null hypothesis says there is no difference. A study can make two kinds of mistake:
| Null hypothesis true (no real difference) | Null hypothesis false (real difference) | |
|---|---|---|
| Study finds a difference (rejects null) | Type I error (α): false positive | Correct: power (1 − β) |
| Study finds no difference | Correct | Type II error (β): false negative |
- α is set before the study, usually 0.05; it is the risk of claiming an effect that is not there.
- β is usually 0.2 or 0.1; power = 1 − β = the probability of detecting a real difference of the chosen size, conventionally 80% or 90%.
- Small studies are prone to type II error: a non-significant result from an underpowered study is absence of evidence, not evidence of absence.
Remember Type I sees an effect that is not there; type II misses an effect that is.
What decides sample size
- A larger sample is needed when:
- the effect size to detect is smaller;
- variability (SD) is greater;
- α is stricter (0.01 rather than 0.05);
- power wanted is higher (90% rather than 80%);
- a cluster design is used (multiply by the design effect), several subgroups will be analysed, or drop-outs are expected.
- For a prevalence survey: n = Z² × p(1 − p) ÷ d², where Z = 1.96 for 95% confidence, p = expected prevalence and d = acceptable margin of error.
- Sample size and precision: SE = SD ÷ √n, so precision improves only with the square root of n. Halving the standard error requires four times the sample.
- Adjusting for drop-outs: divide the required number by the proportion expected to remain. Needing 200 completers with 20% loss: 200 ÷ 0.8 = 250 to recruit. Adding 20% (240) leaves you short, because 20% of 240 is 48 and only 192 remain.
- Power is calculated before the study (a priori); post hoc power adds nothing once results are known.
- Power increases with larger samples, larger effects, less variability, a more lenient α and one-tailed tests.
Exam trap To allow for drop-outs divide by the retention rate; do not just add the percentage.
Randomisation and analysis in trials
- Allocation methods: simple, block (keeps group sizes equal), stratified (balances key prognostic factors), cluster randomisation and minimisation.
- Allocation concealment (central telephone or web system, sealed opaque envelopes) stops recruiters predicting the next assignment.
- Intention-to-treat analysis: every participant is analysed in the group to which they were randomised, whether or not they took the treatment, switched or dropped out. It preserves the balance created by randomisation and gives a realistic, conservative estimate. Per-protocol analysis includes only compliers and can overstate benefit.
Sampling and measurement quality
- A representative sample protects external validity; good measurement protects internal validity.
- Reliability (precision, repeatability) means a test gives the same result when repeated on the same subject, by the same or different observers. Validity (accuracy) means it measures what it claims to measure. A test can be reliable but not valid.
How FCPS asks about this
- Every 10th patient after a random start: systematic sampling.
- Villages selected at random and all children in them studied: cluster sampling; WHO 30 × 7 survey.
- Population divided by age group and each sampled randomly: stratified sampling.
- Recruiting injecting drug users through other users: snowball sampling.
- Power: 1 − β, usually 80%; type II error is missing a real difference.
- 200 needed with 20% expected drop-out: recruit 250 (200 ÷ 0.8).
- Intention-to-treat analysis: participants analysed in the group they were randomised to.
Test yourself on this topic
5 of the 25 questions in our bank on this topic. Choose an answer to see why every option is right or wrong.
A study must estimate prevalence separately in urban and rural areas, where rural households are fewer. Which method ensures both groups are adequately represented?
Answer: Stratified sampling by area
In stratified random sampling the population is divided into non-overlapping strata (urban/rural, age group, sex), and a random sample is drawn from each. It guarantees representation of small subgroups, allows separate estimates for each stratum, and increases precision when strata differ. Allocation may be proportional or disproportionate (oversampling a small stratum, then weighting).
Stratified sampling
- Divide into strata, sample each
- Ensures small groups are included
- Can oversample then weight
Remember Stratify so no group is left out.
Why the other options are wrong
- Simple random sampling A small group may be under-represented by chance.
- Convenience sampling at clinics Biased, not representative.
- Snowball sampling For hidden populations.
- Cluster sampling of one district May miss one setting entirely.
What does the reliability of a test describe?
Answer: Same result when repeated
Reliability (precision, reproducibility) is consistency: getting the same result on repeated measurements. Validity (accuracy) is whether the test measures the true value. A test can be reliable but not valid, like a weighing scale that is consistently 2 kg out.
Reliable vs valid
- Reliable = repeatable (precision)
- Valid = true (accuracy)
- Dartboard: tight cluster off-centre = reliable, not valid
Remember Reliable = Repeats. Valid = Very true.
Why the other options are wrong
- Measures what it is meant to This is validity (accuracy).
- Proportion of true positives found This is sensitivity.
- Proportion of true negatives found This is specificity.
- Chance of disease if positive This is positive predictive value.
Which change requires a larger sample size in a trial?
Answer: A smaller target difference
Sample size increases when the effect to detect is smaller, outcome variability (SD) is greater, required power is higher (lower β), and α is smaller (stricter significance). Dropouts also require inflating the sample.
Sample size
- ↑ sample if small effect, high variability
- ↑ sample if high power, low α
- Allow for dropouts
Remember Small effects need big studies.
Why the other options are wrong
- A higher type I error (0.10) Reduces required sample.
- Lower power (60%) Reduces required sample.
- A larger expected effect Reduces required sample.
- Less outcome variability Reduces required sample.
Forty villages are randomly allocated so that half receive chlorinated piped water, and diarrhoea rates are compared. What type of study is this?
Answer: Cluster (community) trial
The unit randomised is the village, not the person, and the intervention is assigned by the investigators, so this is a cluster or community randomised trial. This design suits interventions delivered to whole groups (water, health education, mass treatment). A field trial randomises healthy individuals, for example to a vaccine. An ecological study only compares existing group data without assigning anything.
Types of trial
- Patients: clinical trial
- Healthy individuals: field trial
- Whole groups: community (cluster) trial
- Each gets both: crossover
Remember Randomise villages, run a community trial.
Exam trap Cluster trials need larger samples because people within a cluster are alike.
Why the other options are wrong
- Ecological study Nothing is assigned in an ecological study.
- Field trial of individuals Here villages, not individuals, are randomised.
- Prospective cohort study The exposure is assigned, not observed.
- Crossover trial Each group does not receive both treatments.
A trial concludes a new drug is no better than placebo when it actually is. What error has occurred, and what reduces it?
Answer: Type II error; increase power
Type II error: failing to reject a false null hypothesis (false negative), probability β; power = 1 − β (usually 80–90%). It is reduced by larger samples, larger effect sizes and less variability. Type I error: rejecting a true null (false positive), probability α (usually 0.05).
Type I vs type II
- Type I (α): false positive
- Type II (β): false negative
- Power = 1 − β; ↑ with sample size
Remember Type II: you missed a real effect.
Why the other options are wrong
- Type I error; lower α That is a false positive.
- Systematic error; randomise Not the description.
- Confounding; matching Not the description.
- Selection bias; blinding Not the description.
Where this sits in the CPSP syllabus
- Paper 1: Biostatistics, Sampling methods and sample size; power