Chapter 3: Sampling Strategies for Small Studies

Sampling in small studies is less about chasing generic sample-size rules and more about matching design ambitions to the population you can realistically access. This chapter explains how to choose between probability and non-probability sampling, how to justify feasible sample sizes honestly, and how to think about power, precision, and adaptation when recruitment is constrained from the outset.

Learning Objectives

By the end of this chapter, you will be able to explain the trade-offs between probability and purposive sampling, select appropriate sampling methods given realistic resource and population constraints, calculate minimum detectable effects for a planned sample size, and justify sample sizes transparently in research proposals and reports.

The Tension Between Ideal and Feasible Sample Sizes

Many introductory guides rely on heuristics such as n of 30 or more per group for simple group comparisons or 10-15 events per predictor in regression. These are rough planning rules, not universal thresholds. In practice, resource constraints, rare populations, and ethical considerations often make even those targets unattainable. Rather than abandoning research in such contexts, we should adopt methods suited to smaller samples and report findings with appropriate caveats.

Transparent reporting of sampling rationale, achieved sample size, and power or precision estimates helps readers judge the strength of evidence. Researchers should distinguish between studies designed to test specific hypotheses (which require adequate power) and exploratory studies that generate hypotheses or provide preliminary effect estimates (which can proceed with modest samples).

Probability Sampling with Small Samples

Probability sampling (simple random sampling, stratified sampling, cluster sampling) ensures that every unit has a known, non-zero probability of selection. This supports generalisation to the target population and enables design-based inference. However, probability sampling requires a sampling frame and may be logistically complex or expensive.

With small samples, probability sampling can still be valuable, but estimates will have wide confidence intervals. Stratified sampling, which divides the population into strata and samples proportionally or disproportionately from each, can improve precision by ensuring representation of key subgroups. Probability sampling is most appropriate when a sampling frame is accessible, generalisability to the target population is a genuine aim, and resources permit systematic random selection even if the total sample remains modest.

Sequential and Adaptive Sampling

When recruitment is costly or uncertain, sequential designs allow researchers to review interim results and decide whether to continue sampling. For example, you might pre-specify that recruitment will proceed in waves of five participants, stopping early if credible intervals for the primary outcome are sufficiently narrow or if feasibility metrics (e.g., consent rates) fall below thresholds. Adaptive sampling can also target underrepresented strata after an initial wave, improving balance without committing to a large upfront sample. To keep this defensible, decision rules should be set in advance, error control should be maintained with methods appropriate for small n, and any adaptations should be documented transparently so readers can see how the design evolved.

Sequential or response-adaptive sampling is especially valuable in rare populations, where pausing after each wave prevents over-committing resources if early data already provide actionable evidence.

Example: Stratified Sampling Calculation

Suppose we are surveying employees in a small organisation with 120 total staff: 60 in Department A, 40 in Department B, 20 in Department C. We can afford to survey 30 employees. Proportional stratified sampling ensures each department is represented in proportion to its size.

NoteInterpretation

Proportional allocation ensures that each department contributes to the sample in proportion to its population size. Department A, being the largest, provides 15 respondents, while Department C, the smallest, provides 5. If employees are then selected randomly within each department, this approach yields unbiased estimates for the overall population. If precision for small strata is a concern, disproportionate allocation (oversampling small strata) can be used, though this requires weighting in analysis.

Purposive and Convenience Sampling

Purposive (judgmental) sampling selects units based on researcher judgement of their informativeness or representativeness. Convenience sampling selects units that are easily accessible. Neither method supports probabilistic generalisation, but both are common in small-sample research where probability sampling is infeasible.

Findings from purposive or convenience samples should be interpreted cautiously and presented as preliminary or context-specific. Replication in independent samples strengthens confidence. These approaches are most appropriate when no sampling frame is available, the work is exploratory or pilot-focused, the population is rare or hard to reach, or the available resources are tightly constrained.

Quota Sampling

Quota sampling (a form of purposive sampling) selects units to match known population characteristics (such as age, gender, or occupation distribution). It mimics stratified sampling but without random selection within strata. Quota sampling can improve representativeness compared to convenience sampling, though it remains non-probabilistic. It is most useful when the researcher knows which population characteristics should be balanced but probability sampling is infeasible.

For example, suppose a student satisfaction study can recruit only 30 participants from a programme where the student body is 60% first-year, 25% second-year, and 15% final-year students. A quota plan could recruit 18 first-year, 8 second-year, and 4 final-year students. That improves coverage compared with accepting the first 30 volunteers, but it still does not justify ordinary probability-based confidence intervals unless participants are randomly selected within each quota cell.

Year group Population share Quota for n = 30
First year 60% 18
Second year 25% 8
Final year 15% 4

The report should therefore state the quota variables, the target quotas, the achieved quotas, and the recruitment procedure. Findings should be framed as balanced descriptive evidence rather than population estimates with known sampling error.

Snowball sampling is another non-probability approach for hard-to-reach populations. Initial participants refer other eligible participants, creating a sample through social or professional networks. This can be appropriate for rare populations or sensitive topics, but the findings should be interpreted as network-specific because people outside the referral chains had little or no chance of inclusion.

Power and Precision with Small Samples

Statistical power is the probability of detecting a true effect of a given size. With small samples, power is limited, meaning that even if a meaningful effect exists, the study may fail to detect it (high Type II error rate). Researchers should conduct power analyses before data collection to understand what effects are detectable given sample size constraints.

If the achieved sample size is smaller than desired, report the minimum detectable effect (MDE): the smallest effect the study can detect with specified power (typically 80%) and alpha (typically 0.05). This helps readers judge whether the study could have detected effects of practical importance.

Finite Population Correction

When sampling without replacement from a small, known population, the variance of estimates decreases because each sampled unit reduces remaining uncertainty. The finite population correction (FPC) captures that reduction and can lower the required sample size.

In this example, the infinite-population calculation gives a required sample size of 30, but the accessible population is only 120 people. Applying the FPC reduces the required sample size to about 24, as Table 3.2 shows.

NoteInterpretation

Sampling without replacement from 120 individuals means that a sample of roughly 24 (instead of 30) achieves the same precision. Always report whether you applied the FPC so readers can replicate the calculation.

Example: Power Calculation for a Small Study

We plan a study comparing two groups with n = 12 per group. Under a two-sample t-test, the power to detect a medium effect size (Cohen’s d = 0.5) is only about 0.22. Reaching 80% power with this design would require sensitivity to a much larger effect, about d = 1.20, as Table 3.3 shows.

NoteInterpretation

With 12 participants per group, power to detect a medium effect (d = 0.5) is low, at about 22%. To achieve 80% power, the study would need to be sensitive to a much larger effect (about d = 1.2). This illustrates the limitation of small samples for hypothesis testing. If the true effect is small or medium, the study is underpowered. Researchers should acknowledge this limitation and interpret non-significant results cautiously.

Sample Size Planning Workflow

Integrating power analysis into a broader planning conversation prevents unrealistic promises and surfaces design trade-offs early. Use the following workflow whenever you scope a small-sample study:

  1. Clarify the question and estimand. Identify the exact parameter the study needs to estimate, whether that is a difference in means, an odds ratio, a correlation, or some other target quantity.
  2. Specify tolerable uncertainty. Define the minimum detectable effect or target confidence-interval width that would make the study actionable in substantive terms.
  3. Map constraints. Document the recruitment limits, budget, timeline, and ethical restrictions that bound what the design can realistically support.
  4. Select design and analysis. Choose the planned test or model, decide on one- versus two-sided inference, and note any covariates, matching, or repeated-measures structure.
  5. Compute required n. Use analytical power formulas, simulation, or resampling as appropriate, and apply finite-population corrections if sampling without replacement from a known population.
  6. Assess feasibility. Compare the required n with the constraints. If the target is infeasible, revise the design, narrow the claim, or shift the emphasis toward estimation or sequential decision-making.
  7. Document decisions. Record the assumptions, software, code, and compromises so that readers can see exactly how the final sampling plan was chosen.

Figure 3.1 summarises the planning sequence and the point at which feasibility constraints feed back into design choices.

Figure 3.1: Sample size planning workflow for small studies.

Figure 3.1 makes the trade-off loop explicit: if the required sample size exceeds what is feasible, researchers can justify an exploratory framing, add interim analyses, or negotiate for additional resources before data collection begins.

Justifying Small Sample Sizes

When sample sizes are constrained, the researcher’s obligation is to be transparent rather than defensive. That means stating clearly both the target and accessible populations, describing the sampling method and the reasoning behind it, and reporting the planned and achieved sample sizes alongside power or precision estimates. If the study cannot detect effects of practical importance, that limitation should be acknowledged directly, and the findings should be framed as preliminary or exploratory when that description is accurate.

Key Takeaways

Sampling in small studies is ultimately about aligning design ambitions with the population you can realistically reach. Probability sampling remains valuable when generalisation is the goal, but purposive, convenience, and quota approaches are often the feasible options in constrained settings and should be described honestly rather than overstated. Across all of these designs, transparent reporting of the sampling rationale, achieved sample, and detectable effect sizes is what allows readers to judge how much weight the findings can bear.

Self-Assessment Quiz

Test your understanding of sampling strategies from Chapter 3.

Question 1. A researcher uses a “rule of thumb” of n=30 per group for all studies. What is the primary problem with this approach?

Explanation.

Sample size should depend on effect size, power, and research question—not arbitrary rules. A small effect requires larger n; a large effect can be detected with smaller n. The chapter emphasizes: “Rather than abandoning research in such contexts, we should adopt methods suited to smaller samples and report findings with appropriate caveats.”

Question 2. Stratified sampling is most useful when:

Explanation.

Stratified sampling divides the population into strata and ensures each stratum is represented. This improves precision when strata differ on the outcome. The chapter states: “Stratified sampling (dividing the population into strata and sampling proportionally or disproportionally from each) can improve precision by ensuring representation of key subgroups.”

Question 3. Power analysis reveals you need n=50 per group, but only n=20 is feasible. What should you do?

Explanation.

Proceed, but report the study as exploratory/pilot and calculate minimum detectable effect (MDE). Transparency about power limitations is essential. The chapter recommends: “If the achieved sample size is smaller than desired, report the minimum detectable effect (MDE).”

Question 4. Which sampling method allows probabilistic generalization to a target population?

Explanation.

Simple random sampling (and other probability sampling methods) ensures every unit has a known, non-zero probability of selection, supporting generalization. The chapter states: “Probability sampling…ensures that every unit has a known, non-zero probability of selection. This supports generalisation to the target population.”

Question 5. Quota sampling differs from stratified sampling in that:

Explanation.

Quota sampling matches population proportions but does not use random selection within strata. It “mimics stratified sampling but without random selection within strata.” This makes it non-probabilistic.

Question 6. A study with n=15 per group has about 25% power to detect d=0.5. The researcher should report:

Explanation.

With only about 25% power for d=0.5, the study is underpowered for medium effects. With n=15 per group, only very large effects, around d = 1.06 or larger, are close to the 80% power target. The chapter emphasizes transparent reporting: “Researchers should conduct power analyses before data collection to understand what effects are detectable.”

Question 7. The finite population correction (FPC) is relevant when:

Explanation.

FPC adjusts required sample size when sampling without replacement from a small, finite population. The chapter explains: “When sampling without replacement from a small, known population, the variance of estimates decreases…The finite population correction (FPC) adjusts the required sample size accordingly.”

Question 8. Sequential sampling allows researchers to:

Explanation.

Sequential sampling allows stopping early based on pre-specified decision rules if interim results show sufficient precision or evidence. The chapter describes: “sequential designs allow researchers to review interim results and decide whether to continue sampling.”

Question 9. A convenience sample from one university is used to test a new teaching method. Which statement is TRUE?

Explanation.

Convenience samples are context-specific and require replication. The chapter states: “Findings from purposive or convenience samples should be interpreted cautiously and presented as preliminary or context-specific. Replication in independent samples strengthens confidence.”

Question 10. Minimum Detectable Effect (MDE) refers to:

Explanation.

MDE is the smallest effect the study can detect with specified power (typically 80%) and alpha (typically 0.05). The chapter defines it: “the minimum detectable effect (MDE): the smallest effect the study can detect with specified power.”