Blog
The Problem With Changing Everything at Once: Confounding and Factorial Design
Imagine you change your diet, your sleep schedule, your training and two other things at the same time, and then something improves. What caused it? There is no way to know, and this situation, called confounding, is the single most common reason that combination claims cannot be believed.
Key takeaways
- Confounding means two or more factors are tangled so their effects cannot be separated.
- Changing one factor at a time is clear but ignores interactions.
- A 2×2 factorial design tests A, B and A plus B together with a control.
- Order effects and timing can also confound.
What confounding looks like
Suppose group 1 gets compound A plus compound B and is compared with an untreated group. Any difference could be due to A, to B, to their combination, or to something else that differs between groups. The comparison does not isolate anything. The same logic applies to bundles and blends: without single-component arms, a result belongs to the bundle as a whole.
One factor at a time
The classic fix is to change one variable at a time while holding the rest constant. It gives clean answers for each factor but has two weaknesses: it needs many experiments, and it cannot detect interactions, where the effect of A depends on whether B is present.
The 2×2 factorial design
A factorial design crosses factors. With two compounds, there are four groups:
- Control (neither).
- A alone.
- B alone.
- A plus B.
The data show the effect of A, the effect of B and whether the combined effect differs from the sum of the two, which is the interaction. Because each animal contributes information about both factors, factorial designs are often more efficient than running two separate experiments. They are also the natural foundation for the models in synergy and additivity.
Other sources of confounding
- Time and order. If all treated animals are tested in the morning and controls in the afternoon, time of day is confounded with treatment (see circadian timing).
- Housing and cage effects. Animals in one cage share conditions, so treat the cage as a unit.
- Batch and lot. Different lots of a reagent between groups.
- Experimenter. Different handlers for different groups.
Tools that break confounding
- Randomization spreads unknown factors evenly across groups.
- Blocking groups similar subjects, then randomizes within blocks.
- Counterbalancing varies the order of treatments across subjects.
- Blinding prevents measurement bias.
Applying it to personal experiments
The same principles explain why self-experiments with several simultaneous changes cannot be interpreted. Our post on placebo and regression to the mean goes through this from the personal side.
Cost of the design
A factorial design with two factors needs four groups, and the groups must be large enough to detect the effect sizes you care about (see statistical power). Good design costs animals and time, but it is cheaper than a result nobody can interpret.
The bottom line
If you want to know what each part of a combination does, you need an experiment in which each part appears alone. Everything else is a story.
Related reading
- Does 1 + 1 Equal 2? Synergy, Additivity and How Combinations Are Tested
- Why Self-Experiments Mislead: Placebo, Regression to the Mean and the n-of-1
- Statistical Power and Sample Size: Why Tiny Peptide Studies Mislead
- Browse the full Research Library
Explore Lucid Aminos
- Shop all research compounds
- Read the lab reports (COA)
- Monthly Research Panel
- Frequently asked questions
- Partner program
- About Lucid Aminos
- Contact us
For laboratory research use only. Not for human or veterinary use, and not intended to diagnose, treat, cure or prevent any condition. This article is educational and is not medical advice.