Quick answer: Changing more than one variable does not automatically make an experiment invalid. A deliberately designed multi-variable experiment can be appropriate where several factors or interactions matter, but changing several things ad hoc can make it impossible to tell what caused the result. Where an outcome cannot be known or determined in advance by a competent professional on the basis of current knowledge, information or experience, the experimental design should make clear what is being varied, what remains comparable and how the observations will test the hypothesis.
Multiple changes are not the problem — unclear causality is
A common rule of thumb in technical development is:
Change one thing at a time.
It is useful advice.
It is not a universal scientific rule.
Australian Government guidance uses one-variable-at-a-time testing as a simple explanatory example, but it also recognises that experiments can take many different forms. The relevant question is whether the design can test the hypothesis and support a reasoned evaluation of the effects and interactions being investigated.
Imagine a manufacturing team trying to eliminate instability in a new process. During one trial, it increases temperature, changes feed rate and modifies airflow. The output improves substantially.
The trial has produced an encouraging result.
But what has it established?
Perhaps the higher temperature caused the improvement.
Perhaps airflow was responsible.
Perhaps neither variable mattered individually, but the combination did.
Perhaps feed rate changed the residence time enough to alter the entire mechanism.
Or perhaps another uncontrolled condition changed at the same time.
The difficulty is not that three variables moved.
The difficulty is that the test may not allow the team to distinguish among those explanations.
Current Australian Government guidance describes an experiment as a scientific procedure undertaken to test a hypothesis. It states that records should explain how the experiment was conducted or planned and may, for example, describe the parameters varied, held constant, observed or measured. Evaluation should then consider the causal relationships between the parameters varied, held constant, tested or measured. business.gov.au
That is the important standard.
The experimental design has to support the conclusion being drawn from it.
A project does not necessarily need to isolate every variable completely. In real engineering systems, that may be impossible or scientifically unhelpful.
Some technical behaviours arise precisely because variables interact.
A material may only fail when temperature and loading rate change together.
A machine-learning model may only become unstable when data sparsity and class imbalance occur simultaneously.
A process may perform adequately at high temperature or high throughput independently, but become unstable when both conditions occur together.
In those cases, deliberately varying more than one factor can be part of a sound research design.
The distinction is between planned interaction testing and unstructured simultaneous change.
Start with the mechanism, then decide what needs to vary
The number of variables should follow from the research question.
Suppose a food manufacturer is investigating why a reformulated product loses texture after processing.
One approach is to change the ingredient ratio and see whether texture improves.
That is a simple, interpretable test if the hypothesis concerns that ingredient alone.
But suppose prior observations suggest the ingredient only behaves differently above a certain processing temperature.
Now the research question involves an interaction.
Holding temperature fixed throughout the investigation could hide the mechanism the team actually needs to understand.
A stronger design may deliberately compare combinations of ingredient level and processing temperature.
The important step happens before the runs begin.
The team should decide what relationship it is testing and what evidence would support or weaken that explanation.
As a practical experimental-design framework, a useful plan should make several things explicit:
• which technical outcome remains uncertain;
• which variables are believed to influence that outcome;
• whether the hypothesis predicts an individual effect or an interaction between variables;
• which conditions need to remain sufficiently comparable;
• what will be measured;
• and what result would change the next technical decision.
This is different from changing several settings because the previous run did not work.
Current Australian Government guidance makes clear that unstructured trial-and-error selection among available configurations is not enough to establish a core R&D activity. The systematic progression must instead be hypothesis-driven and based on principles of established science. business.gov.au
That does not mean ordinary trial-and-error engineering is poor practice.
It can be an efficient way to tune a known system.
It simply answers a different kind of question.
If the objective is to find a workable setting among known possibilities, optimisation may be sufficient.
If the objective is to determine an unknown technical relationship, the design needs to show how each run contributes evidence about that relationship.
This distinction matters particularly when several variables interact.
A structured multi-factor experiment can produce useful knowledge.
A sequence of undocumented setting changes may only produce a configuration that happened to work.
If several things already changed, do not invent certainty afterwards
Many projects only discover this problem after the test has been run.
The team changes the material, updates the control logic and alters the geometry in the same prototype.
The new version works.
Now somebody asks which change solved the problem.
There may be no defensible answer.
The wrong response is to reconstruct a clean causal explanation after the fact.
The better response is to record exactly what changed and treat the result according to what it can actually support.
The trial may still be useful.
It may establish that the combined configuration can achieve the desired outcome.
It may rule out an earlier belief that the target was impossible.
It may reveal a new failure mode.
What it may not establish is which individual change caused the improvement.
That distinction should survive in the project record.
Australian Government record-keeping guidance states that records should be created at the time the activity is conducted. Records should show relevant background research and what was done, how the work was performed, when it occurred and who carried it out. Examples of records that may demonstrate the systematic progression include experimental plans, records of trial runs and test reports showing results and evaluation. business.gov.au The ATO likewise notes, in the context of supporting R&DTI claims, that self-assessment requires evidence and that documents created after the fact will generally not be adequate on their own without contemporaneous records. Australian Taxation Office
Where an earlier multi-change trial is ambiguous, the next experiment can be designed to separate the explanations.
The team might revert one variable while holding the others at the successful configuration.
It might deliberately test combinations that distinguish between competing mechanisms.
Or it may conclude that understanding the individual effects is unnecessary because the commercial decision only depends on whether the combined system performs reliably.
That last possibility is important.
Not every uncertainty has to be decomposed indefinitely.
Research should generate the evidence required for the decision being made.
If understanding the mechanism affects safety, scale-up, robustness or future design, the experiment may need to distinguish the relevant individual and interaction effects.
If the combination itself is the object of the research, separating every contribution may add cost without adding useful knowledge.
Design the experiment around the decision you need to make
For an engineering lead, the question is not simply whether a test was scientifically elegant.
It is whether the evidence supports the next technical judgement.
For a founder or CFO, the same issue appears as an investment decision.
Suppose three changes together produced the first acceptable result.
Is that enough to fund production tooling?
Probably only if the business is comfortable carrying uncertainty about which change mattered and whether the result will survive normal variation.
If the next stage is expensive or difficult to reverse, it may be worth resolving that ambiguity first.
A useful decision gate asks:
What uncertainty would we still be carrying forward if we scaled this result today?
That framing prevents two opposite mistakes.
The first is treating every simultaneous change as invalid and forcing unnecessarily slow one-variable-at-a-time testing.
The second is treating a successful multi-change prototype as proof of a mechanism the experiment never isolated.
Both approaches can waste time.
A sound research design sits between them.
It varies as many factors as the question requires, but no more casually than the evidence can support.
Under the current R&DTI rules, the statutory focus remains on whether the activity has an outcome that cannot be known or determined in advance on the basis of current knowledge, information or experience, and whether that outcome can only be determined through a systematic progression of work based on principles of established science, proceeding from hypothesis to experiment, observation and evaluation and leading to logical conclusions. The activity must also be conducted for the purpose of generating new knowledge. Federal Register of Legislation The program remains self-assessed. business.gov.au
The number of variables is therefore not the test.
The quality of the reasoning connecting them is.
Conclusion
Changing several variables in one experiment is not inherently unsound. The question is whether the design allows the team to distinguish the explanations that matter to the hypothesis and the next decision.
Where interactions are themselves part of the technical uncertainty, a deliberately structured multi-factor experiment may be more informative than changing one factor at a time.
The risk arises when several changes are made without a plan for interpreting their individual or combined effects.
Sources & Further Reading
Australian Government — Conducting core R&D activities for the R&D Tax Incentive
Australian Government — Assuring integrity in the R&D Tax Incentive
Australian Government — Check if you are eligible for the R&D Tax Incentive
Australian Taxation Office — Helping you get R&D claims right
Federal Register of Legislation — Income Tax Assessment Act 1997, Division 355
Talk to Ignition Research if your team is changing several technical variables at once and needs to determine whether the next experiment can distinguish a real mechanism from a configuration that merely happened to work. As a Registered Research Service Provider, Ignition Research assists businesses with research question definition, experimental design, technical investigations and contemporaneous supporting records within its registered research fields. business.gov.au
RSP registration confirms that the provider has met the relevant registration requirements for its registered research fields. It does not constitute government endorsement of a project or guarantee R&DTI eligibility.
We do not determine R&DTI eligibility or provide tax advice. The R&D Tax Incentive is a self-assessment program, and companies remain responsible for assessing their activities, expenditure and circumstances and obtaining appropriate tax advice.
Working on a project like this?
Tell us what you're building and the part you're not sure will work. A researcher from our Adelaide team (registered Research Service Provider RSP000047) reads every enquiry and replies within two business days. No documents needed.
