Quick answer: Not every commercial R&D experiment needs a formal control group, but it does need a credible basis for comparison. Where an outcome cannot be known or determined in advance by a competent professional on the basis of current knowledge, information or experience, the experiment should be designed so the team can evaluate whether the observed result is meaningfully connected to the change being tested rather than to unrelated variation.
A control is a way to interpret change
In research, the word “control” can make commercial teams think of clinical trials: two groups, one receiving the intervention and one not.
That is only one form of comparison.
An engineering or software experiment may use a baseline configuration, the previous production method, an accepted reference model, a standard material, a historical operating condition or an unchanged system version.
The purpose is the same.
When the result changes, the team needs some basis for asking:
What changed when we introduced the variable being tested, and what other explanations remain?
Current Australian Government guidance describes an experiment as a scientific procedure undertaken to test a hypothesis. It states that records should explain how the experiment was conducted or planned and may, for example, describe the parameters varied, held constant, observed or measured. Evaluation should then consider the causal relationships between the parameters varied, held constant, tested or measured. Evaluation should consider causal relationships between those parameters. business.gov.au
The Income Tax Assessment Act 1997 does not prescribe a particular experimental format or require every project to maintain a formal control group. Instead, section 355-25 requires an experimental activity whose outcome cannot be known or determined in advance on the basis of current knowledge, information or experience, but can only be determined through a systematic progression of work based on principles of established science, proceeding from hypothesis to experiment, observation and evaluation and leading to logical conclusions. The activity must also be conducted for the purpose of generating new knowledge. Federal Register of Legislation
That gives commercial R&D some flexibility.
The important question is not:
“Do we have a control group?”
It is:
“What are we comparing the experimental result against, and does that comparison let us interpret what happened?”
Choose the reference condition to match the hypothesis
A useful baseline depends on the hypothesis.
Suppose a software team is testing whether a new scheduling algorithm reduces processing latency under highly variable workloads.
The relevant baseline may be the existing algorithm running against the same representative workload.
If the new version is tested on different data, different infrastructure and under different traffic conditions, the team may obtain an impressive result without knowing what caused it.
The problem is similar in manufacturing.
A company may change a material formulation to reduce cracking. If the new formulation is produced using a different machine setup, different curing profile and a different raw-material batch, comparing the output with an earlier production result may not isolate the effect the team intended to test.
Sometimes the baseline is not the current commercial system.
It may be a known reference condition chosen specifically to test a mechanism.
For example, if the research question concerns whether a new surface treatment changes moisture behaviour, the most useful comparison may be an untreated specimen prepared under the same conditions rather than a product taken from ordinary production.
The comparison should follow the uncertainty.
This is why “before versus after” is not automatically a strong experiment.
If the environment changed between the two periods, the difference may reflect season, operator behaviour, supplier variation, data mix or another uncontrolled condition.
A baseline becomes useful when it makes the competing explanations easier to separate.
When a perfect control is impossible, preserve comparability
Commercial R&D rarely occurs in perfectly controlled environments.
A field trial cannot stop the weather.
A production line cannot always dedicate identical equipment to experimental and reference runs.
A construction trial may involve conditions that cannot be recreated.
A live software system may receive different user behaviour every day.
That does not make structured experimentation impossible.
It changes the design problem.
The team should identify which variables can be controlled, which can be measured and which sources of variation need to be accounted for when results are evaluated.
CSIRO provides a useful example in its description of field experiments investigating grass-fire behaviour. The researchers could not control weather, but they could choose the timing of the experiment, establish controlled test plots, monitor relevant environmental conditions and measure those conditions alongside the experimental observations. CSIRO
Commercial teams can apply the same logic.
If raw-material variation cannot be eliminated, record the batch characteristics.
If field temperature cannot be fixed, measure it.
If user traffic changes between software tests, preserve workload information.
If operators differ between production runs, record who conducted the run and how the procedure was followed.
The goal is not to remove all real-world variation.
The goal is to understand and record enough of that variation that it does not remain an unrecognised competing explanation for the result.
This is particularly important when a business is deciding whether to scale, deploy or invest further.
An experiment does not need laboratory perfection to be useful.
It does need enough comparability for the team to understand what evidence it has actually produced.
Treat the reference condition as part of the experiment record
Commercial teams often document the experimental version carefully and treat the baseline as obvious.
Six months later, it may no longer be obvious at all.
The “standard process” may have changed.
The reference software may have received updates.
The supplier may have altered a material specification.
The production settings may have drifted.
The dataset used as the comparison may no longer be reconstructable.
Current R&DTI record-keeping guidance states that records should be created at the time the activity is conducted. Records should show what was done, how the work was performed, when it occurred, who carried it out, and the resources used in the R&D activities and how those resources were used. Examples of records that may demonstrate the systematic progression include experimental plans, records of trial runs and test reports showing results and evaluation. Examples include experimental plans, trial-run records and test reports containing results and evaluation. business.gov.au
For a comparison-based experiment, that discipline should apply equally to the reference condition.
The useful record should identify the experimental configuration and the baseline configuration with enough precision that somebody reviewing the work later can understand the difference between them.
In software, that may mean version identifiers, environment configuration and datasets.
In manufacturing, it may mean material batch, machine settings, operating conditions and measurement method.
In a field trial, it may mean site conditions, timing, monitoring information and any known differences between test locations.
This does more than support later compliance work.
It protects the technical value of the experiment.
If the team later discovers an unexpected result, it can return to the comparison and ask whether the effect came from the intended variable or from something else that changed.
That is often the difference between a result that looks interesting and a result that can support a decision.
Conclusion
A control does not need to look like a textbook control group.
What matters is whether the experiment has a credible reference condition that allows the team to interpret the observed change and distinguish the explanation being tested from plausible alternatives.
Where that reference cannot be held completely constant, the experimental method should make the remaining differences visible rather than pretending they do not exist.
Sources & Further Reading
Australian Government — Conducting core R&D activities for the R&D Tax Incentive
Australian Government — Record keeping for the R&D Tax Incentive
Federal Register of Legislation — Income Tax Assessment Act 1997, Division 355
Talk to Ignition Research if your technical project needs a meaningful baseline, reference condition or comparison method and it is unclear what must remain controlled for the next experiment to produce interpretable evidence. As a Registered Research Service Provider, Ignition Research assists businesses with research question definition, experimental design, technical investigations and contemporaneous supporting records within its registered research fields. business.gov.au
RSP registration confirms that the provider has met the relevant registration requirements for its registered research fields. It does not constitute government endorsement of a project or guarantee R&DTI eligibility.
We do not determine R&DTI eligibility or provide tax advice. The R&D Tax Incentive is a self-assessment program, and companies remain responsible for assessing their activities, expenditure and circumstances and obtaining appropriate tax advice.
Working on a project like this?
Tell us what you're building and the part you're not sure will work. A researcher from our Adelaide team (registered Research Service Provider RSP000047) reads every enquiry and replies within two business days. No documents needed.
