One Good Result Is Not a Decision — When Should You Repeat an Experiment Before Scaling?

One Good Result Is Not a Decision — When Should You Repeat an Experiment Before Scaling?

·September 16, 2026

Quick answer: One successful experiment may support a hypothesis without establishing that the result is repeatable or robust enough for the next engineering decision. A further experimental step may be appropriate where a competent professional still could not determine in advance, from current knowledge, information or experience, whether the observed outcome will persist across repeated runs, expected variation or a change in operating scale. Repetition should therefore test a defined remaining uncertainty, not simply produce more examples of the same result.

The 2026–27 Federal Budget announced R&DTI reforms for income years starting on or after 1 July 2028. Until those changes take effect, the R&DTI continues to be administered under the current legislation.

A successful run answers only the question it was designed to test

A prototype reaches the target.

A new material survives the load test.

A process achieves the required output.

The immediate temptation is to move forward.

But one successful result can mean several different things.

It may show that the proposed mechanism is technically possible under the tested conditions. It may support the hypothesis strongly enough to justify another experiment. Or it may simply show that one combination of variables produced the desired outcome once.

Those are not the same conclusion.

Australian Government guidance describes experimental R&D as a systematic progression from hypothesis through experiment, observation and evaluation to logical conclusions. The conclusion should follow from what the experimental evidence actually supports.

That creates an important distinction for technical teams:

Did the experiment establish that the proposed mechanism can work, or did it establish that the result can be produced reliably?

The first question may require one type of experiment.

The second usually requires evidence about variation.

If the business decision is whether to invest in a pilot, redesign tooling or move towards production, the uncertainty may no longer be whether the result is possible. It may be whether the result survives repetition.

Decide what repeating the experiment is meant to reveal

Repeating a test without a reason is not automatically stronger research.

Suppose a materials team develops a new component and the first mechanical test achieves the target performance.

The team could simply manufacture another ten specimens and repeat exactly the same test.

That may be useful — but only if the team knows what the repetition is intended to determine.

For example, the remaining uncertainty might be whether:

• small manufacturing differences materially change performance;

• the same preparation method consistently produces the required properties;

• measurement variation explains part of the first result;

• an uncontrolled parameter contributed to the apparent success;

• performance remains within an acceptable range across repeated specimens; or

• the result depends on a narrow condition that will be difficult to reproduce outside the laboratory.

Each question implies a different experimental design.

If the concern is measurement error, repeating measurements on the same specimen may be informative.

If the concern is manufacturing variability, independently produced specimens matter more.

If the concern is environmental sensitivity, changing temperature, humidity or another relevant condition may be necessary.

The point is not to maximise the number of runs.

It is to identify which source of uncertainty the next runs are designed to separate.

Government guidance similarly states that records should explain how an experiment is conducted or planned and may, for example, describe the parameters that are varied, held constant, observed or measured. It also recognises that a series of related experiments may be grouped within one core R&D activity where each experiment tests the same technical or scientific idea.

Repeatability and robustness answer different questions

It is useful to separate two stages.

First ask whether the result is repeatable under substantially the same conditions.

Then ask whether it remains robust when relevant conditions are expected to vary.

These are different questions.

Imagine a process achieves the target output during its first controlled trial.

The next three runs use the same equipment, material specification, settings and measurement method.

If the result cannot be reproduced, the immediate research problem may be repeatability. The team may need to identify a hidden variable, unstable mechanism or measurement issue before considering scale-up.

If the result is repeatable, the question can become broader.

What happens when raw material characteristics vary?

What happens when equipment temperature changes?

What happens when cycle time increases?

What happens when a component comes from another production batch?

At that point, controlled variation can help define the limits within which the technical conclusion remains supported.

This is particularly important before scale-up.

Current business.gov.au guidance specifically cautions that simply making something larger or moving it to a larger-scale process does not by itself establish a core R&D activity. Where existing knowledge, including knowledge generated at smaller scale, is sufficient to determine the expected outcome, the scale-up work may fall outside core R&D and instead involve implementation, standard engineering or other non-core activities. However, scale-up can present a further technical uncertainty where the outcome still cannot be determined in advance and a hypothesis-driven systematic progression is required to resolve it.

So the useful question before scaling is not:

“Have we run enough tests?”

It is:

“What uncertainty remains between the result we have demonstrated and the decision we now want to make?”

Preserve the differences between runs

Repeated experiments become difficult to interpret when the runs are treated as interchangeable even though the conditions were not.

A specimen comes from a different batch.

A fixture is adjusted.

A sensor is recalibrated.

A software parameter changes.

An operator uses a slightly different preparation sequence.

These differences may be irrelevant.

They may also explain the result.

For that reason, repeated testing should retain enough context to reconstruct how each result was produced.

This is a practical traceability framework rather than a prescribed R&DTI record format. Useful records might identify:

• experiment or specimen ID;

• material or component batch;

• equipment and configuration;

• software or firmware version where relevant;

• preparation method;

• controlled parameters;

• deliberately varied parameters;

• environmental conditions;

• measurement method;

• deviations from the test plan; and

• the evaluation following each run.

The Australian Government's current record-keeping guidance gives experimental plans, records of trial runs and test reports showing results and evaluation as examples of evidence supporting a systematic progression of work.

This does not mean every experiment requires a complex documentation system.

The objective is traceability.

If Run 4 differs materially from Run 1, the research team should be able to determine whether that difference was deliberate, incidental or unknown.

Otherwise, more experiments can create a larger dataset without producing a clearer technical conclusion.

Know when further repetition has diminishing research value

There is also a point where repetition stops adding useful information.

A team can become trapped in a cycle of requesting “one more test” because more evidence feels safer than making a decision.

A better stopping question is:

What result from the next experiment could change what we do?

If another run could expose an unresolved source of variation, distinguish between competing explanations or determine whether the project should proceed to another stage, it has a defined purpose.

If substantially the same result has already been demonstrated under the relevant conditions and another identical run is unlikely to change the technical conclusion, materially reduce the remaining uncertainty or strengthen the basis for the next decision, further repetition may have limited research value.

The next step may instead be engineering validation, implementation, production preparation or a newly defined experiment addressing a different uncertainty.

A systematic research programme should therefore become narrower as evidence accumulates.

Each experiment should be designed to reduce uncertainty, distinguish between explanations, test the limits of the current conclusion or inform the next decision.

Simply accumulating successful runs is not the objective.

Conclusion

A successful experiment may establish that an outcome is possible without establishing that it is repeatable, robust or ready for the next engineering stage.

The purpose of further experimentation should therefore be defined by the uncertainty that remains — whether that concerns measurement, manufacturing variation, environmental sensitivity, operating limits or scale.

Once further repetition is unlikely to materially reduce that uncertainty, the research team should decide whether the next step is a different experiment, engineering validation or implementation.

Sources & Further Reading

Talk to Ignition Research if a promising experimental result has been achieved but the team is uncertain whether to repeat the test, widen the operating conditions or move towards scale-up. As a Registered Research Service Provider, Ignition Research assists businesses with research question definition, experimental design, technical investigations and contemporaneous supporting records within its registered research fields.

RSP registration confirms that the provider has met the relevant registration requirements for its registered research fields. It does not constitute government endorsement of a project or guarantee R&DTI eligibility.

We do not determine R&DTI eligibility or provide tax advice. The R&D Tax Incentive is a self-assessment program, and companies remain responsible for assessing their activities, expenditure and circumstances and obtaining appropriate tax advice.

This article provides general information only and is not tax, legal or financial advice.

Working on a project like this?

Tell us what you're building and the part you're not sure will work. A researcher from our Adelaide team (registered Research Service Provider RSP000047) reads every enquiry and replies within two business days. No documents needed.

Describe your projectTry the 2-minute pathway check
Joy Fang
Written byJoy FangFounder, Ignition Research

Joy Fang is the Founder of Ignition Research, helping Australian businesses solve uncertainty through structured, well-documented R&D.

View LinkedIn profile