Quick answer: A technically sound experiment can still be incapable of answering its question if the measurement system cannot distinguish the expected effect from measurement variation. Routine calibration or verification is ordinarily an implementation task, but a further technical uncertainty may arise where a competent professional cannot determine in advance whether the available measurement method can resolve the effect under the required conditions. In that situation, the measurement method may need to be tested before the main experimental result can be relied on.
The experiment is only as informative as the measurement
A team can design a clear hypothesis, hold the right variables constant and run the equipment exactly as planned — and still learn very little.
The problem may be the measurement.
Consider an engineering team investigating whether a change to a control method reduces vibration in an industrial system.
The first configuration produces one result.
The modified configuration produces a slightly different result.
Is that difference caused by the control method?
Possibly.
But it may also reflect sensor placement, instrument drift, sampling behaviour, mounting conditions, environmental interference or ordinary measurement variation.
Until the team knows whether its measurement system can distinguish between those possibilities, the comparison remains difficult to interpret.
This matters because Australian Government guidance does not describe experimentation simply as trying something and recording a number. A core R&D activity must be conducted for the purpose of generating new knowledge, involve an outcome that cannot be known or determined in advance, and require a systematic progression of work through hypothesis, experiment, observation, evaluation and logical conclusions to determine that outcome. The evaluation stage should consider the relationships between the parameters varied, held constant, tested or measured.
A weak measurement system can undermine that progression.
If the observation cannot reliably distinguish the effect being tested, the team cannot confidently evaluate whether the experiment supported the hypothesis.
That does not make measurement uncertainty itself an R&D activity.
Using an established calibration procedure, replacing an unsuitable sensor or applying a recognised measurement method may be routine technical work rather than a separate core R&D activity.
A separate research question may arise only where the measurement problem itself contains an outcome that cannot be known or determined in advance from current knowledge, information or experience and can only be determined through a hypothesis-driven systematic progression of work.
Test whether the method can see the effect
Before committing to a large experimental campaign, it can be useful to ask a narrower question:
Can the measurement method resolve a technically meaningful change in the variable we care about?
This is different from asking whether an instrument turns on, has a calibration certificate or produces repeatable-looking numbers.
The method includes the entire observation chain.
For an industrial trial, that chain might include the physical sensor, mounting position, sampling frequency, signal processing, environmental conditions, data cleaning and the rule used to convert the raw signal into a result.
For a software or AI experiment, the equivalent issue may be the evaluation dataset, annotation consistency, benchmark definition or performance metric.
For a materials experiment, it may be specimen preparation, fixture behaviour or the analytical technique used to characterise the sample.
A pre-experiment measurement brief might therefore define the following. This is a practical experimental-design framework rather than a prescribed R&DTI record format:
• what physical or technical quantity is actually being measured;
• the expected size of the difference that would matter to the hypothesis;
• the conditions under which measurements will be compared;
• which parts of the measurement chain must remain fixed;
• whether repeated observations of the same reference condition are stable enough for the intended comparison;
• how drift, missing data or anomalous observations will be handled;
• what reference or baseline will be used; and
• what result would make the measurement method inadequate for the planned experiment.
The purpose is not to eliminate every source of uncertainty.
That is rarely practical.
The purpose is to know whether the experimental effect is distinguishable from the uncertainty introduced by observing it.
Australia's National Measurement Institute describes measurement traceability as an unbroken chain of calibrations to primary measurement standards and notes that traceability helps measurements remain comparable and gives researchers and industry confidence in their accuracy.
Traceability is not the same thing as experimental validity, but the principle is important: a result is more useful when the team knows what it is being compared against and how confidently that comparison can be made.
Separate measurement variation from system variation
One of the most common interpretation problems occurs when several forms of variation are mixed together.
Suppose a manufacturer is testing a redesigned component.
Five components are produced.
Each is tested once.
The results differ.
The immediate conclusion might be that the manufacturing process is inconsistent.
But the data do not yet tell the team whether the variation came from the components or from the measurement process.
A different design might measure the same component several times before moving to independently produced components.
If repeated measurements of one unchanged component move substantially, the measurement system itself needs attention.
If repeated measurements remain stable but independently produced specimens differ, the manufacturing process becomes a more plausible source of variation.
If both vary, the experiment needs to separate the two.
This is not merely a statistical housekeeping issue.
It changes the technical conclusion.
Without that separation, the business may redesign a process that was not causing the problem, reject a promising prototype because the measurement was unstable, or invest in further trials that cannot resolve the original question.
It also affects what should be recorded.
Current R&DTI record-keeping guidance recommends establishing a record-keeping system before R&D activities begin and states that records should be created at the time the activity is conducted. Examples of relevant evidence include experimental plans, records of trial runs and test reports showing results and evaluation.
For measurement-sensitive work, the experimental record should therefore preserve more than the final reported value.
It should make clear which instrument and configuration were used, which version of the test method applied, which reference condition was used and whether any relevant change occurred between runs.
If the measurement method changes midway through the project, that change should become part of the research record rather than being silently absorbed into the dataset.
Better measurement should inform the decision
Measurement work can expand indefinitely if the team aims for precision without asking what decision the precision needs to support.
The useful stopping point is not perfect measurement.
It is sufficient measurement for the research question.
Suppose two design configurations appear different.
If the measurement system is already capable of distinguishing a difference large enough to affect the engineering decision, further refinement may not add much value.
If the apparent difference cannot be distinguished reliably from the variation introduced by the measurement method at the required decision threshold, the team has a different problem: the existing experiment cannot yet resolve the hypothesis.
The next activity may therefore be a measurement-method investigation rather than another prototype iteration.
That distinction can save substantial effort.
It prevents the team from producing more observations when the real limitation is its ability to interpret them.
It also helps define the boundary between research and ordinary engineering.
Where an established measurement approach already provides the required resolution and the team simply needs to apply it correctly, implementation or routine technical work is the more accurate description.
Where the required measurement capability cannot be determined in advance and must itself be established through a systematic experimental progression, a further technical research question may arise. Whether any particular activity meets the R&DTI criteria remains a matter for the company to assess against the applicable requirements.
The central question is therefore simple:
Before asking whether the experiment worked, ask whether the experiment was capable of detecting the result you were looking for.
Conclusion
A technically sound experiment can still produce ambiguous evidence if the measurement system cannot resolve the effect being tested.
The measurement method should therefore be treated as part of the experimental design: its resolution, configuration, stability and limitations determine what conclusions the observations can support.
When measurement uncertainty becomes the limiting factor, the next step should address that uncertainty before further prototype or system testing simply produces more difficult-to-interpret data.
Talk to Ignition Research if your technical project is producing ambiguous or conflicting results and it is unclear whether the unresolved problem sits in the technology being tested or in the measurement method used to observe it. As a Registered Research Service Provider, Ignition Research assists businesses with research question definition, experimental design, technical investigations and contemporaneous supporting records within its registered research fields.
RSP registration confirms that the provider has met the relevant registration requirements for its registered research fields. It does not constitute government endorsement of a project or guarantee R&DTI eligibility.
We do not determine R&DTI eligibility or provide tax advice. The R&D Tax Incentive is a self-assessment program, and companies remain responsible for assessing their activities, expenditure and circumstances and obtaining appropriate tax advice.
Sources & Further Reading
Australian Government — Conducting core R&D activities for the R&D Tax Incentive
Australian Government — Record keeping for the R&D Tax Incentive
Department of Industry, Science and Resources — National Measurement Institute — Australia's measurement system
R&DTI note: Measurement-method work does not become a core R&D activity simply because it supports an experiment. Where an established method is sufficient, the work may be routine technical activity; where a separate unknown outcome requires hypothesis-driven experimentation, that activity needs to be assessed against the R&DTI requirements on its own facts.
Working on a project like this?
Tell us what you're building and the part you're not sure will work. A researcher from our Adelaide team (registered Research Service Provider RSP000047) reads every enquiry and replies within two business days. No documents needed.
