Quick answer: A blinded assessment can help determine whether an apparent improvement is influenced by expectations about the new design. By concealing the identity of the sample, output or configuration being evaluated, researchers can reduce some forms of observer bias. Blinding is particularly useful when human judgement influences the result, but it does not replace appropriate measurement methods, a meaningful comparison or a well-designed experiment.
20 October 2026 — This article describes the current R&D Tax Incentive (R&DTI) rules. Changes announced by the Australian Government are due to start from 1 July 2028. Until then, the program continues to operate under the current rules.
When Expectations Influence Technical Evaluation
Engineering teams often know which prototype is new, which configuration is expected to perform better and which design has received the greatest development effort.
That knowledge can influence how results are interpreted.
Consider a company developing a protective surface treatment for construction materials.
The new formulation is expected to reduce visible surface defects after environmental exposure.
After testing, engineers examine the samples and assess cracking, discolouration and surface deterioration.
If assessors know which material contains the new formulation, they may unintentionally interpret ambiguous differences in its favour.
This does not imply deliberate manipulation. Human judgement can be influenced by expectations, previous experience and contextual information.
Such effects matter particularly when outcomes involve visual grading, subjective classification or qualitative assessment. A test may be technically sophisticated yet still contain avoidable uncertainty in the evaluation process.
Blinding provides one way to address this issue.
Rather than asking an assessor whether the new prototype looks better, the experimental design can require the assessor to evaluate coded samples without knowing which represents the experimental treatment.
The resulting assessment may provide a more credible comparison, particularly when the outcome depends partly on human judgement.
Designing a Blinded Technical Comparison
Blinding works best when it is planned before testing begins.
Separate sample preparation from evaluation
Where practical, one person or team prepares and codes the samples while another independently evaluates the results.
The evaluator should not have access to information identifying the experimental condition. Sample identifiers, packaging, presentation and evaluation instructions should be designed to avoid revealing which treatment is being assessed.
For physical materials, samples may be assigned neutral identifiers.
For software outputs, the displayed results may be presented without revealing which algorithm generated them, provided the output format itself does not disclose the identity.
The order in which samples are assessed can also be randomised where appropriate, reducing the risk that presentation order systematically influences the evaluation.
The person responsible for the sample-identity key should preserve it securely and document when it is disclosed to the assessors or wider research team.
The aim is to remove unnecessary information that could influence judgement, not to conceal information that is necessary to conduct the experiment safely or correctly.
Define the evaluation criteria first
Blinding alone cannot correct an unclear performance measure.
Before evaluation begins, the team should decide what will be assessed and how differences will be classified.
A materials study might define categories of cracking, adhesion failure or surface deterioration.
A software study might define error severity, output usefulness or the frequency of technically unacceptable results.
The criteria should be sufficiently clear that different assessors can apply them consistently.
Where relevant, multiple independent assessors can evaluate the same samples. Their level of agreement can then be examined to identify whether the assessment method produces consistent classifications.
Agreement between assessors does not automatically establish that a result is correct. However, substantial disagreement may indicate that the scoring criteria or measurement method needs further refinement.
Check whether the blinding actually worked
Blinding should not simply be assumed because samples have been assigned anonymous labels.
A visible colour difference, distinctive shape, interface design or recognisable output pattern may allow assessors to infer which treatment they are evaluating.
Where this occurs, the intended blinding may be compromised.
The research team should therefore record whether assessors could identify the samples, whether any information was disclosed during testing and whether the assessment process deviated from the planned procedure.
The method should also specify how missing observations, damaged samples and unexpected results will be handled.
If complete blinding is impossible, the team can still reduce bias through predefined criteria, standardised assessment conditions, automated measurements where suitable and independent review.
The important point is to understand the limitations of the assessment rather than assume that coding alone has removed bias.
When Blinding Is Useful — and When It Is Not
Blinding is particularly relevant when technical assessment contains a human judgement component.
Possible applications include material appearance assessment, defect classification, visual inspection, usability evaluation and expert assessment of generated outputs.
For instance, a company comparing software-assisted defect detection with an established manual inspection method may ask independent reviewers to assess anonymised inspection results.
The reviewers' classifications can then be compared against a defined reference standard, where one is available.
This helps separate the reviewers' judgement from their knowledge of which method produced each result.
However, blinding may add little value when an outcome is measured automatically using a suitable, validated method and the evaluator's knowledge of the treatment cannot materially influence the measurement.
It may also be impractical when the treatments have obvious physical differences or when the person conducting the experiment must know which configuration is in use.
In such cases, other safeguards may be more useful, including predefined acceptance criteria, automated measurement, independent review or assessment by multiple evaluators.
Importantly, a blinded result is not necessarily a conclusive result.
A small difference may still fall within measurement variation. A favourable observation may apply only to the conditions tested. Blinding can reduce one potential source of bias, but it cannot resolve every limitation in an experimental design.
Sound conclusions require attention to the experiment as a whole.
What a Blinded Experiment Should Record
Blinding is valuable partly because it makes the evaluation process more transparent and reproducible.
The research record should preserve the experimental hypothesis, sample preparation method, coding approach, assessment criteria, observations and analytical conclusions.
For a blinded assessment, a practical traceability record may also identify:
• who prepared and coded the samples;
• who evaluated them and what information was available to those assessors;
• how the order of assessment was determined;
• when the sample identities were disclosed;
• whether the intended blinding was maintained;
• any accidental disclosure, coding error or procedural deviation; and
• how the observations were analysed and interpreted.
This is a practical research-recording framework rather than a prescribed R&DTI template.
Australian Government guidance on core R&D activities describes a systematic progression from hypothesis through experiment, observation and evaluation to logical conclusions. The associated record-keeping guidance states that records should be created at the time the activity is conducted. Examples of relevant records include experimental plans, records of trial runs and test reports showing results and evaluation.
For a blinded technical study, contemporaneous records can help establish how the assessment was designed, what information was available to the evaluators and how the conclusions followed from the observations.
This matters for technical credibility regardless of whether the activities are ever considered for R&DTI purposes.
Blinding also has limits in that context.
The use of a blinded comparison does not itself establish an eligible core R&D activity. Under the current R&DTI rules, a core activity must be conducted for the purpose of generating new knowledge, involve an outcome that cannot be known or determined in advance on the basis of current knowledge, information or experience, and require a systematic progression of work based on principles of established science.
Routine product testing can employ rigorous evaluation methods without meeting those requirements.
Where genuine technical uncertainty exists, a blinded assessment may form part of an appropriately structured experimental investigation.
For engineering leaders, the practical question is therefore not whether every prototype should be tested blindly.
It is whether knowledge of the expected result could influence the observations on which a technical or investment decision depends.
If the answer is yes, reducing that influence may improve the reliability of the evidence.
Conclusion
A prototype that looks better may genuinely perform better. But where the evaluation depends on human judgement, expectations about the new design can influence what assessors observe and report.
Blinding can reduce that risk by separating the assessment from knowledge of the sample's identity. Its effectiveness depends on how the samples are prepared, what information evaluators can access and whether the assessment remains genuinely blinded.
The objective is not to make every test blind. It is to recognise where expectations could influence the result and design an evaluation that produces evidence the team can trust.
Sources & Further Reading
Australian Government — Conducting core R&D activities for the R&D Tax Incentive
Australian Government — Record keeping for the R&D Tax Incentive
Cochrane — Handbook, Chapter 8: Assessing risk of bias in a randomised trial
Australian Government — Get help from a research service provider
Talk to Ignition Research if your prototype comparison depends partly on human judgement and you need to determine whether a blinded assessment would improve the reliability of the evidence. As a Registered Research Service Provider, Ignition Research assists businesses with research question definition, experimental design, technical investigations and contemporaneous supporting records within its registered research fields.
RSP registration confirms that the provider has met the relevant registration requirements for its registered research fields. It does not constitute government endorsement of a project or guarantee R&DTI eligibility.
We do not determine R&DTI eligibility or provide tax advice. The R&D Tax Incentive is a self-assessment program, and companies remain responsible for assessing their activities, expenditure and circumstances and obtaining appropriate tax advice.
This article provides general information only and is not tax, legal or financial advice.
Working on a project like this?
Tell us what you're building and the part you're not sure will work. A researcher from our Adelaide team (registered Research Service Provider RSP000047) reads every enquiry and replies within two business days. No documents needed.
