Fit for Purpose
Fitness of Purposes asks: is your measurement good enough for the task it is being applied? This dictates hoe much rigor is required in a measurement process.
When you measure an unknown amount of something the best science can do is provide us with an estimate. Some estimates might be extremely close to a true value, but they are still just estimates.
For example, if you want to know how much alcohol is in a blood sample (taken from a person suspected of DUI) there are sophisticated machines to perform that measurement. However, at the end of the process, the result is merely an estimate.
On the other hand, let’s say you combine a known amount of ethanol to a blood sample. Specifically, if you added .190 of ethanol to a sample of blood (containing no alcohol), then the result can be compared to a true value. In this instance, it’s possible to determine accuracy. This is because you start off knowing the true value (.190).
What about when you test an unknown value? In DUI cases the blood sample seized from the person is always an unknown value. Consequently, there is no true value to compare the result at the end of the measurement process.
Thus, the end result is an estimate. It is not intended to represent a true value. The true value is unknowable.
The Wrong Question
Therefore, asking is the result accurate? isn’t the right question. We need to know if we can trust the measurement system (to work as well as when we are testing a known value). We must determine if the measurement system is good enough to rely on.
This inquiry is directly connected to and shaped by, the purpose of the measurement. How good a measurement system must be – depends on – the purpose of the measurement. What you are using the measurement for dictates how much rigor is required in making the measurement.
A measurement system where there are lots of unknown variables and unanswered questions is not fit for making life and death decisions (or questions of guilt or innocence). However, when we merely need to get into the ballpark, then we can skimp on a few things or leave some questions unanswered. This is because an incorrect decision doesn’t result in an important consequence.
The Measurement System
With all the above in mind, we see why asking is it accurate? is not the right question. When presented with a forensic measurement we have to focus on the measurement system, not the result. We have to ask – is it fit for its purpose?
Whether a proponent of a measurement has proven accuracy, reliability and validity also depends on the intended purpose of the measurement. Has the proponent provided enough evidence of these qualities so that the measurement is “fit for the purpose” of determining a person’s alcohol concentration.