This is the text from my article, "Analysis of an EDM Baseline Comparison," from the March 2011 issue of Missouri Surveyor. See link under Published Articles at right.
--------------------------------------------------------------------------------------------
For those who have ever wondered how to interpret the "Calibration Report" of an EDM baseline comparison that was produced by a Missouri Department of Natural Resources (MoDNR) application, perhaps the following discussion will help.
Referring to NOAA Technical Memorandum NOS NGS-10, "Use of Calibration Base Lines" (click here to see the publication), the following discussion is found under the heading of "Analysis of Calibration Base-Line Observations" (page 9 of that publication):
"Most EDMI manufacturers routinely attribute certain accuracies to their instruments. Although these accuracies should reflect the instrument's ability to measure a "true value," they may, in fact, indicate only the repeatability (precision) of the instrument or test results performed under laboratory conditions. Theoretically, if the accuracy statistic is given in terms of a standard error (sigma), 68.3% of the differences between a "true value" and an observed value should fall within the stated specification. Therefore, this value could be used for decision purposes, i.e., as a test statistic. However, the above is true only for large samples and for known standard errors. Both of these requirements are rarely satisfied. In addition, by using this test statistic for rejection purposes, another type of error may be committed, i.e., the rejection of valid observations. To reduce the possibility of rejecting a valid observation, a limit of 3sigma (three times the standard error value) is usually chosen for deciding if an observation is acceptable or not acceptable. Theoretically, 99.7% of the differences should fall within the 3sigma range...
If 99.7% of the observations fall within three times the manufacturer's stated accuracy and 68.3% fall within the manufacturer's stated accuracy, the instrument can be accepted as working accurately and reliably."
One method of analysis of EDM baseline observations, then, is to examine how the observations compare to a standard specification. Since the distance observations from a baseline comparison are not different measurements of the same segment, some means must be used to ensure an "apples to apples" comparison. This is accomplished by computing the difference between the known distance and the observed distance of each segment measured. On the MoDNR report these differences are shown at the lower portion of the page.
A typical comparison on a Missouri EDM baseline will result in twelve (12) distance observations.
68.3% of 12 observations = (0.683)*12 = 8 observations
99.7% of 12 observations = (0.997)*12 = 12 observations
The standard specification for comparison is the manufacturer's stated precision of the instrument being tested, so, if 8 of the differences are equal to or smaller than the manufacturer's stated precision and all 12 of the differences are equal to or smaller than 3 times the manufacturer's stated precision, then "the instrument can be accepted as working accurately and reliably."
For an instrument with a manufacturer's stated precision of +/-(0.002 m + 2 ppm*Distance) and a known distance of 1234.5678 meters,
sigma = 0.002 m + (2 ppm) * (1234.5678 m)
sigma = 0.002 m + (2 / 1,000,000) * (1234.5678 m)
sigma = 0.004 m
and
3sigma = 3 * [ 0.002 m + (2 ppm) * (1234.5678 m) ]
3sigma = 3 * [ 0.002 m + (2 / 1,000,000) * (1234.5678 m) ]
3sigma = 0.013 m
See example data below:
Manufacturer's stated precision: constant = 2 mm
scale factor = 2 ppm
Known Observed One Sigma 3 Sigma
Distance Distance Delta Value Rejection Limit
(meters) (meters) (meters) (meters) (meters)
-------- -------- -------- -------- --------
149.9649 149.9633 0.0016 0.002 0.007
399.9523 399.9523 0.0000 0.003 0.008
1374.9235 1374.9207 0.0028 0.005 0.014
149.9649 149.9644 0.0005 0.002 0.007
249.9874 249.9899 -0.0025 0.0025 0.0075
1224.9584 1224.9572 0.0012 0.004 0.013
399.9523 399.9520 0.0003 0.003 0.008
249.9874 249.9897 -0.0023 0.0025 0.0075
974.9712 974.9683 0.0029 0.004 0.012
1374.9235 1374.9171 0.0064 0.005 0.014
1224.9584 1224.9540 0.0044 0.004 0.013
974.9712 974.9664 0.0048 0.004 0.012
For this set of data, nine observations are equal to or smaller than the manufacturer's stated precision and all twelve observations are smaller than the rejection limit of three times the manufacturer's stated precision. Therefore, according to this evaluation criteria, the instrument that was tested "can be accepted as working accurately and reliably."
Referring again to NOAA Technical Memorandum NOS NGS-10, the discussion there continues:
"If the differences do not agree within above specifications, then a different method must be used to determine an instrument's acceptability... One such approach is to examine the differences between observed values and published values and determine if the difference is a constant or is proportional to the distance being measured (scale error)... The preferred approach is a least-squares solution that simultaneously determines a scale and a constant correction. This solution is based on the supposition that the differences can be attributed either to a scale correction or to a constant correction, or both."
The MoDNR processing application performs a least-squares computation, using the formulas presented in NOAA Technical Memorandum NOS NGS-10 and the results can be found on the report near the center of the page. If the instrument is performing acceptably, the computed system constant and scale factor should be comparable to the manufacturer's stated precision.
For the example data shown above, the computed system constant is -1.4 millimeters and the computed scale factor is 4.2 parts per million.
A primary assumption of this discussion and an important fact that should be recognized by the user is that an EDM baseline comparison is meaningful only if the known distances of the baseline have been determined to a higher degree of precision than that of the equipment being tested, as described in NOAA Technical Memorandum NOS NGS 8, "Establishment of Calibration Base Lines" (click here to see the publication).
----------------------------------------
Original composition by Steven E. Weible
