MISSOURI DEPARTMENT OF NATURAL RESOURCES
P. O. Box 250, Rolla, Missouri 65401
MEMORANDUM
Date: March 12, 1984
To: Users of the EDM Baselines
From: Robert E. Myers, State Land Surveyor
Subject: Explanation of the Calibration Report
The computer program used to analyze the results of the calibration measurements on the calibration baselines provide essentially two pieces of information. A system constant and a system scale factor. The system constant is composed of two components, the prism constant and the error of the phase difference determination. In the ideal situation, the prism constant will be known, so the error in phase determination will be the only unknown. For most EDM equipment the phase difference determination error will be plus or minus five millimeters. When the results of the calibration report shows the system constant to be greater than 5 millimeters, one needs to determine whether this reflects an error in the determination of the system prism constants or reflects a problem with the phase determination quality of the instrument. If the allowable error in the determination of the prism zero constant is 10 millimeters and the allowable error in the phase determination is 5 millimeters, the possible error in the system constant correction would be approximately 11 millimeters. For this reason we believe that when the computed system constant is greater than 12 millimeters, the owner should further analyze his system and determine the prism constant by other measurements.
The system scale factor is also computed in this analysis of the data. The scale factor is composed of three separate error quantities. They are the error in the velocity of the propagation of light in a vacuum, error in the modulation frequency, and error in the refractive index. The error in the velocity of the propagation of light in a vacuum is extremely small and one which we do not have to contend with. If we have correctly measured temperature and barometric pressure, the error in the refractive index should be very close to zero; therefore, all the error which would be shown by the system scale factor would be a result of the error in the modulation frequency. The standard error in frequency would be approximately 10 megaherts [sic] or approximately an error of 5 parts per million. If the computed system scale factor is greater than 5 parts per million, one must worry about whether or not the resulting error is a function of errors in temperature and pressure. Surveyors should first determine that his temperature and barometric pressure reading equipment is properly functioning and calibrated. He should then remeasure the distances on the calibration baseline so as to reflect the different conditions. This result may then show whether the error lies with the modulation frequency or the atmospheric conditions.
The computer program also computes whether or not the statistical value of the system constant and scale factor are equal to zero. When the value is computed as statistically equal to zero, we can say that the instrument is definitely in good adjustment for that value. If the value is not equal to zero, we have to make a judgement of the equipment based on the computed value as previously outlined. The calibration report also shows the observed data, its difference from the know value and a rejection limit. The rejection limit is based on three times the standard deviation determined by this calibration procedure. Some of the values will be outside the rejection limit. This is for our use and should be disregarded by the owner of the equipment.
In summary, the owner of the equipment should consider his equipment to be satisfactorily operating if the constants are statistically equal to zero. If one or more of the constants are not equal to zero, then the owner must evaluate each of those constants to see if they are within an acceptable limit. An acceptable limit for the system constant is 11 millimeters and approximately 7 millimeters for the scale factor for most short and medium range equipment. Any greater values should be further analyzed.
-------------------------------
transcribed by Steven E. Weible
See transcribers comments in the article,"MODNR's EDM Baseline Comparison Rejection Limit"
