Geophysical Classification for Munitions Response Frequently Asked Questions (FAQs)

Geophysical Classification Applied to Munitions Response

Site Specific Planning

Data Collection

Data Analysis

QC/QA

Project Conclusion

After the ESTCP Demonstration Program


Geophysical Classification Applied to Munitions Response

  • 1. What is different about advanced geophysical classification compared to things that have been termed classification or discrimination in the past?

    Advanced geophysical classification describes the use of intrinsic properties of a buried metal object to classify that object as either a target of interest or not. Most commonly, the intrinsic properties used are the principal-axis polarizability decay curves (sometimes called EMI fingerprints) which reflect the size, symmetry, material composition, wall thickness, etc. of the object. These polarizability decay curves are recovered from the measured data through a model-matching process called geophysical inversion.

    Previous discrimination efforts have been based on properties of the geophysical data such as spatial footprint of the anomaly, temporal decay of an individual sounding, etc. While these data-based parameters have some relation to the buried object, they are convolved with object’s depth and orientation relative to the sensor making unambiguous analysis difficult, if not impossible.

    Because geophysical classification utilizes more parameters for making determinations that reveal specific, intrinsic information about the buried items, it is inherently more selective than previously utilized classification techniques. This increase in selectivity allows for geophysical classification to be more effective at discerning TOI from non-TOI.

  • 2. What is a target of interest?

    Target of interest, or TOI, is the term used for items that must be correctly classified and excavated to accomplish site remediation goals. All munitions, QC and validation seeds, and other items designated by the site team such as significant pieces of munitions, etc. are targets of interest. Some site teams may even want to include certain fuzes and other munitions components to the TOI list. Munitions do not have to contain high-explosive filler to be TOI; anything that has to be excavated and examined to determine if it is hazardous is, by definition, a TOI.

  • 3. What types of munitions can be classified?

    All types of munitions can be classified but the efficiency of correctly classifying clutter depends on munitions types found at a site. The greater the difference of the EMI response of the munitions from that of the predominant site clutter, the greater the classification efficiency. Large projectiles look almost nothing like the clutter found on most sites and can be classified with great efficiency. Many common clutter items have an EMI response similar to that of 20-mm projectiles so classification is less efficient if the targets of interest include 20-mm projectiles and other small munitions.

  • 4. Is geophysical classification effective if there are unexpected munitions on the site?

    During the ESTCP demonstrations, geophysical classification has been proven to be effective even when there were unexpected munitions on site. In the early days of the demonstration program, we tested this question by seeding the site with munitions that were not among those known to be present from the historical documents. All demonstrators were able to identify these unexpected munitions in all cases.

    There are two aspects to this question. The first case involves munitions that are not expected at the site but are contained in the munitions library available to the analyst. The standard operating procedure for data analysis must involve a step where responses observed on the site are compared to the full library of munitions. This procedure will identify any unexpected but known munitions.

    The second, more difficult case, concerns unexpected munitions whose response is not in the master library of signatures. In the early days of the ESTCP program as the library was being assembled, this was not an uncommon occurrence. Even though the signatures in question were not in the library, they still exhibited all the hallmarks of a munition (size, symmetry, and wall thickness) and thus were classified as TOI. The Ft. Sill demonstration site contained items described by the intrusive crew as “40mm Frag Ball Cups Containing HE” that were relatively thin-walled and thus incorrectly classified by most analysts. The site managers knew to expect these items but this information was never relayed to the analysts proving again that communication is one key to a successful project.

    In both of these cases, information gained during the site surface clearance can help minimize the problem. The surface clearance report should include evidence of any unexpected munitions encountered during the clearance.

    ESTCP has initiated a project to compile a master library of munitions signatures with complete metadata. A number of ordnance museums have been visited during this project in an attempt to make the library as complete as possible and minimize the chances of encountering unknown munitions on a production site. After completion of the ESTCP project, the library will be maintained and updated as necessary by personnel from the US Army Corps of Engineers Environmental and Munitions Center of Expertise (EM CX). At the beginning of a munitions response project, the Government project manager will obtain the current version of the library and distribute it to the analysis contractor. Any item unique to the site that is not included in the master library can be added to the “site-specific library” to be used on that site.

  • 5. How deep can the advanced sensors detect? Classify?

    For small and medium munitions, TEMTADS and MetalMapper can detect to depths similar to the familiar EM61. The transmit moment (a measure of the excitation strength of the sensor) is somewhat smaller for the advanced sensors but that is compensated by the additional receive channels available for noise suppression. For larger munitions such as 105-mm and 155-mm projectiles and bombs, the detection depth of the advanced sensors is shallower than an EM61 but still deeper than most items are found.

    The advanced sensors collect up to two orders of magnitude more data than an EM61. These additional data also allow buried items to be adequately classified down to their detection depth.

    The figure on the right below shows the response in channel 2 of an EM61-MK2 to a 60mm mortar as a function of depth. For a reasonably low-noise site, the maximum depth of reliable detection is reached for a mortar in its least favorable orientation at about 65 cm. The plot on the left shows the corresponding amplitude response for a TEMTADS 2x2 sensor. Here, the amplitude-based detection limit is reached at about 58 cm.

    Detection data for a 105-mm projectile are illustrated below. The detection depth of a 105-mm with a TEMTADS is ~90 cm while it is ~120 cm with an EM61-MK2. Of course, if the munitions of interest at a site are exclusively large, deep munitions then a magnetometer remains the detection sensor of choice.

    As mentioned above, the advanced sensors collect much more data than the simple z-axis amplitude plotted in the two figures. These extra data can be used in methods such as Informed Source Selection to increase the detection depth of these advanced sensors.

  • 6. Are there limitations in terms of terrain and vegetation cover for the advanced sensors?

    Anywhere that the sensors can be maneuvered and data collected within 20 to 30 cm of the ground, classification can be successful. One of the advanced sensors can be used anywhere that traditional sensors like the EM61 can be deployed. Shown in the photos below are the tractor-borne MetalMapper sensor (left panel), the TEMTADS 2x2 in litter-carry mode (center panel), and the handheld MPV sensor in the woods (right panel).

  • 7. Where did geophysical classification not work and why?

    Geophysical classification works almost everywhere. The primary exception is the case where the anomaly density is so high that there is not enough information in the sensor signals to resolve the overlapping responses of the closely-spaced objects. This can occur at the center of heavily used targets, in disposal pits and dumps, and near OB/OD areas. High and variable (on a short length scale) geologic background may also make classification impractical, particularly if the munitions of concern are small. If the background signal is a significant percentage of the response of the TOI and varies on a length scale of tens of centimeters, it will not be possible to adequately subtract the geologic background response and the polarizabilities derived will be unreliable.

    There are other sites on which it may not make sense to apply classification. On sites with an unusually high ratio of TOI to clutter (an ESTCP demonstration on a strafing range fell into this category) there may not be enough clutter digs saved to make the economics of classification work. Similarly, sites with very small numbers of anomalies may not provide enough opportunities to repay the cost of mobilizing the classification sensor and crew

  • 8. Will regulators accept the use of this technology?

    Classification technology has received regulatory approval for use on several sites. As more and more regulators gain familiarity with classification, an increasing number are accepting its use on their projects. The Interstate Technology & Regulatory Council (ITRC) has completed a Geophysical Classification for Munitions Response technical and regulatory guidance and accompanying training to prepare state regulators to be informed participants in site teams contemplating the use of classification. ESTCP has had a Classification Advisory Group since the initiation of the demonstration program. The Advisory Group, which includes State and Federal regulators, has been instrumental in the design and conduct of the ESTCP demonstrations. They have vetted all the major conclusions of the demonstrations and been heavily involved in the program Final Report.

    The Intergovernmental Data Quality Workgroup has prepared a Geophysical Classification for Munitions Response (GCMR) UFP-QAPP template for use in munitions response projects. The DoD Environmental Data Quality Workgroup (EDQW) is finalizing the DoD Advanced Geophysical Classification Accreditation Program (DAGCAP) to ensure that performing contractors have the appropriate quality systems in place and their employees have the training and experience to successfully complete munitions response projects using geophysical classification.

  • 9. What resources are available for determining if advanced geophysical classification is appropriate to my site?

    There are several sources for independent information about advanced geophysical classification and the conduct of classification projects. The Interstate Technology & Regulatory Council (ITRC) Geophysical Classification for Munitions Response team has published Technology and Regulatory Guidance on this subject and is offering Internet-based training. This material can be accessed from the ITRC web site.

  • 10. How do I know that a contractor is qualified to conduct a geophysical classification project?

    We have demonstrated during the ESTCP program that training and experience are the best predictors of performance success on a classification project. The primary criterion for evaluation of contractors should be successful past performance on previous classification projects. In addition, the DoD is in the process of establishing the DoD Advanced Geophysical Classification Accreditation Program (DAGCAP). When fully implemented, only contractors that have been accredited by third-party Accrediting Bodies will be eligible to perform classification on munitions response projects. This will ensure that performing contractors have the appropriate quality systems in place and their employees have the training and experience to successfully complete munitions response projects using geophysical classification.

  • 11. What do I need to know about my site to decide whether geophysical classification is appropriate?

    An up-to-date Conceptual Site Model (CSM) and a clear and detailed remedial objective are required to assess whether geophysical classification is appropriate on a site. The CSM should contain information on the expected munitions, their expected depths, any terrain or vegetation issues that would impede classification, geologic formations that may cause interference with the geophysics measurements, and the expected density of anomalies throughout the site. All of these factors have a bearing on the efficiency of classification.

    Vague remedial objectives such as “remove all munitions to the depth of detection” make it very difficult to design a detection and classification program for maximum efficiency. It is far better to define the depth of concern based on anticipated land uses and list the munitions expected to be encountered. This will allow the project geophysicist to determine in advance if the remedial objectives can be met using advanced EMI sensors.

  • 12. What constitutes an easy, typical, and hard site? What is typical classification performance on each? What is the range?

    Example easy, typical, and hard sites from the ESTCP demonstration program and their characteristics are detailed in the table below.

      Characteristics Example Demonstration Sites Expectation
    Easy Limited munitions types, low anomaly density, terrain and vegetation allow for high quality data collection, no geologic interference Pole Mountain, WY
    Camp George West, CO
    SW Proving Ground, AR
    Almost all analysts correctly classify all seeds, eliminate ~90% of the clutter from the final dig list
    Typical Mixed munitions, none smaller than 37-mm, low to moderate anomaly density, terrain and vegetation allow for high quality data collection, moderate geology Spencer Range, TN
    Camp Beale, CA
    Most analysts correctly classify all the seeds, eliminate ~80% of the clutter from the final dig list
    Hard One or more of mixed munitions, smaller than 37-mm TOI, high anomaly density, conditions make data collection challenging, complex geology Massachusetts Military Reservation, MA
    Ft. Bliss, TX
    Waikoloa Maneuver Area, HI
    Only the most skilled analysts detect all TOI, eliminate ~50-70% of clutter from the final dig list

  • 13. When do I use geophysical classification in the CERCLA process?

    Geophysical classification can be used in any step in the CERCLA process where information about buried metal is required. It will be most commonly used in the Investigation (Remedial Investigation and Feasibility Study) and Remediation (Remedial Design and Remedial Action) phases of the CERCLA process. Most of the demonstration effort has been focused on remedial actions as this is where the biggest potential for savings is found. ESTCP has supported several demonstrations that employed classification either as a component of the Remedial Investigation (RI) or as part of a Feasibility Study after completion of the RI. At some sites, classification may be a required component of a Remedial Design effort.

Site Specific Planning

  • 14. Do I need to do a site-specific treatability study? What can the conclusions of the ESTCP program tell me?

    In many cases a site-specific treatability study is not required. The ESTCP demonstration program comprised 26 demonstrations on a wide variety of sites. The similarity of clutter size and depth and munitions depths among the sites was striking. Performance data from one of the demonstrations will likely be applicable for purposes of planning and cost estimating. If the site presents unusual conditions (terrain, mix of munitions, clutter, geology, etc.) then a treatability study may be necessary for most accurate estimation of costs and performance.

  • 15. Do I need to do a surface clearance prior to geophysical classification?

    Yes, it is recommended. The decision to conduct a surface clearance prior to classification is no different than the decision to conduct a surface clearance prior to any munitions response. On most sites, safety concerns will dictate a surface clearance before geophysical investigation begins. In addition, it is almost always less expensive to pick up the source of potential clutter anomalies before the geophysical work begins than to process them afterwards.

  • 16. I have 20-mm projectiles on my site. Will geophysical classification work?

    The limited data in hand show that geophysical classification can be successful at sites with 20-mm projectiles although the efficiency is likely to be lower. ESTCP has conducted demonstrations at two sites with 20-mm projectiles but both have been less than ideal sites. A site with 20-mm projectiles is planned for the final year of the program. There are two things to keep in mind when considering classification on a site with 20-mm projectiles. The first is that the anomaly selection threshold will have to be quite low to reliably detect 20-mm projectiles to any appreciable depth. This will result in a large number of anomalies as a lot of small clutter is also detected. This is no different than searching for 20-mm projectiles with a traditional sensor. The second issue is that classification will not be as efficient if the targets of interest include 20-mm projectiles. There are many more fragments from larger munitions that look (in an EMI sense) like 20-mm projectiles than 37-mm projectiles. Initial indications are that no more than 60 or 65% of the clutter will be able to be correctly classified on a site with 20-mm projectiles.

  • 17. If I’m concerned about fuzes and other munitions components, can I still use geophysical classification?

    Geophysical classification can still be used if the targets of interest include fuzes and other munitions components but at a cost in classification efficiency. Just as in a case where 20-mm projectiles are included in the targets of interest, reliably detecting fuzes and small munitions components requires a lower detection threshold resulting in a larger number of anomalies to consider. Classification is also less efficient if these smaller items are considered TOI because there are many more fragments that have an EMI response close to that of fuzes and other small munitions components. Site managers should not plan on 80% or more correct classification of clutter in these cases; 50 to 60% is more likely.

    The preceding discussion does not apply in the case of large munitions components such as baseplates on an artillery range. These large components will not result in decreased classification efficiency as their response to an EMI sensor is far different than that of small munitions.

  • 18. Will geophysical classification identify all the munitions for removal?

    Yes, within the detection range of the instrument. The first requirement for success in an ESTCP demonstration or a production use of geophysical classification is 100% correct classification of TOI. TOI include all munitions, any surrogates used as seeds, and any large munitions parts that the site team decides need to be removed. We have repeatedly demonstrated that production geophysicists are capable of 100% correct classification of munitions.

  • 19. Can classification tell me the type of munition?

    Most classification schemes in use today are based on library matching. The derived EMI responses of the unknown object are compared to a library of polarizabilities for known munitions items and, if there is a sufficient match, an identification is made. The magnitude of the polarizabilities is a function of the size of the munition so it would be very unlikely to confuse a 37-mm projectile with a 105-mm projectile for example. There is always some noise in field measurements so an exact match cannot be guaranteed; it is possible to mistake an 81-mm mortar for a 75-mm projectile. Within size bands, it is perfectly reasonable to expect identification of a munition using classification methods.

  • 20. What density of anomalies is too dense to do classification?

    There is no one answer to this question; it depends on a number of factors. One limiting factor in anomaly density is the information content in the signals measured by the advanced sensors. Due to a finite number of receivers, bandwidth, and geometric coverage, present day sensors are only able to resolve up to three to four individual anomalies in the footprint of the sensor (~ 1 m square). The size of the munitions of concern, and thus the size of the anomaly footprint, is also a factor in the anomaly density that can be resolved. Classification has been successful at densities as high as 3,000 to 4,000 anomalies per acre when the majority of anomalies are small but users should not expect to be able to resolve anomaly densities much higher than that.

  • 21. What effect does geology have on classification ability? How do I determine whether the geology on my site is too challenging for classification to be reliable? How does this effect frequency of background measurements?

    In all but the most extreme cases, geologic response by itself does not impact classification. Short-scale (10’s to 100’s of centimeters scale) variation of the geologic response is the problem. To properly derive the EMI response of a buried metal item, the part of the measured signal due to the sensor itself and the nearby soil and rock must be subtracted. This is usually accomplished by collecting a measurement at a background location that has no buried metal near the item to be classified. This works well as long as the geologic response at the background location is close to that at the location of the unknown. If the geologic response is large and varying then this background subtraction scheme no longer works.

    If the anomaly detection survey is performed using an advanced EMI sensor the magnitude and variation of the background response will be apparent in the dynamic data. If a conventional instrument was used to conduct the detection survey, a spot check with transects collected using an advanced sensor may be necessary. If there is any question if the geology is a significant factor at a particular site, a small treatability study is probably needed.

    Moderate geologic response may not impact the frequency of background measurements but will necessitate increased attention to the spatial distribution of background locations. The field geophysicists should strive to collect a background measurement in close proximity to each unknown measurement to guard against geologic variation.

  • 22. How long does a classification project take? Is it significantly different than traditional projects?

    A classification project can often be significantly shorter than the same project conducted using traditional methods. The initial detection survey may take longer using advanced sensors, particularly if the site lends itself to use of arrays of traditional sensors. After that, every aspect of a classification project is faster. A team collecting cued data can cue roughly twice the number of anomalies a day that an intrusive crew can prosecute. On many sites, 80% of the digs can be avoided using classification. Over the course of a large project these factors add up to a large time saving using classification.

  • 23. How do I estimate potential cost savings?

    On a typical site where 80% of the clutter can be correctly classified, cost savings of 40% to 50% are possible using classification. Implementation of classification does require extra costs for additional data collection and analysis and the QC required for a quality project but the savings are the result of clutter digs avoided. A good rule of thumb is $125 to $200 saved per avoided dig. Sites with easy soil and shallow clutter will have a lower cost per dig and those with tougher conditions and deeper clutter will have a higher cost. In the ESTCP demonstrations, detection survey costs have been about the same as a survey using an individual EM61, cued data collection and analysis has been about $35 per anomaly, and the additional QC required by this approach has been about 10% of project costs.

Data Collection

  • 24. What is the difference between a detection and cued survey?

    A detection survey, sometimes referred to as a dynamic survey, is a 100%-coverage geophysical survey of a site. A detection survey is often accomplished using back and forth parallel survey tracks (akin to mowing the grass) with the track spacing a function of the sensor width, smallest munition to be detected, and anomaly density. The data from the geophysical sensor are combined with geolocation data (usually GPS data but other geolocation systems are used under tree cover) and mapped. Detections are declared at the locations of anomalous geophysical response compared to background. A detection survey can be conducted using either a traditional EMI sensor or one of the advanced EMI sensors developed for classification. Use of an advanced sensor for the detection survey allows for more sophisticated data analyses such as Informed Source Selection.

    A cued survey is a stationary data collection over a previously-identified anomaly for the purpose of acquiring higher-quality data to use for classification. Cued surveys only make sense using an advanced EMI sensor; not enough information is available in a single-coil, single-axis sensor such as the EM61-MK2 to make cued data collection worthwhile.

  • 25. How do I set the criteria for anomaly selection?

    Anomaly selection criteria should be set based on the site remedial action objectives and CSM. Once the munitions of concern (TOI) and the depth of concern are specified, it is straightforward to determine which combination of munition and depth results in the lowest amplitude anomaly and set that as the amplitude threshold for anomaly selection. If response amplitude is the sole criterion being used then the process is almost complete; the only remaining task is to compare this amplitude threshold to measured site noise. If this minimum amplitude is greater than five times the RMS noise at the site then the remedial objectives are achievable with the sensor selected and geophysical surveys can proceed. If, however, the minimum amplitude is within the site noise the anomaly selection amplitude will have to be increased and the site team will have to decide if they can tolerate the reduced depth of detection that results.

    If additional anomaly selection criteria in addition to amplitude will be employed then the amplitude threshold is only the first step. If anomaly detection is being performed using a traditional sensor such as the EM61-MK2, the choices for additional criteria are limited. Analysts have had some success in the past using criteria such as areal extent of the anomalous response and signal decay of the peak sounding to remove anomalies that could not possibly be due to a TOI from further consideration.

    If the anomaly detection survey was conducted with an advanced EMI sensor then the possibilities for additional anomaly selection criteria are much larger. These additional criteria can range from use of additional receive channels to discriminate against noise spikes at the simplest to a full analysis of all data collected over each anomaly to reject small, shallow clutter that could not possibly be TOI. This latter process is termed Informed Source Selection.

  • 26. What is “Informed Source Selection” and how does it work?

    “Informed Sourced Selection” refers to the use of the extra information inherent in the signals from advanced EMI sensors to select in the detection step only those buried metal sources that could be caused by a target of interest for further consideration. Note that this technique focuses on the buried metal items (sources) that result in the anomaly rather than the anomaly itself. The remedial objective is to remove TOI (that is, sources) rather than anomalies.

    Traditional anomaly selection has been based on the amplitude of the response observed during a geophysical survey. If the remedial objective is to identify 37-mm projectiles at one foot, the known minimum EM61 signal in gate 2 for that target is 5.2 mV. Thus, all anomalies with peak signal above 5.2 mV would be selected. This method definitely identifies all anomalies resulting from 37-mm projectiles down to one foot but it also identifies a large number of anomalies due to much smaller clutter near the surface. In some cases secondary filters such as areal extent of the anomaly and decay rate over the four channels of the EM61-MK2 are used to trim the list but these have proven to be only partially successful.

    The advanced EMI sensors can be configured to excite the target along multiple axes and sense the induced fields along three axes for much longer times. This provides at least an order of magnitude more information to use for source selection compared to a traditional sensor. This extra information can be used to discriminate against noise spikes caused by environmental interference, lessen the interference caused by site geology, and discriminate against anomalies caused by small, shallow clutter. Informed Source Selection has been shown to reduce the number of items that require cued data collection by up to a factor of three with no impact on the detection of munitions.

  • 27. What are the advantages and disadvantages of using an advanced sensor for the detection survey?

    There are two big advantages to using an advanced sensor for the detection survey. The receiver cubes in the advanced sensors (8- or 10-cm cubes) are much smaller than the 50- x 100-cm loop receiver in a traditional sensor. This, coupled with the precise position and orientation sensors on the advanced systems, results in much better location of the observed anomalies – often within 15 to 20 cm as opposed to 50 to 75 cm with traditional sensors. The second advantage is the ability to perform much more sophisticated anomaly selection (e.g. Informed Source Selection) when working with data from the advanced sensors. The additional information provided by the advanced EMI sensors affords the analyst the opportunity to use more than the observed signal amplitude to select only those anomalies that could result from a target of interest for further consideration.

    At present, the disadvantage to the use of advanced sensors for detection surveys is the production rate. The limited size and inability to configure them as arrays limits the daily survey coverage to 0.75 to 1.5 acres. On small sites this is not an issue but on larger, open sites, it is common to make an array of EM61s and survey up to 5 acres per day.

    Depending on site conditions, this limited survey coverage can be offset by the reduction in the number of anomalies that required cueing. On a site with moderately high anomaly density (1000 anomalies per acre or above) mostly consisting of small clutter, Informed Source Selection can reduce the number of cued measurements by up to a factor of three. On sites with low anomaly densities such as buffer areas, the limited number of anomalies does not afford enough reduction in cued data collection effort to offset the lower survey rates of the advanced sensors.

  • 28. If I do a detection survey with an advanced sensor, can I plan to do classification using that data?

    Advanced sensors blur the line between detection and classification phases of a munitions response project. Survey data collected using an advanced sensor can be used in multiple ways along the continuum from detection through classification. To take advantage of this, the sensor must be configured so that the buried metal object is interrogated from enough different directions to allow extraction of reliable response curves. This is more easily accomplished with the TEMTADS 2x2 sensor at present than the MetalMapper but variants of both sensors are being developed that will make this more routine.

    There are three possibilities for the use of dynamic data. In the first, sometimes referred to as “Informed Source Selection,” an initial analysis is performed for each anomaly and any anomalies that could not have been caused by a target of interest at the site are removed from the anomaly list. This process can radically pare the number of anomalies that require further investigation by eliminating those that result from small, near-surface fragments that give a large signal due to their shallow depth. During the San Luis Obispo Treatability Study this procedure was used to decrease the number of anomalies that required cueing by a factor of three. Details of this are in the table below.

    Statistics from San Luis Obispo Demonstration
    Amplitude Response Anomalies 16,202
    Informed Source Selection – Anomalies for Cueing 7,035
    Total Digs (5.1% of Amplitude Response Anomalies) 828
       Can’t Analyze 104
       Potential TOI (including 169 QC and validation seeds) 493
       Training 148
       Verification and Validation 83

    The second way to use dynamic data is to make classification decisions on anomalies that are clearly clutter or targets of interest using the dynamic data and only collecting cued data over those that are ambiguous from the dynamic data. This can result in an even greater reduction in the number anomalies requiring cued data collection.

    Finally, on sites with moderate anomaly density and low geologic background, it may be possible to perform classification using the dynamic data only. The example below is from the ESTCP demonstration at Spencer Range. The panel on the left shows the results achieved analyzing the cued data. These are very impressive with the initial part of the curve almost vertical indicating very efficient classification of TOI and a sharp break after 100% of the TOI had been identified. More than 85% of the clutter was correctly classified in this analysis. The panel on the right shows the results achieved analyzing the dynamic data only. The classification of TOI is not as efficient in this analysis as indicated by the slant of the initial part of the curve. Additionally, only 75% of the clutter is correctly classified by the analyst meaning about 30 extra digs would be required at this small site. The cost of these extra digs is less than the cued data collection step would have required.

  • 29. I have an old magnetometer or EM61 survey. Can I use that for the detection step? Can I use that for classification?

    It is possible that an existing magnetometer or EM61 survey can be used for anomaly detection in a classification project but it is not likely. The existing data must be closely scrutinized to see if they meet project DQOs before their use can be considered. Even if the lane spacing and detection threshold were appropriate for the new use the underlying positioning precision of the existing survey may lead to an inefficient classification process. Position uncertainties in anomaly location greater than 20 to 30 cm (either caused by inaccurate positioning system or excessive lane spacing) greatly increase the cost of a cued data collection on all but the least dense sites. On a site with normal to high anomaly density, the extra cost of a precisely-positioned survey will likely be repaid during cued data collection.

    Magnetometer and EM61 data cannot be used for classification. At any reasonable distance from the sensor, the magnetometer anomaly observed contains no information about shape and does not lend itself to classification. Early demonstrations in the ESTCP series showed that EM61 is unsuitable for efficient classification.

  • 30. What can I expect for production rate?

    Both the TEMTADS 2x2 and MetalMapper sensors can routinely achieve rates of 150 to 200 cued interrogations per day. The TEMTADS is more maneuverable so achieves the highest productivity when the anomalies are close together; the MetalMapper in its tractor-borne configuration moves faster so achieves the highest productivity when the anomalies are more widely spaced. The advanced EMI sensors are not optimized for deployment as arrays so the survey productivity is limited to what can be achieved with a single sensor. In survey mode, the MetalMapper routinely covers 1 to 1½ acres per day and the TEMTADS 2x2 covers ¾ to 1 acre per day.

Data Analysis

  • 31. What causes variation in the performance of analysts?

    Over the course of the ESTCP demonstrations, we have observed that there is a strong correlation of analyst training and experience with ability to correctly classify clutter. Although there is some variation within the bands, novices tend to perform worst and those with experience on many projects almost always achieve near-perfect results. Almost all analysts were able to correctly classify 100% of the targets of interest (only a handful of analysts that never seemed to understand the method missed TOI) but the efficiency with which they were able to correctly classify clutter varied with experience. The plot below illustrates this for the ESTCP Spencer demonstration. The best-performing analyst, and one of the most experienced, only required 53 clutter digs to identify all the TOI while one of the novices required over 800.

    One important component of the DoD Advanced Geophysical Classification Accreditation Program (DAGCAP) is the requirement for accredited organizations to properly train their employees, maintain records of that training, and document results of each employees demonstration of capabilities for the tasks they are assigned. This will allow site teams to be confident that skilled analysts are deployed to their projects.

  • 32. How does the performance of production contractors compare to developers?

    At all but the easiest sites, the developers of the classification software were more effective in removing clutter than the production geophysicists but not by enough to compromise the remediation goals. An example from the ESTCP demonstration at the former Spencer Range, TN is shown below. The left panel shows the performance of one of the developers when analyzing MetalMapper data. After about 50 digs for training, the results are almost perfect. This analyst identified 100% of the targets of interest while correctly classifying more than 90% of the clutter. The production geophysicist (right panel) also identified 100% of the targets of interest while correctly classifying 80% of the clutter. This level of performance (80% clutter reduction) is the basis for all cost projections in this document.

  • 33. What went wrong with the analysts that performed poorly?

    Although analysis of classification data is not inordinately difficult, it does require a level of care and attention to detail above that required for the routine analysis of EM61 data. Most of the handful of analysts with very poor performance did not pay enough attention to the project until near when their analysis was due. Then, if they were confused, there was no time to seek help or they rushed through the analysis at the deadline.

    Detailed SOPs with required QC checks would have taken care of most of the poor performance in the ESTCP demonstrations. The one or two remaining problems resulted from the wrong employee in the wrong place. The training and documentation requirements in the DoD Advanced Geophysical Classification Accreditation Program are designed to eliminate this problem.

  • 34. What is the TOI library?

    The TOI library refers to a collection of responses (polarizability decay curves or EMI fingerprints) corresponding to commonly occurring munitions items. Most classification schemes in use today involve comparing the EMI responses of the unknown objects to each entry in the library and using the match (or lack of match) to decide if the unknown is likely a munition or clutter.

    ESTCP is compiling a master TOI library of munitions’ responses along with complete metadata. This library will be maintained and updated as required by the US Army Corps of Engineers and hosted on a Government site. The current version of the library will be downloaded by the Government program manager at the start of the project and distributed to the geophysical contractor. For most projects, this will be the only library needed. Some sites may have unique munitions specific to the former mission and, in those cases, the master library will need to be supplemented with additional site-specific responses. Procedures for constructing these site-specific libraries will be described in the on-line help for UX-Analyze.

QC/QA

  • 35. Do I need to seed the site? What seeds should I use? How many? How deep? What accuracy of emplacement and documentation do I need?

    Yes, all munitions response projects should include blind seeds. How, and to what extent, to emplace seeds will be a site team decision. A robust QC/QA program is founded on blind seeds. QC seeds will be emplaced by the contractor as one component of their quality program and should be blind to the field geophysicists and analysts as specified in the contractor’s approved firewall plan. Validation (or QA) seeds will be emplaced by the Government or its representative and should be blind to all contractor personnel. The contractor’s performance on both the QC and validation seeds will be one of the lines of evidence that the site team will rely on when assessing project results.

    In many cases the seeds will be predominantly munitions surrogates such as Industry Standard Objects (ISOs). ISOs ( ESTCP GSV report) have the advantage of being inexpensive and readily available and, since they have been used at many sites across the US, it is easy to compare their response to that observed at other sites. Sometimes, the site team or the public will feel more comfortable if some of the seeds are inert versions of the munitions expected on the site. Whatever is chosen as seed items should not be so different from the TOI expected at the site that the seeds drive the anomaly selection criteria or classification thresholds. Correspondingly, the seeds should not be so easy to identify that they do not effectively test the contractor’s procedures.

    One of the primary purposes of a blind seeding program is to provide continual confirmation that project quality is being maintained. Emplacing enough seeds that each team will encounter a seed each day will ensure this. If the survey crew can cover one acre per day in dynamic mode, this will result in one seed per acre. If the cued team can cue 200 anomalies per day then one seed per 200 anomalies will be needed. This may be more than one per acre depending on anomaly density. Finally, if the intrusive crew can recover 100 items per day, there will need to be one seed per 100 digs expected.

    The point of the seed program is to confirm quality, not investigate sensor capabilities. All seeds should be emplaced at a depth at which they are expected to be detected and classified. Otherwise it is difficult to draw conclusions from a missed seed. This will normally be within the depth of concern in the remedial objectives. At ESTCP demonstrations, the seeds have been distributed down to the depth of concern with most of the seeds matching the depth where the munition is expected. For the San Luis Obispo Treatability Study this meant the 37-mm surrogates were emplaced down to 30 cm with the majority of the seeds in the 4 to 12 cm range at which 37-mm projectiles had been observed in the RI. Seed locations should be measured with a precision greater than the specifications for seed location accuracy in the QAPP. Usually this will mean cm-level GPS precision.

    It is good practice for the contractor to emplace their QC seeds so that they fully cover the remedial depth and conditions expected on the project; they may even bias the depths to the lower end of the remedial objective. This will allow the contractor to uncover any process issues that may cause them to be unable to accomplish the remedial objectives. The validation seeds will normally be emplaced at depths more in line with the expected depth at the site as the consequences of a validation seed failure are significant.

  • 36. What products and documentation can the project team expect to see during the course of the project?

    The first document a stakeholder should expect is the project QAPP. Among other things, the QAPP will list the Data Quality Objectives, the corresponding Measurement Performance Criteria and Measurement Quality Objectives, and the preliminary Validation and Verification plans. As the project proceeds a surface clearance technical memorandum, a QC seed report, and an IVS memorandum will be submitted along with weekly QC summary reports. At the conclusion of all geophysical measurements, a ranked anomaly list will be submitted with a preliminary stop dig point. This will be accompanied by a summary memo that describes the classification approach used, the various thresholds employed in the ranking, and any other details necessary to document the decisions made. At the conclusion of intrusive work, the contractor will submit the final ranked anomaly list, the results of all intrusive investigations and a proposed final validation and verification plan.

  • 37. How much QC will be required?

    Site teams contemplating a project using classification should plan for extra QC and QA effort compared to a traditional project. Part of this extra expense is due to the level of confidence required to leave significant metal objects in the ground and part is due to the additional quality checks that are possible as the field transitions to the use of GCMR-QAPPs. Based on a handful of ESTCP demonstrations, we have found that about 10% of the total project funding has been devoted to QC with a smaller amount required for Government QA. As contractors gain experience with classification and the QC required and the analysis software evolves to better support these needs this cost will come down but will likely always be higher than projects in the past.

  • 38. How much effort is required for QA vs. QC?

    QA requires less effort than QC during most phases of the project. As part of the site prep task, the contractor QC geophysicist will prepare a surface clearance tech memo, a seed emplacement report and an IVS report. Each of these will require QA effort. During the data collection phases of the project, the QC geophysicist will have daily QC checks to perform and will combine these daily reports into a weekly QA submission. The government or stakeholder QA geophysicist may choose to examine these submissions in great detail for the first one or two weeks but after that the burden should be lower. The government QA geophysicist for the San Luis Obispo demonstration reports spending about ½ day on each weekly QA report.

    As the analysis software matures and the QC and QA report formats are standardized, the effort required for both QC and QA will likely be reduced.

  • 39. Who can help me with QA and oversight?

    US Army Corps of Engineers program managers can contact USACE geophysicists for help with QA and oversight. Army Environmental Command and Naval Facilities Engineering Command PMs can expect some assistance from headquarters but may need to contract for third-party QA help. Some states have hired third-party QA consultants as well to help them with oversight on geophysical classification projects. This is an option for stakeholders associated with a munitions response involving geophysical classification.

  • 40. During the digging, what information should I be collecting and what should it be used for?

    At a minimum, the intrusive crew should record the depth and size of all recovered items along with a good photograph and a precise position measurement. These data will be used by the analyst to confirm that the recovered item(s) are consistent with the geophysical data. The position measurement can be used to confirm that an incorrect nearby item was not dug.

  • 41. Do I need to have a qualified geophysicist on the government QA team?

    Yes. It is a wise policy to have a qualified geophysicist on the Government QA team. A geophysicist will be able to quickly verify the correctness of data collection and analysis reports, judge the completeness of root-cause analyses, and render an informed judgement on the adequacy of any proposed corrective actions. During the verification and validation phase, the QA geophysicist will be able to quickly choose items for validation digs.

Project Conclusion

  • 42. If we don’t dig all of the anomalies, how can we demonstrate that classification was successful at my site?

    A number of lines of evidence will contribute to confidence that classification was successful at a particular site. Throughout the project, QC checks described in Worksheet 22 of the GCMR-QAPP are being made daily and QA checks are being performed weekly. Perhaps the most important of these are the successful detection and correct classification of both the QC and validation (or QA) seeds. If all the Measurement Quality Objectives (MQOs) have been satisfied throughout the project then the site team can have confidence that the Measurement Performance Criteria have been satisfied and consequently the Data Quality Objectives have been met.

    At the conclusion of field work the verification and validation procedures outlined in Worksheets 34 and 35 of the QAPP will be performed as the final evidence of success. For most projects these will include a verification that the correct stop-dig threshold has been chosen and validation of the entire classification process.

    Verification of the stop-dig threshold can be accomplished by simply digging some number of anomalies past the contractor’s proposed stop-dig threshold. The beta draft of the GCMR-QAPP template recommends digging 200 anomalies past the last TOI encountered. If the contractor chooses a very conservative threshold then no (or few) extra digs may be required. Alternatively, if the contractor is aggressive and only recommends digging a handful of anomalies past the last TOI then 200 more digs will be required.

    Validation of the classification process can also be accomplished relatively easily. The successful detection and classification of all blind seeds has already shown that given accurate EMI responses (features) that correct decisions can be made and that the intrusive crew can recover the correct item. All that is left is to show that the features that were the basis for the decision on each item were correct. Excavation of a number of anomalies (the GCMR-QAPP template recommends 200 for a large project) that were classified as not-TOI and comparison of the physical size and shape of the recovered item with the analyst’s basis for her classification decision will provide confidence that the classification decisions were based on an accurate description of each item.

  • 43. What are my options for verification and validation? How does the team decide that the site response action is done?

    Verification and validation activities at the conclusion of a project are only two of the many lines of evidence that will give site managers confidence that response at the site is “done.” Throughout the project the QC checks being made daily and the weekly QA checks are also very important confidence-building activities. These include the successful detection and correct classification of both the QC and QA (or validation) seeds. If all the Measurement Quality Objectives (MQOs) have been satisfied throughout the project then the verification and validation procedures outlined in Worksheets 34 and 35 of the QAPP are all that remain. For most projects these will include a verification that the correct stop-dig threshold has been chosen and validation of the entire classification process.

    Verification of the stop-dig threshold can be accomplished by simply digging some number of anomalies past the contractor’s proposed stop-dig threshold. The beta draft of the GCMR-QAPP template recommends digging 200 anomalies past the last TOI encountered. If the contractor chooses a very conservative threshold then no (or few) extra digs may be required. Alternatively, if the contractor is aggressive and only recommends digging a handful of anomalies past the last TOI then nearly 200 more digs will be required.

    Validation of the classification process can also be accomplished relatively easily. The successful detection and classification of all blind seeds has already shown that given accurate EMI responses (features) that correct decisions can be made and that the intrusive crew can recover the correct item. All that is left is to show that the features that were the basis for the decision on each item were correct. Excavation of a number of anomalies (the GCMR-QAPP template recommends 200 for a large project) that were classified as not TOI and comparison of the physical size and shape of the recovered item with the analyst’s basis for her classification decision will provide confidence that the classification decisions were based on an accurate description of each item.

  • 44. What types of metal debris should I expect to find on my site after a munitions response using classification?

    The vast majority of metal items on a typical munitions site are thumb-sized fragments and smaller. The photograph below shows a collection of fragments collected during a recent ESTCP demonstration. If the question is posed as “What is the average (or median or typical) metal debris on my site?” then the answer is in the photograph.

    There are a small number of larger, recognizable fragments on most sites. Items such as fin assemblies on mortar targets, baseplates on artillery ranges, box fins on practice bomb targets, etc. are normally classified as not targets of interest and left on the site. In cases where particular site use or stakeholder sensitivity requires the removal of these larger items they can be added to the list of TOI.

  • 45. At the end of a project, can you tell me what the metal items are that were left on the site and where?

    Yes. The master project database delivered by the contractor will have an entry corresponding to every item classified as clutter with the item’s EMI signature and rough size. Because the classification process is designed to only leave items behind that can be affirmatively classified as clutter, much more is known about the site after a remedial action using classification then after a traditional project.

After the ESTCP Demonstration Program

  • 46. How do I determine the capability of new sensors that have not been demonstrated by ESTCP?

    The sensors that have been used successfully to date are all multi-axis, multi-coil electromagnetic induction (EMI) sensors. Other sensor modalities may be developed to perform the classification function but any new sensor will have to share several characteristics with the advanced EMI sensors. The primary threshold for a new sensor is “no black boxes allowed.” Without a transparent process, stakeholders will not be able to judge the validity of a classification project. This means that the purveyor of the new sensor should be able to provide a detailed, written description of the sensing modality and analysis approach, predict in advance the signals and characteristics expected from the munitions items likely to be encountered on the site, and have demonstrated the performance of the new sensor on a test site or previous project.

  • 47. How do I determine the veracity of alternative classification methods?

    The primary measure of the usefulness of any classification method is demonstrated performance. Results from a blind test site, analysis of the data from a prior ESTCP demonstration, or a previous project will allow a prospective user to judge the reliability of a proposed method. Successful detection and correct classification of all TOI, including all seeds, and the fraction of clutter correctly classified are the measures that should be examined.

  • 48. What remains untested/unknown at the conclusion of the program?

    There are a handful of issues regarding the applicability that remain to be tested at this point in the ESTCP program:

    • What are the limits of geologic interference for successful classification? We have documented the effect of moderate geology at the Beale demonstration and have shown that the severe geology at Waikoloa makes classification of small items very difficult. The break point between “extra care required” and “too difficult to be effective” is still to be determined precisely.
    • Sites with 20-mm projectiles. Several demonstrations have shown that 20-mm projectiles can be classified but we have not yet conducted a demonstration at a site (other than a strafing range) where 20-mm projectiles are mixed with other munitions. Because of this we are unable to document the decreased classification efficiency due to the presence of smaller munitions items. A search for a good example site is underway.
    • Urban and industrial areas. The clutter environment in urban and industrial areas is likely to be quite different from that on an isolated range. The ESTCP program has not documented the classification performance possible in these environments.
    • OB/OD areas. These sites are likely to contain many partial and distorted munitions that are still hazardous. The ability to correctly classify these items using the current TOI library has not been determined. ESTCP has a demonstration planned at the Pueblo Army Depot that should address this issue.

    If a site in the planning stage presents one of these characteristics, it may be wise to perform a treatability study at the site.

Share