The objective of this project is to further develop and investigate the utility of a method of remotely operating robotic systems that require dexterous and precise motion – in particular when making physical contact with other objects such as munitions. This project will develop and test technology for safely and efficiently remediating military munitions at underwater sites. The concept of using non-contact sensors to stream visual renderings and haptic (force) feedback to a remote pilot for subsea munitions remediation was explored (at the proof of concept level) under SERDP SEED MR-2323. Results of that work motivate further research and development and comprehensive testing to overcome challenges associated with removal of partially-buried objects and implementation onboard a moving platform in the field.

The overarching goal is to remove ordnance from marine environments with a remotely-operated co-robotic system (human operator working in conjunction with a robot) that provides augmented feedback in a way that effectively leverages human perceptive capability and decision-making. This approach, which emphasizes use of an advanced, but intuitive, user interface that renders visual and haptic feedback to the pilot, maximizes the benefit and performance of the operator and increases the scope of tasks that can be completed with low-cost, easily-deployed robotic platforms. Several key innovations make our approach transformative – enabling complex tasks to be executed precisely according to the operator’s intent. The use of low-cost, precision robotic manipulators, real-time non-contact underwater sensors, and algorithms running in real-time on Graphical Processing Unit (GPU)-based processing systems, allows for an immersive experience including ‘sense of touch’ feedback to assist and guide remediation operations. Researchers believe this approach to combining low-cost hardware with an advanced interface and augmented feedback offers the highest likelihood for Department of Defense (DoD) to mitigate underwater munitions in a safe, cost-effective, environmentally preferable manner – without requiring diver intervention or blow-in-place strategies.

The primary benefit of this work is advancement of remediation technology that reduces dependence on diver intervention and blow-in-place strategies. A key advantage is the ability to guide operations with force feedback (sense of touch) so that a pilot can direct a robot to grasp munitions safely and predictably. The benefits of this are co-robot actions executed with a high level of precision and repeatability. This is done using haptic force feedback and visualization, enabling an interactive experience that makes precise planning and execution intuitive. The operator may plan and evaluate maneuvers before contact is made between grippers and the ordnance. With haptic rendering, the operator feels the object as the end effector approaches a safe grasp configuration. Once an effective maneuver is identified, execution follows seamlessly. The concept is much like the “snap-to-grid” function of a computer mouse. The approach also maintains the authority of the human operator but provides cues that facilitate operations, seamlessly combining sensor fusion and automation with human control.