New framework for oceanographic research provides potential for broader access to deep sea scientific exploration

Woods Hole, Mass. (August 23, 2023) -- Scientific exploration of the deep ocean has largely remained inaccessible to most people because of barriers to access due to infrastructure, training, and physical ability requirements for at-sea oceanographic research.

Now, a new and innovative framework for oceanographic research provides a way for shore-based scientists, citizen scientists, and the general public to seamlessly observe and control robotic sampling processes.

The Shared Autonomy for Remote Collaboration (SHARC) framework “enables remote participants to conduct shipboard operations and control robotic manipulators” – such as on remotely operated vehicles (ROVs) –“using only a basic internet connection and consumer-grade hardware, regardless of their prior piloting experience,” according to a paper in Science Robotics, “Enhancing scientific exploration of the deep sea through shared autonomy in remote manipulation.” The framework has been developed by a research team from the Woods Hole Oceanographic Institution (WHOI), the Massachusetts Institute of Technology (MIT), and the Toyota Technological Institute at Chicago (TTIC).

The SHARC framework enables real-time collaboration between multiple remote operators, who can issue goal-directed commands through simple speech and hand gestures while wearing virtual reality goggles in an intuitive three-dimensional workspace representation.

Through SHARC, “we can open up the operational aspects of deep sea exploration to citizen scientists, whether they be kids in a classroom or people who can’t be present on a ship because of logistical or physical requirements,” said co-author Richard Camilli, a principal investigator for the project and scientist in WHOI’s Applied Ocean Physics and Engineering Department. “Citizen scientists can interact with the ROV’s robotic manipulator arm in a virtual world, somewhat analogous to the science fiction ‘holodeck’ holographic system used on Federation starships on Star Trek.”

The human-robot interaction—sometimes referred to as shared autonomy—that SHARC enables, delegates responsibilities between the robot and the human operator based on their complementary strengths. The robot, for instance, can handle kinematics, motion planning, obstacle avoidance, and other low-level tasks, while human operators take responsibility for high-level scene understanding, goal selection, and task-level planning. In addition, SHARC allows for parallel, rather than sequential operation.

“We just give the robot its goal, and it finds a solution,” said Camilli. “People and the robot can collaborate together, where we’re not waiting for one thing to happen in order to do the next thing. While the robotic arm is executing a task, we can be focusing on the next goal.”

In September 2021, during the height of the Covid pandemic, scientists successfully tested SHARC. During an oceanographic expedition in the San Pedro Basin of the Eastern Pacific Ocean, SHARC team members operated WHOI’s Nereid Under Ice (NUI) hybrid ROV from thousands of kilometers away using SHARC’s virtual reality and desktop interfaces. The team members – physically located in Chicago, Boston, and Woods Hole – collaboratively collected a physical push core sample and recorded in-situ X-ray fluorescence measurements of seafloor microbial mats and sediments at water depths exceeding 1000 meters.

“This paper really highlights shared autonomy's potential to help democratize access to the deep sea,” said lead author Amy Phung, who is a student in the MIT-WHOI Joint Program in Oceanography/Applied Ocean Science and Engineering, graduate degree program. Phung was one of the scientists operating the NUI vehicle during the 2021 test of SHARC. “With SHARC, our shore-side team was able to collect seafloor samples from over 4000 kilometers away without specialized hardware or extensive prior training. In the future, I believe that further advancements in robotics and autonomy research can someday enable shore-side scientists, students, and enthusiasts to actively participate in and contribute to deep-ocean exploration operations as they occur, which in turn can help to foster ocean literacy among the general public.”

“Whether it is on land, air, or in the ocean, most robots that operate today do so in one of two distinctly different ways: full autonomy or full remote control by highly trained pilots, with the latter being standard for settings like underwater manipulation that involve complex interactions between robots and their environment. This paper describes a new framework that enables robots to operate in between these two extremes in a way that takes advantage of the complementary capabilities of robots and humans,” said co-author Matthew Walter, associate professor at TTIC. Walter also currently is a WHOI guest investigator; previously, he was a student in the MIT-WHOI Joint Program. “SHARC allows people with little-to-no training to perform sophisticated tasks with deep-sea robots, with pilot oversight, from the comfort of their homes and offices through a combination of speech and virtual reality and, in turn, promises to redefine how we use robots for marine science and engineering.”

In addition, SHARC is not dependent on a specific kind of ROV, manipulator arm, or other factors. “We can apply the same SHARC technology with totally different robotic arms and vehicles in completely different contexts,” said Camilli. The SHARC framework “is flexible and hardware-agnostic.”

“By  using the SHARC framework for scientific exploration in the deep sea – which is a very challenging and unstructured environment – it highlights that this technology can be transferable to many different operational contexts, as well, potentially including subsea scientific infrastructure maintenance, deep space operations, nuclear decommissioning, and even unexploded ordnance remediation,” Camilli added.

Funding for this research was provided by the National Science Foundation (NSF) National Robotics Initiative, NASA’s Planetary Science and Technology from Analog Research (PSTAR) program, NSF Graduate Research Fellowship, and the Link Foundation. The NOAA Ocean Exploration Cooperative Institute provided in-kind support with ship and robotic vehicle operations during 2021 Pacific Ocean demonstrations in the San Pedro Basin.

###

Authors: Amy Phung^1,2, Gideon Billings², Andrea F. Daniele³, Matthew R. Walter³, Richard Camilli^{2 *}

Affiliations:

¹Massachusetts Institute of Technology; Cambridge, MA

²Woods Hole Oceanographic Institution; Falmouth, MA

³Toyota Technological Institute at Chicago; Chicago, IL

^*Corresponding Author

About Woods Hole Oceanographic Institution

The Woods Hole Oceanographic Institution (WHOI) is a private, non-profit organization on Cape Cod, Massachusetts, dedicated to marine research, engineering, and higher education. Established in 1930, its primary mission is to understand the ocean and its interaction with the Earth as a whole, and to communicate an understanding of the ocean’s role in the changing global environment. WHOI’s pioneering discoveries stem from an ideal combination of science and engineering—one that has made it one of the most trusted and technically advanced leaders in basic and applied ocean research and exploration anywhere. WHOI is known for its multidisciplinary approach, superior ship operations, and unparalleled deep-sea robotics capabilities. We play a leading role in ocean observation and operate the most extensive suite of data-gathering platforms in the world. Top scientists, engineers, and students collaborate on more than 800 concurrent projects worldwide—both above and below the waves—pushing the boundaries of knowledge and possibility. For more information, please visit www.whoi.edu

Key takeaways:

•          Scientific exploration of the deep ocean has largely remained inaccessible to most people because of barriers to access due to infrastructure, training, and physical ability requirements for at-sea oceanographic research. Now, a new and innovative framework for oceanographic research provides a way for shore-based scientists, citizen scientists, and the general public to seamlessly observe and control robotic sampling processes, with pilot oversight.

•          The Shared Autonomy for Remote Collaboration (SHARC) framework “enables remote participants to conduct shipboard operations and control robotic manipulators” – such as on remotely operated vehicles (ROVs) –“using only a basic internet connection and consumer-grade hardware, regardless of their prior piloting experience,” according to a new paper in Science Robotics.

•          The SHARC framework enables real-time collaboration between multiple remote operators, who can issue goal-directed commands through simple speech and hand gestures while wearing virtual reality goggles in an intuitive three-dimensional workspace representation.

•          “SHARC’s ability to relax infrastructure requirements enables remote scientists and other novice shore-side users to participate without requiring additional bandwidth from the ship or specialized hardware. Thus, SHARC provides a promising avenue for democratizing access to deep-ocean science and expanding scientific engagement to a broader audience, including classrooms and the general public.”