Another DARPA SubT team that’s been doing it’s own in-cave practice to prepare for the final event next year is Team CoSTAR, from NASA JPL and Caltech. CoSTAR, of course, won the SubT Urban Circuit earlier this year with their team of wheeled and legged robots, which was awesome—but you’d maybe expect that for a group developing planetary exploration robots, places like urban environments and man-made tunnels wouldn’t necessarily be their top priority, right? Unless there’s something they’re not telling us, and I’m sure it’s aliens.*
NASA’s been working on robotic cave exploration for a long, long time, and Team CoSTAR (and the SubT Challenge) fit right in with that. The team and its robots have been spending some time in lava tubes, and we asked some folks from NASA JPL how it’s been going.
This interview features the following roboticists from NASA JPL:
- Ali Agha, CoSTAR Team Lead
- Ben Morrell, Deputy Task Manager, Team CoSTAR
- Jen Blank, Science Lead, NASA BRAILLE
IEEE Spectrum: What cave environments were you able to test in? What were your criteria for these places, and how did you find them?
Ali Agha: Natural caves are an important target for future NASA missions, offering potential locations of past biology, current biology, and future locations to provide protection for human habitation. In a collaboration with NASA Science Mission Directorate, we have been looking into exploring Martian-analog caves at Lava Beds National Monument in Northern California. We have a long-term relationship with these caves for NASA projects; and these caves have also been our testing location to prepare for DARPA Subterranean Challenge Cave Circuit.
Ben Morrell: The cave segment of the Subterranean Challenge is an extremely exciting one for our team because the test locations align so well with NASA’s long term goals: To explore caves on the moon and Mars, and specifically, lava tubes, the caves formed from volcanic flows that we know are present on these other worlds. A NASA team, led by Jen Blank, was already testing robotic science exploration of lava tubes, and had selected the Lava Beds National Monument as an excellent analog for lava tubes on Mars. While great for NASA, this site also provides a rich diversity of challenges, with over 800 caves in the National Monument.
Team CoSTAR took its Spot robot to explore Martian-analog extreme terrains and lava tubes in Lava Bed National Monument, Tulelake, Calif.Photo: Team CoSTAR
What do you feel like the biggest difference was, going from an urban environment to a cave environment?
Agha: The biggest difference is on traversability. Urban underground environments have higher levels of verticality, including stairs, multi-levels, and challenging maze-like structures. Caves, on the other hands, have very harsh and extreme terrains, even difficult to traverse for humans. This puts a lot of stress on the traversability and hazard avoidance components of the autonomy solution.
Morrell: The 3D maps are infinitely more beautiful, revealing the wonder of mother nature as the robots explore the environment. The interesting shape of the caves both helps lidar-based mapping, with many distinct geometric features, as well as bringing additional challenges with more subtle and consistent vertical variations than in previous environments.
Can you give some examples of cave-specific challenges that were surprising to you?
Agha: The lava flow terrain (aa and pahoehoe) was more extreme than what we were expecting; to the extent that our team members were not able to walk on certain parts of the terrain.
Morrell: The caves brought such a rich variety of traversability challenges that called for new ways to look at local planning that consider the whole path over a hazard. Steep “lava-fall” slopes, for instance, had paths our robots could traverse, but only if approached in the appropriate way. This is a challenge even for humans, yet we needed to look at how to program our robots to do that.
One surprise particular to lava tubes was the otherworldly friction of the surfaces. The lava flows are by far the most unforgiving and grippy surfaces we have tested on. This turned out to be a large benefit for legged robots, but caused a lot of issues for wheeled robots relying on skid-steering to turn.
Jen Blank: The whole team was surprisingly encouraged with the facility with which the [Boston Dynamics] legged robot was able to navigate the different lava terrains—from the pahoehoe or rope-like flow texture of much of the cave floors to the aa or blocky, irregular and angular, cauliflower-sized lava. The robot was able to move across table-sized, loose tabular fragments of ceiling collapse and approach the edges of a cave where ancient flows cooled to leave a ledge or “bathtub ring” of chilled lava behind. Also positive was the robot’s ability to enter and exit the primary cave of our study—though we picked a cave with the easiest access we could find (i.e., one with a combination of natural and constructed steps and pathway).
How did your approach change from the systems you used for the Urban Circuit?
Agha: Due to COVID-19 restrictions and challenges, we haven't had a lot of chances to work with hardware during the last several months. So our hardware solutions have not changed much. But from a software perspective, we have been improving various components of the algorithms in simulation environments, including our planning and perception methods.
Morrell: One of the large areas of development in our team has been in our planning algorithms, with substantial theoretical and implementation upgrades. These developments were motivated from experiences in the Urban competition, with large scale environments containing many rooms, as well cave environments that could vary from narrow corridors to large open caverns. We have also focused on support for the operator, building tools to offload tasks and simplify the actions required to manage the robot team.
What kind of experience did your robot operators have in the cave, and how was it different from Tunnel and Urban?
Morrell: Our test locations were smaller, and less complex relative to Urban, hence we adjusted testing to shorter timeframes and fewer robots. This adjustment, along with advancements in global planning, autonomy and operator assistance tools, made the operator’s experience more relaxed than in the previous circuits. This was a win for the team as a relaxed operator is the goal we are all aiming for, and we feel we are making gradual but consistent progress towards this goal.
One aspect that was more challenging compared to Urban was recognizing potentially hazardous features in the environment. Rather than stairs, the operator had to recognize low ceilings and sudden drop-offs in an unstructured 3D map.
Did you hold a mock Cave Circuit Competition? If so, how did it go, and what did you learn?
Morrell: We did aim to hold a mock Cave Circuit at our test site in Lava Beds National Monument. Our approach to setting up a mock competition leveraged the previous scans of these caves by the NASA science team, which we used as our ground truth map (similar to those provided by DARPA after the Tunnel and Urban Circuits). We used the map to select artifact locations, and survey marker locations. In the field, we used a total station with those survey markers, to measure the position of our portable calibration gate relative to the map. Using these gate coordinates for robot calibration, we could then run the tests like the DARPA competitions!
Lessons learned in the setup? Urban environments are wonderfully structured with convenient right angles, corners, straight corridors amenable to surveying and floor plans with which to set up ground truth artifact locations and calibration gates. You can also rely on the expectation for flat floors to sanity check final configuration results as well. We found this much more challenging in the cave environment where it is difficult to tell if you are off by a degree or two in your survey, and hard to be confident in the final result.
The mock competition was invaluable to components that are hard to otherwise evaluate: operations under stress, artifact global localization and whether our coverage planning actually allows us to see all artifacts. We learned the value of working on reducing the operator load, how to adjust our planners to accommodate our artifact sensing, and some of the areas for improvement in our artifact localization pipeline.
The researchers were impressed by Spot’s ability to navigate the different lava terrains.Photo: Team CoSTAR
Can you describe any particularly notable successes or failures?
Morrell: In one of our tests, we were extremely happy to have a fully autonomous run of our Spot robot, with exploration of all of the cave environment. Full autonomy is the goal we are pushing towards for NASA’s cave exploration goals, as well as for the competition, hence demonstrations towards this end are great successes for the team.
Another success was the remarkably pain-free transition of many components from sim to real, such as planners, autonomy and operations tools. This is testament to the team’s efforts in setting up powerful simulation systems and in doing monthly mini games in simulation (our own “mini-Virtual Cave Circuits”).
We found that wheeled robots struggled in the lava tube environments, with wear and tear causing some fatal hardware failures in the field that were hard to address under the restrictions with COVID.
If you could go back and repeat the Tunnel Circuit and Urban Circuit, how do you think you would do?
Agha: Different aspects of our overall autonomy solution, referred to as NeBula, have been improved during the last several months. In particular, the traversibility and planning aspects have been improved; and we believe with our current solutions we would have been able to explore further grounds in both tunnel and urban competitions.
Morrell: Adding to Ali’s comments, it would be very exciting to be able to try again! We believe our systems will be much more efficient in exploration, more accurate in localization, and have less downtime through enhanced autonomy and operator support. How would we go compared to the other teams? We can’t wait to find out in the final competition! We know the other teams have been making incredible progress and we are doing all we can to keep up, and look forward to testing with the other teams soon!
How are you feeling about the combined circuit for the SubT Final?
Agha: We are really excited to see what DARPA has in mind for the finals circuit and curious to see how three environments a tunnel cave and urban can be merged in one course.
Morrell: The final circuit really presents an unrivalled opportunity to assess just how capable our autonomous systems are. It is the perfect test, after years of research and development, to show what our team has been able to accomplish. However, there is still a long way to go and a lot to do. The ever motivating competitive element and knowledge of the continuing advances of other teams is driving us to keep pushing and improving.
Now that you’ve been through Tunnel and Urban and your own version of Cave, do you feel like you’re approaching a generalizable solution for underground environments?
Agha: Yes our approach from the beginning was targeting a solution that can be generalized across a wide range of environment types. Field testing in different types of environments has definitely helped with identifying aspects of the solution that are not resilient to environment change and has given us the opportunity to enhance those components and come up with more general solutions that can work across different types of environments.
Morrell: Commendations to the DARPA staff designing the competition, as the variety of challenges through the large scale of Tunnel, the complexity and multi-level aspects of Urban and the extreme terrains of Cave truly drive for a generally capable solution. We know, however, that there is a greater diversity of mines, caves and urban environments than what we have tested in, and hence we will be pushing to continue testing in a variety of environments to unveil the unknown-unknowns (other than the ever-secret final competition setup), to continue to push our solution to be more robust and generalizable.
*It’s not aliens.
