It’s not a matter of if AI-driven, autonomous technology will make it to the recreational marine scene. It’s a matter of when.
Back in the mid-᾽80s, in the midst of the Gulf of Mexico, well south of Galveston, Texas, and well north of the huge semi-submersible oil rigs that inhabit the Gulf’s truly deep, ink-blue water, there used to be—and still may be for all I know—a confluence of seven or eight shipping lanes that navigators in the offshore supply boat biz used to call “Malfunction Junction.” At just about any time of day or night, the locality was predictably overrun with immense, fast-moving ships, some inbound for Galveston’s Traffic Separation Scheme, some outbound for the rest of the world, some monitoring their VHFs, some not monitoring their VHFs and all of them maintaining different speeds and headings. As you’d imagine, shaping a safe, reasonably efficient course through this radically congested miasma with little more to rely on than a carefully tuned radar, a radio and a good pair of binoculars was often challenging, sometimes frightening and always instructive.
Indeed, just one jaunt through Malfunction Junction on a pitch-black night was usually all it took to convince virtually any skipper that, in light of the speed and intensity of the traffic involved, the majority of the U.S. Coast Guard’s Navigation Rules, often colloquially known as “The Rules of the Nautical Road,” offered no guidance or consolation whatsoever. Because you had to simultaneously deal with so many vessels traveling in so many different directions, there was no way to immediately determine which one of the more common rules should apply. While you were overtaking one vessel, you could also be meeting another head-on and crossing the bow of a third. Circumstances that often bordered on in extremis as defined by the rules were simply too complex to handle with most of the time-honored strictures. Fast navigational decisions had to be made, in large part, based on past experience and intuition.
I reminisced about these true-life navigational tribulations toward the end of a phone call I recently had with Capt. Lauren Lamm, a test pilot for Boston-based Sea Machines Robotics, an up-and-coming enterprise that, since its founding in 2015, has been developing AI-driven, autonomous vessel technology for military and commercial markets. Lamm was in her shoreside office, having just returned from Boston Harbor and the cockpit of a 29-foot Metal Shark Defiant, a welded-aluminum, Sea Machines-outfitted autonomous test vessel of the type the U.S. Coast Guard has just put into service (reportedly in New London, Connecticut) for research and development purposes. A tech maven with Sea Machines since 2018, Lamm is a graduate of Massachusetts Maritime Academy with five years of on-the-job experience as a Dynamic Positioning Officer aboard OSVs (Offshore Supply Vessels) servicing oil rigs in the Gulf of Mexico. She laughed at my tales of Malfunction Junction, saying she was no stranger to such navigational hotspots herself.
“So, Lauren,” I asked, toward the end of our very informative conversation about vessel autonomy, a subject that’s gaining traction with mariners the world over, “you’re telling me that the Sea Machines module that’s installed on that Metal Shark of yours is capable of handling just about all marine traffic situations, including those that arise in places like that little jewel south of Galveston? And it does this autonomously—look ma, no hands?”
“Yes,” Lamm replied, without missing a beat. “Right now, we have ARPA and AIS as our main sensors for collision avoidance but by the end of the year we’ll have camera-based perception as well. So, machine learning will soon help us identify objects in the environment, whether fishing boats, vessels under sail, different kinds of light configurations, debris—that sort of thing. And this will push the technology’s capabilities even further.”
The American Bureau of Shipping presently defines an autonomous vessel as “a marine vessel with sensors, automated navigation, propulsion and auxiliary systems, with the necessary decision logic to follow mission plans, sense the environment, adjust mission execution according to the environment and potentially operate without human intervention.” And because autonomous technology is gradually reshaping the modern maritime industry, adds the ABS, it is currently working closely with members, industry regulators and other interested parties “in the development of autonomous vessel design, the management of associated risks and the overall implementation of vessel autonomy.”
Both Sea Machines Robotics and Metal Shark Boats are deeply committed to advancing the technology that regulatory bodies like the ABS are so intensely focused on today. And there are many other high-profile business entities that are equally committed, including Google, Northrup Grumman, Lockheed Martin, Kongsberg, Wartsila, Honeywell International and IBM. Moreover, just three months ago, the U.S. Department of Defense announced that one of the U.S. Navy’s already extensive array of autonomous ships—part of its Ghost Fleet Overlord Program—had safely and successfully completed an autonomous voyage of some 4,700 nautical miles, starting from Mobile, Alabama, and arriving in Port Hueneme, California, after transiting the Panama Canal, where a small Navy crew only temporarily took control to deal with lines, pilots and incidentals.
“Overlord is part of an effort to accelerate the Navy’s push to incorporate autonomous vessels within its fleet to better expand the reach of manned vessels,” announced DOD officials. “Autonomy includes more than just straight-line passage through large areas of the ocean; it also involves such things as collision avoidance and following the rules of the sea.”
The motives behind the development of autonomous military vessels are only somewhat different from those that are currently driving commercial shipping companies to either build new ships with varying levels of autonomy or retrofit older ones. Both military and commercial interests, after all, expect to increase safety on the high seas by either sharing vessel operations with AI or perhaps ultimately turning control over altogether. The vast majority of marine accidents, military as well as commercial, are caused by human error. And it’s commonly acknowledged that autonomous or remotely operated military, firefighting, hydrographic research and police vessels can perform routine or repetitive assignments with a level of focus and situational awareness that transcends human capability. And, of course, autonomous vessels can deal with dangerous circumstances (fires at sea, submerged mines, enemy combatants) without endangering human lives.
Modern commercial interests, however, are understandably more focused than the military on operational costs, primarily because worldwide marine markets are expected to grow exponentially in the coming decades. Accountants on the commercial side see billions of dollars in savings related to the reduction or eradication of crews, crew salaries and crewing requirements; the increased cargo-carrying capacities inherent in smaller or even non-existent accommodation spaces; and the diminution of groundings, shipboard accidents, and personnel and medical problems that automation and autonomy will likely bring.
But regardless of what impetus lies behind all these fairly recent developments, the pressure to create, refine and expand autonomous marine technology in military, mercantile and other sectors is already beginning to push enhancements and expansions toward the realm of recreational boating.
The Self-Driving Trawler
Complete or even near-complete vessel autonomy—if such a thing were to exist—would make little sense for the modern recreational powerboat, at least under most conditions. Just about any boater worth their salt will tell you, when all is said and done, that actually operating a boat amounts to at least half of the fun of owning it. But what if even partial autonomy or perhaps high-level automation was affordable, quick to install and easy to use for the recreational cruiser? What if a super-sophisticated, MFD-based autopilot, say, could be relied upon to handle routine navigational chores on long, coastal trips or even ocean-crossing voyages via a few extra sensors and a software enrichment or two? What if such a device could be relied upon to outperform the capabilities of a human lookout or watch stander and thereby perhaps make a given voyage safer? And what if it could be relied upon to offer reasonable navigational options in the event of an emergency and independently act if necessary?
“Our technology was originally developed for the commercial market and open-ocean applications,” says Phil Bourque, director of business development for Sea Machines Robotics, “but nevertheless, we see a product like our SM300 as a natural fit for owners of long-range recreational cruising boats. With its route manager and collision avoidance technology, the unit is a level up from the typical MFD and is an extra set of eyes and control. Instead of simply displaying radar targets and other sensor information, the SM300 will alert and react if no action is taken. We’ll be announcing, by the way, the first pleasure craft OEM using the Sea Machines technology later this year.”
While exact pricing was not available at press time, Bourque estimated that the cost of retrofitting a SM300 system aboard a passagemaking vessel from a builder like Kadey-Krogen or Nordhavn would be comparatively minor when stacked up against the cost of the boat itself.
“Most retrofit installations including the kit, sensors and installation could be performed for less than $150,000,” he adds. “For a new build, the cost, of course, would be less. And the SM300 autonomous control system can be integrated with most any propulsion control system and marine electronics package.”
While Metal Shark’s CEO Chris Allard is at least as enthusiastic as Bourque about introducing autonomous technology to recreational boaters, he pushes the envelope a good deal further by envisioning a future where recreational vessels not only cruise autonomously but also handle traffic and other issues by actually “talking” to each other. Allard, like Bourque, started his career on the pleasure craft front (Pro-Line Boats and Donzi Marine), albeit on the side that specializes in big-league government and commercial contracts. But in 2006, he decided to partner with a Louisiana boatbuilding family to found a stand-alone company, Metal Shark, again focusing on military and commercial contracts, but sticking with recreational boats as well. Successes with conventional technologies ultimately paved the way for projects that featured vessel autonomy. In 2019, Metal Shark was selected by the U.S. Navy to create a program “covering multiple topics in the autonomous space.” In 2020, the company collaborated with Sea Machines Robotics to produce the two aforementioned Defiant 29s. And most recently, Metal Shark signed a deal to design and build a fully autonomous Long Range Unmanned Surface Vessel (LRUSV) System for the U.S. Marine Corps.
“It’s clear to me that autonomous systems will have a lot of value for the recreational user,” Allard says, “if only from the standpoint of reducing workload and fatigue. With an autonomous system on board, a skipper can let the system make the decisions for him or her and simply supervise.”
Allard suggests that today’s plotter-connected autopilots, with auto-routing, AIS, ARPA and radar overlays, represent a level of automation that can be expanded or extended to offer autonomous functions, so that machine learning can “self-adapt” to deal with all kinds of navigational situations, both expected and unexpected.
“But I think this thing goes a whole lot further,” he continues. “I think it’s inevitable that autonomous and semi-autonomous systems on separate vessels will start communicating—networking, you might say—with each other. In a way we’re already there—with AIS especially, where vessels are in a very real sense talking with each other, each one saying I’m going this way at a certain speed and you’re going that way at a certain speed and there’s either no problem or ALARM! ALARM! ALARM! So, you’ve got to assume, I think, that eventually autonomous vessels will be able to network with each other either via the radio or by satellite and decide all by themselves on the appropriate navigational actions that should be taken.
“Think of your average recreational boat today. You already have most of the sensors on board—radar, AIS, FLIR—and you already have most of the hardware in your chart plotter. To get autonomy to work then, it’s simply a matter of software development, right? And we all know how fast that goes in today’s world. Yeah, I would have no trouble predicting that at some point autonomous, networking-capable technology will be directly integrated into most MFDs on most trawlers and other recreational cruising vessels.”
On the morning of June 18, 2018 when Volvo Penta deployed its “self-docking boat” in a marina in Gothenburg, Sweden, for hundreds of spectators, I remember thinking to myself while standing next to the skipper, who had his arms raised in a hands-off gesture, You gotta be kiddin’ me. The performance of the software-driven, Dynamic Positioning-disciplined, IPS-equipped Azimut 68 underfoot was flat-out amazing, despite the fact that the vessel was constrained to ultimately park itself using one onboard sensor and four more sensors on the four corners of the target slip, an arguably tough-to-mass-produce kind of configuration that did not exactly promise rapid adoption by marina owners around the globe.
Just three years later, Volvo Penta is selling another and, in my opinion, more marketable kind of technology—what it calls “Assisted Docking.” Capitalizing on the potential inherent in synergizing software, Dynamic Positioning, joystick technology and independently operable pod drives, Assisted Docking offers a level of automation—a term best defined as a less sophisticated, less independent form of autonomy—that essentially removes the problems of set and drift, occasioned by current and wind, from the boathandling equation. Wherever the skipper points the joystick, the boat goes directly without responding to external forces. And if the skipper lifts his hand from the joystick, allowing it to return to neutral, the boat simply holds station, again without responding to external forces, while awaiting further directives and parenthetically removing virtually all of the stress associated with backing into a slip in a cross wind or coming alongside a fuel dock with a rousing tide running.
Raymarine’s recently introduced DockSense has similar, stress-busting capabilities apparently, although it seems to be somewhat closer to an autonomous system than Volvo Penta’s automated one. By adding multiple FLIR machine-vision cameras to a boat’s hardware, DockSense uses imagery, a heading-reference system, a computer processing module, an MFD-based app, one of any number of joystick controls on the market and a twin-engine powerplant to integrate and computerize propulsion and steering to help a skipper better negotiate close-quarters maneuvering. According to Raymarine, the new technology creates a “Virtual Bumper” around the boat, thereby obviating the chances of an inadvertent collision with pilings and other objects during dockside maneuvers.
But while both the Volvo Penta and Raymarine systems seem innovative and useful, it’s Japan’s Yanmar that seems to have the only auto-docking system that’s truly autonomous, at least at the present time. More to the point, Yanmar debuted a flybridge cruiser last year that it says can actually enter a marina, find a slip and safely dock itself, either stern-first or alongside, with absolutely no help from a human being.
According to the manager of Yanmar’s Robotics Group, Hisashi Sugiura, the key to this rather staggering feat is the same Simultaneous Localization and Mapping (SLAM) technology that animates self-driving cars. By means of a sophisticated array of onboard sensors, including radar, super-accurate GPS and 3D-LIDAR (an ultra-fast laser scanner), SLAM lets Yanmar’s cruiser move through a given marine space, creating a real-time, continuously updated 3D map of its surroundings while precisely orienting itself within the map. Ultimately, the boat docks itself autonomously in accordance with its own virtual reality, while simultaneously superimposing that reality upon highly accurate electronic cartography.
“Generally speaking, of course, this kind of technology can be applied to either recreational or commercial marine markets,” says Sugiura, “and Yanmar will decide in upcoming years what our customers want. But the possibility to make things safer, easier and more peaceful for commercial users as well as recreational users is obvious.”
Understandably, neither automated nor autonomous docking options, of the sort purveyed by Volvo Penta, Raymarine and Yanmar, seem to be as popular with military and commercial users these days as they are with recreational boaters who, as a group, tend to be comparatively less practiced at boathandling. Autonomous vessels launched by the military and the commercial sectors within the last few years have typically been docked by human beings ashore using remote-controlled dashboards. But this state of affairs is changing. Both Bourque at Sea Machines and Allard at Metal Shark say their respective companies are working on auto-docking capabilities to complement their autonomous open-water technologies. So, it looks like a future that hosts recreational as well as commercial vessels, going about their business with only a whiff of human interaction, traveling from dock to dock with whole oceans in between, is perhaps not beyond the realm of possibility.
Psychology, Hackers and Ancient Traditions
Of course, as with all developing technologies, iffy questions obtrude—some big, some small—and stand in the way of progress. And one of the biggies, at least for many of the designers and engineers in the vessel autonomy field, is not really knowing how recreational and commercial seafarers will respond to unmanned vessels sharing their waterways, or how seafarers in general will respond to autonomous vessels that they must learn to amicably “operate” or “partner with.” Are human beings going to trust vessel autonomy? And how much autonomy are they going to be comfortable with?
Jim Leishman is an experienced recreational boater and, as an owner and vice president of Pacific Asian Enterprises, the purveyor of a long-standing line of Nordhavn’s passagemakers, he certainly qualifies as a deeply committed boatbuilder as well. And what’s more, he certainly sees promise in the technologies being developed by companies like Sea Machines Robotics and Metal Shark. During a recent phone call, he even went so far as to wonder if trying out an autonomous or semi-autonomous system on one of his recently introduced Nordhavn 41s might be an interesting exercise.
But Leishman is also leery. Although he’s a fan of the machine intelligence that inspires such products as Tesla’s Autopilot driver-assistance system and Garmin’s Autoland, a newly marketed marvel that will land a small plane all by itself in an emergency, he’s just a tad doubtful about turning control of a boat—his or anyone else’s—over to an artificially intelligent helmsman, at least entirely.
“Sure, I think this stuff could be useful in the future,” he says. “But then I also gotta wonder. I mean, I’d really like to know, when the wind’s blowin’ about 25 knots and the seas are all kinda frothing up, how this autonomous stuff is gonna see a submerged log up ahead. But then again—who knows—maybe it can.”
The question of hacking is another biggie that haunts the progress of autonomous technology. Proponents contend that sophisticated cyber-security measures and programs will safely deal with all the dark possibilities. Yet, it was only a few years ago that engineering students at the University of Texas hacked into a superyacht’s GPS in the Mediterranean Sea (with the owner’s permission) and altered her course subtly but significantly, just to show that such a thing could be done. And more recently, a geo-political thriller, co-authored by combat veteran Elliot Ackerman and highly respected retired naval officer Admiral James G. Stavridis, imagined a situation in which a sophisticated American F-35 stealth fighter is hacked, hijacked and remotely flown to Iran where the pilot is taken prisoner. Ravings of over-active imaginations? Stavridis, especially, is a cyber-security expert. Prior to his retirement, he headed up the U.S. Southern Command before becoming the supreme allied commander of NATO from 2009 to 2013. Today, among other things, he is the chief international security analyst for NBC. And the thriller he co-authored with Ackerman—2034: A Novel of the Next World War—was recently touted as a “cautionary tale” by Wired magazine, which devoted its entire February 2021 issue to an excerpt.
Another obstacle to further development of vessel autonomy is the question of how to make it exactly and efficiently conform to maritime tradition as embodied in the Nautical Rules of the Road, according to Henrik Tunfors, a Swedish maritime expert who heads up the high-profile safety committee of the International Maritime Organization (IMO), the entity responsible for publishing and administering the rules. For the past few years, Tunfors’ safety committee has been examining the rules (formally known as COLREGs or International Regulations for Preventing Collisions at Sea) as they exist now, albeit in the light of vessel autonomy’s rapid advance. Other IMO committees are looking into legal, insurance-related and liability ramifications.
Current consensus at the IMO, says Tunfors, is that as manned and unmanned vessels begin navigating the same bodies of water over the coming decades, the rules, which are essentially based on age-old traditions and customary law, may have to be “tweaked” to promote safe, trustworthy navigation. But, he adds, because the IMO must somehow accommodate autonomous seafaring without significantly disturbing the essence of the long-standing, traditionally-based COLREGs, performing the tweaks is likely to take “at least 10 years more and quite likely more after that.”
“When I examine the wording of Rule 5, for example, where it talks about maintaining a ‘proper lookout,’ I see language about vision and hearing, and these are human characteristics,” he explains. “So the way the rule reads today, a human being is envisioned as the lookout. But is there a way we can accept a camera or some other technical means to accommodate Rule 5? How do we change the regulation? And really, Rule 8, which stresses the observance of ‘good seamanship,’ is almost more difficult to address for us. After all, what is good seamanship vis-à-vis an autonomous vessel? The question is very difficult and time-consuming to answer.”
A Necessary Clarification
Although the phrase “vessel autonomy” is tossed around quite cavalierly in maritime circles today, in large part due to the mind-blowing capabilities of everything from Siri to self-driving cars, another phrase—“human in the loop”—is part of the picture too, although it’s often given short shrift in the news. The meaning of the human-in-the-loop phrase is virtually self-evident. While there may be no one on board a given autonomous vessel for lengthy periods of time, a human being is always present somewhere within the vessel’s purview, either participating in or capable of participating in operations, whether via a cellular network, satellite or some other link. The autonomous technologies from Metal Shark and Sea Machines Robotics both seem to fall into this category, as do the aforementioned technologies touted by the U.S. Navy.
Higher levels of vessel autonomy do exist, though. In fact, at the present time, berthed not far from the Mayflower Steps Memorial in Great Britain’s Plymouth Harbor (the spot from which the original Mayflower departed for America in 1620), there’s a 50-foot, carbon-fiber, solar-powered trimaran—the Mayflower Autonomous Ship or MAS—described by her developers at IBM (the folks who sprung the supercomputer WATSON on the world in 2010) and ProMare (a Connecticut-based non-profit dedicated to marine research and exploration), as a “fully autonomous AI-powered marine research vessel.” At press time, MAS was scheduled to depart her berth on April 19th for an unmanned, fully insured (by GARD AS of Norway) Atlantic crossing to Plymouth, Massachusetts, first to prove the viability of unmanned vessels tirelessly, intelligently and safely working at sea, collecting oceanographic data, and second to very publicly retrace the route of the pioneering Mayflower some 400 years ago, thereby “showing the way forward for the next 400 years.”
Certainly, MAS is capable of human-in-the-loop interactivity, but according to IBM, she is also capable of wholly autonomous operation whenever her connectivity fails, as is likely when she is well out to sea and/or during extremely bad weather. Trained using nautical images and situations harvested from Plymouth Sound and elsewhere, an onboard “AI Captain,” IBM says, will use sophisticated edge-computing systems and a plethora of sensors to assess traffic and other circumstances as they arise, make decisions based on sensor input and act on those decisions, with total independence from shoreside control, if necessary.
Without doubt, such capabilities will garner publicity and seriously advance the cause of vessel autonomy, if MAS indeed makes the crossing successfully. But there’s a good chance as well that at least some commentators will use the trip to envision a future in which machine intelligence outperforms human intelligence in dark and destructive ways. However, according to the actual creators of MAS, the truth of the vessel’s autonomous voyage is not even close to such dire conceptualizations. In spite of its sophistication, IBM’s AI Captain, they say, is wholly incapable of replicating any of the ineffable aspects of human nature, whether it be intuition, inspiration or a tendency to harbor malign intent.
“The way to look at AI versus human intelligence today,” says Brett Phaneuf, co-director of the MAS project, “is it’s like the difference between using calculus and actually inventing it. When people start talking about systems that actually think like human beings? I’m sorry—they’re just wrong. These systems don’t think. It doesn’t happen. What we’re really dealing with here, in terms of vessel autonomy, is simply advanced mathematics and some very sophisticated programs, models and micro-electronics, all working collaboratively. That’s it.
“If you come up to me today and say you’ve got something that is totally, completely autonomous, I immediately want to ask you some questions: Does it dynamically replan its own route due to external stimuli? Is it goal-oriented? Can it deal with novel scenarios and apply what it’s been trained to do in a way that is unique to the situation? Can it learn new behaviors or actions to resolve novel situations?
“Really, I just don’t get it,” Phaneuf concludes, with obvious passion in his voice. “For years now we’ve been fed this crazy stuff in movies, books and TV—intelligent robots taking over the world! Frankly, that kind of sophisticated AI, the kind of very strong or general AI that would make such a thing possible—it’s been 100 years away for the past 100 years. And it’s still not part of our reality, right? And from where I sit in Plymouth, England, today, I’ve got to tell you, my friend, I don’t think there’s much point in thinking that it ever will be.”