Captain's Eye: Keeping the Overview

Artificial intelligence is predicted to reshape much of the world we live in, transforming everything from product design to complex management systems. Its effects are already being felt in the maritime world, particularly in commercial shipping, and systems are being refined for superyachts.

The possibilities for AI integration are huge. By analysing vast amounts of data, algorithms will be able to suggest design modifications to improve performance and reduce fuel consumption. Systems are being developed that are not only able to detect and avoid obstacles, but can rapidly optimise routes to changing weather conditions. These will all reduce the risks of human error.

We already see the seeds of what is to come in SEA.AI, a system for yacht and motorboat crews to help detect objects in the open ocean surface that could present a collision risk. This system has been in use as an aid to watchkeeping on solo round the world race yachts for several years. Vessel maintenance, safety systems and security are where the greatest efforts are being made, and AI algorithms tailored for large commercial ships are beginning to filter down to privately run vessels. A market leader in this area is Captain’s Eye.

The company was set up just over two years ago by Uri Ben Dor, a former head of planning and organisation in the Israeli Navy and Doron Oizerovich, a former head of R&D at the Israeli Ministry of Defence, specialising in UAV, computer vision, AI and radar. Having taken early retirement from military careers in their late forties, the pair were perfectly placed to develop a tech business that could be applied to commercial vessels. Their Captain’s Eye system has already been installed on over 100 commercial vessels.

Captain’s Eye connects to cameras installed aboard ships and yachts. Every frame captured by these cameras is analysed to recognise changing circumstances and identify objects. If certain parameters are met, an alert is immediately sent to an owner or captain. Alerts and notifications appear on the Captain’s Eye computer system on board and also remotely via an iOS or mobile app. Remote connectivity is with a SIM or on board wifi. Should there be a break of connection, Captain’s Eye will keep recording and upload video once reconnected.

Its main objective is to avert serious safety issues. “Fires are one of the biggest problems” explains Zev Edgar, sales and marketing officer at Captain’s Eye. “The current detection systems are not 100% bulletproof. With lithium batteries there have been many overheating incidents, and by the time it is recognised it is way too late. This system can be set to detect smoke and a send notification about the exact location.

“It will also recognise oil leaks, or any custom parameter, for example, doors that should be closed, or where something has changed, such as rags left in an engine room,” he says. The AI system can be set up to reinforce watch keeping routines, sending a notification if no one makes a round each hour, for example. It can also recognise if people are not wearing the correct protective gear, and it can send alerts if someone who is not on a permitted access list is detected on board.

“Face recognition from a camera based on entrance to the yacht is one option,” explains Edgar. “The owner can list people with permanent or temporary access and the system compares with this. Other systems can give a lot of false alarms so people get so annoyed with it and they switch them off,” he adds. “This system is very smart and it can separate [circumstances] and understand them. It will differentiate between a person and a dog, and between different people, so you don’t get these false alarms.”

Captain’s Eye connects to a network of four or more cameras placed around a vessel using an existing CCTV system (any normal definition IP camera is suitable). These can be added to if there are additional locations an owner or captain wants to oversee. A typical installation would have six cameras, one at the main entrance, two directed at the engines and hydraulics, another rotating camera outside at the highest point of boat and one or two cameras on the deck and entrance to the main living saloon.

Installation of the system takes one or two days, followed by a process of calibration and measurement. Captain’s Eye has already been installed and is being routinely used on over 100 large ships and container vessels, and a ‘mini’ version is being marketed for superyachts. The cost of the hardware, software installation and cloud space starts at €10,000 for yachts of 30-50m and rises to €30,000 for superyachts over 100m, not including the cameras, plus a yearly subscription for the service.

Nir Geva is CEO of ASM Maritime BV, an Amsterdam-based ship management company that operates chemical tankers, bulk carriers, car carriers and container vessels. He is already seeing the real world benefits of AI systems such as Captain’s Eye, which has been installed on the company’s fleet.

“We noticed a gap between the office requirements and the ship performance – often there is a gap of knowledge between different ships – and Captain’s Eye provide a solution to supervise these and see incidents quickly,” he says. “It can provide the crew with the first alert if something is going wrong and, on top of that, the office is alerted to something going wrong or going to happen, such as lying down on the floor that nobody noticed. We can then ask the captain to check.”Geva works in a world where the cost of human error slip-ups can be astronomical. He believes Captain’s Eye will help to reduce, perhaps eliminate, these. “The consequences of a leakage during a cargo operation can run to millions of dollars,” he says. “We had one ship detained in the US because of a crew error. They were handling the ballast water treatment system outside of the system and the ship was detained with a $1 million fine, despite the crew saying they didn’t do it.

“So we asked Captain’s Eye to customise two specific cameras to record these areas and alert us if the systems [in place] were not being correctly used.”

For the next phase of customisation, the system will be customised to ensure no one who is unauthorised embarks the ships. “Every compartment will be inspected every 3, 5 or 7 hours, and the system would trigger a specific alert so that we in the office can be sure these areas are visited each time,” he explains.

Another use would be to ensure that the Bridge Navigational Watch & Alarm System is never switched off. “It will also check that nobody is falling asleep on the bridge, disappearing from the bridge or is there who shouldn’t be.”

There are many other potential tasks that Captain’s Eye can be customised to alert for, he adds. “We could request a portable camera when we need office support to connect live or to put a camera in a location to protect an area if we want to check crew are using personal protective equipment,” he says.

Other similar systems for the yacht market are emerging. DZ Technologies was founded by Dennie Zaad, also based in Amsterdam. Zaad has a background in superyacht IT solutions and hotel CCTV, and is working with AI analytics partners to hone a rival system that uses Axis IP cameras to “detect if there is a fire, do man overboard detection and fall detection and object detection,” he says. Zaad is currently installing the system on a yacht as part of a refit being carried out in the Netherlands.

These AI systems have a valuable place on board now and in the future, agrees Holger Flindt, head of the claims department at Pantaenius. “Back in the Eighties, there were engine rooms without any watch keepers, with alarm systems instead, but this comes at a much higher level. The technology is good for identifying problems on board and possibilities to manage that,” he says.

“The question is always how you use this technology. It is good to have technical assistance on board but it should never replace crew who should be permanently on board.”

Flindt worries about the possibility crews could become reliant on smart tech doing their jobs. When superyachts come into port, crew are often all too keen to move ashore. “I think there is a question when an alert pops up on a mobile phone how long it will take to get back to the yacht and deal with it – and who does that,” he cautions.

“I think you need training for crew to handle problems and I prefer to have educated crew on board. Vessels are getting bigger and tech levels are increasing and a lot of people are not qualified to deal with that. But, yes, it will be helpful and it will give crew more freedom and allow them to be more flexible.”

There is no doubt in Flindt’s mind that we are on the verge of these systems being widely, even universally adopted. “Will it be [the norm] in future? Yes. Even if you think you have everything under control at any moment it will also give you documentation. You could combine it with a tech logbook, making it easier to identify problems on board, even in harbour.

"There are a lot of advantages in the opportunity to have more data about changes of crew, changes of engineer, watch keeping. It will bring everyone ahead, management companies and owners, and may also drive down engine or technical breakdowns. So from that perspective there is a safety factor to it.”

AI systems will be able to incorporate many other peripheral functions in the near future. We will see systems devoted to the realm of onboard experience providing personalised services to owners, guests and crew. AI-driven systems using natural language processing will be able to adjust lighting, temperature, and entertainment preferences. They will be used to guarantee privacy and security for owners, yet at the same time give them a window into the world on board their yacht from a mobile device anywhere in the world.