Kevin Kerins
Mark Philip
Douglas Smith
Shil Srivastava
Mark Stevens
Oceaneering
Over the past 30 years, the remotely operated vehicle (ROV) industry has seen many iterative advances in technology, but few that might be considered revolutionary or even a substantial step change. Within the past decades, there has been an ever-increasing need for greater horsepower, deeper operating depths, greater reliability for ROVs, and the ability to accomplish more intervention tasks with sophisticated subsea tooling.
In the early 1980s, one such major improvement was the use of control data and video over fiber optic in the offshore oil and gas ROV industry. This allowed the development of ROVs that could operate in greater depths when compared to ROVs that previously used data over copper. Since then, there has been little breakout technology.
Now, the industry is poised to make a significant step change, thanks to advances in automation control, machine vision technology, and new algorithms in the artificial intelligence space.
Control programs
If the ROV industry attempted to create a fully autonomous ROV control program that anticipated the challenges of environmental conditions and possible failure modes and responses, we would be attempting to build a program approaching the intelligence of the HAL 9000 computer character in Stanley Kubrick’s 1968 film2001: A Space Odyssey. Such a program would be enormous.
An ROV’s tasks can be viewed as sequences of smaller simpler steps that can be easily automated. Unexpected environmental conditions or a crippling failure would cause the execution of the step to halt, break the sequence, and return control to the human operator. Most of the desired actions of the ROV can be constructed from these smaller steps. By utilizing the human in the decision-making control loop, the program is broken down into steps issued as commands by the pilot. These steps can be commanded individually or as sequences with linking logic that tests for successful completion of a step before proceeding to the next one.
With these incremental steps, the complexity of the “autonomous” software is dramatically reduced, and the software becomes much easier to build and maintain. The use of small autonomous steps also allows the operator greater flexibility in constructing sequences for new types of tasks, or in altering tasks that must be slightly modified for changing environments or for different ROVs and tools.
Ultimately, the human operator is the decision-maker. Commands such as “stab the tool in the receptacle” and “grab on to the hand hold on the panel” are automated steps that enable hands-free piloting. This capability will greatly limit lost time due to damage that is often currently incurred on ROV tools, manipulators, and subsea interfaces. Instead of an ROV pilot’s hands on the control joystick and manipulator/tool controller, commands will be executed via a computer mouse or voice command and executed by the more-efficient control computer.
Vision control
The most challenging ROV piloting tasks (which include manipulator tasks such as hot stab insertion, tool placement, or close visual inspection) require that an ROV be maneuvered to a precision of 5-10 mm. Completing delicate tasks with hands-free piloting demands an automated ROV navigation and control system that reliably delivers a high level of precision while operating in a rugged offshore environment.
The recent explosion of autonomous robotic vehicles - including self-driving cars, unmanned aerial vehicles, autonomous underwater vehicles (AUV) and domestic robots - has driven intense interest and groundbreaking research in vision-based robotic navigation that has enabled new ROV automation.
The latest development is a hybrid approach to precise hands-free ROV navigation in the immediate vicinity of structures or panels. Machine vision allows the ROV to recognize objects in order to control navigation. ROV control is difficult using computer vision alone, which is why motion sensors are also incorporated.
With this hybrid navigation system, precision is achieved by leveraging these two innovative and cost-effective new technologies:
Vision technology. Simultaneous location and mapping (SLAM) technology reverses the traditional stereo vision model. Instead of modeling the viewed scene relative to a pair of stereo cameras, SLAM tracks the 3D position of a moving camera relative to a coordinate system fixed to the viewed subsea object (panel or structure).
ROV motion dynamics. These are measured using a suite of motion sensors, including a microelectromechanical sensor (MEMS) attitude and heading reference system (AHRS), a fiber optic gyro, a bottom-tracking doppler velocity log (DVL), and pitch and roll sensors. Knowledge of ROV motion dynamics is what allows the automated control system to “look into the future.” By fully understanding ROV motion dynamics, control needs are anticipated and the brakes can be applied as the ROV approaches a target position or heading.
Manipulator control
The development of an advanced kinematic control software for ROV manipulators simplifies the operation of spatially correspondent manipulators, thereby improving operational efficiency.
ROV manipulators typically have six degrees of freedom: three Cartesian (x, y, z) and three angular (pitch, roll, yaw). A modern spatially correspondent (master/slave) manipulator allows the operator to simultaneously control all six degrees of freedom in real time, but, for many tasks, Cartesian and angular motions are somewhat mutually exclusive, and it is not always desirable to control both at the same time.
It was recognized that too much time is wasted at the jobsite during the process of stabbing tools into receptacles. More specifically, existing manipulator control methods rely too heavily on the skill of the operator. So, a decision was made to automate as much of the process as possible by reducing dependence on operator skill.
The result is a proprietary kinematics software system that allows the operator to select which of the six degrees of freedom he or she wishes to control. This essentially allows the operator to control Cartesian motion while the software automatically controls orientation, or vice versa.
Intelligent diagnostics
For years, the basic approach to fault detection has been the use of sensors that are threshold tested for low, high, or band range limits. The advent of digital control systems allowed these thresholds to be adjusted in situ, and faster telemetry allowed real-time monitoring and ranking of abnormal operating conditions. However, these sensors have added a burdensome cost in their material, manufacture and especially maintenance, since correct calibration is essential if the sensors are the only diagnostic layer that is protecting the ROV from damaging itself or other equipment. Critically, the sensor itself is a component that can fail, such as when large subsea systems are removed from service and recovered to the surface for repair, at significant expense, only to find that the single fault was in the fault-detecting sensor itself.
As more subsystems are being added to ROVs (and as the overall complexity of the command-control communications system is increasing to accommodate more sophisticated acoustic and optical sensors, as well as modern customer payloads), the method of using one sensor to detect one fault quickly scales into the realm of unsupportable, unintelligible, and uninformative. Multiple simultaneous faults cannot easily be distinguished from cascading or avalanching failures unless the operator has wide and deep system knowledge, along with the time and ability to resolve the origin(s) of the failures.
To address this problem, the Intelligent Diagnostic System (IDS) was created. This system combines remote sensors in the target control system, a real-time telemetry path, basic equations of electrical/hydraulic power flow, and a virtual model of the complete power distribution system to produce a program that can determine whether the system is healthy or not, and whether it is operating normally or abnormally.
If the operation is out of nominal ranges, then the fault may be isolated to the enclosure; the subsystem, such as a particular printed circuit board (PCB); or, often, the offending component, such as a specific power supply or electronic component on that PCB. Because the software retains a memory of the sensor values, the surface-based IDS program, located either on a vessel or onshore, can determine the most likely cause of a telemetry failure by looking at trends in sensor values that would indicate that a failure was imminent just prior to the telemetry break. The program can isolate multiple simultaneous failures, and can identify the origin of a single failure that causes multiple simultaneous failures, i.e., a failure cascade or an avalanche.
Avatar control
Oceaneering recently demonstrated the ability to pilot an ROV on an offshore vessel via a satellite link from an onshore base.
The Remote Piloting and Automated Control Technology (RPACT) system utilizes the latest control system programming (Computer-Aided Piloting and Video Machine Vision Technology) and the latest network acceleration technologies (satellite and radio-frequency wireless) to provide real-time access to operate an ROV remotely, either from on shore or from a vessel nearby.
The recent rollout of low-earth-orbit satellite access and the availability of LTE cellular networks offshore ensure a high-throughput, low-latency link to the operator over a secure network. The utilization of a custom network design provides for the prioritization of the best network path available to transmit control signals to the ROV from the remote ROV pilot.
Remote Connection Technology allows the use of multiple virtual connection technologies and potentially creates cost-saving opportunities, such as the reduction and/or elimination of onsite second-shift crewing for low-intensity operations. RPACT allows the utilization of offsite or remote experts who specialize in advanced tooling and operational techniques that may not be present during normal operations, thus enabling the rapid deployment of experts’ knowledge to the onshore ROV operating stations to ensure quick access to operations during incidents and other periods of operational need. The use of time-synced data to correlate ROV movements back to the ROV pilot ensures that the user experience is similar to operating the ROV directly from the ship that it is deployed from. The control inputs are updated by using various techniques, including fly-by-wire controls that enable the ROV pilot to select a point on the screen to update where the manipulator arm needs to go, as opposed to steering it there manually. This fly-by-wire capability allows the ROV operator to select enhancements such as station-keeping and auto-depth hold.
Looking ahead
The industry is beginning to see the convergence of ROV and AUV technologies. Currently, AUVs navigate via predetermined and preprogrammed sets of coordinates. The new technology currently being developed for ROVs will also soon enable AUVs to determine their own courses, based on changes in environment and by means of obstacle-avoidance technology and vision technology. In the near future, AUVs will be able to visually follow pipelines and recognize objects.
Soon, AUVs will also be equipped with decision-making capabilities and the same automated functions that are being developed for ROVs, allowing AUVs to carry out tasks more in line with current ROV capabilities, such as tooling tasks and manipulator functions.
As ROV and AUV technologies continue to evolve and converge, we will see resident ROVs that are semi-permanently installed subsea for long durations independent of a vessel or platform, along with real-time AUV control for intervention tasks.
By leveraging these advanced technologies, an entirely new standard is being set for ROV/AUV performance and capabilities. Extended ROV/AUV capabilities will enable more complex remote operations to be performed. Technical experts and engineers will be able to collaborate further for real-time troubleshooting and inspection tasks. Additionally, automating difficult tasks will increase productivity, along with the safety of these operations.
And, at the fast rate that these ROV/AUV technologies are converging, these advancements will soon be considered as more than just a step change for the subsea industry - they will be clearly recognized as integral components of the revolutionary way forward.
