How Oceaneering Runs Wowza Video Intelligence Framework on Live Subsea Video
Who is Oceaneering?
Oceaneering operates one of the largest fleets of remotely operated vehicles in the world. Those ROVs inspect pipelines, anodes, and subsea infrastructure thousands of feet below the surface, and a single dive can produce up to eight hours of video. Wowza Streaming Engine has carried that video for over a decade, beginning with warehouse and yard monitoring and expanding into the client-facing ROV feeds Oceaneering delivers today.
Will Stucky, Oceaneering Product Manager, recently joined Wowza Chief Architect Viki Bonzo and me for a live session covering how Oceaneering layered Wowza’s Video Intelligence Framework (VIF)® onto that existing pipeline to flag conditions during live dives and tag moments of interest in recorded footage. Watch the full webinar here.
How Does Oceaneering Use Wowza’s Video Intelligence Framework?
Oceaneering samples frames live from remotely-operated vehicles (ROVs) and deck streams that already run through Wowza Streaming Engine. Then, Oceaneering runs inference on GPUs it already owns. VIF returns the results as timed metadata attached to the video, detecting object classes the team trained using custom datasets. Oceaneering is then able to ask questions in plain, natural language about what is happening in the video. That could be determining whether there is any activity on deck, reading telemetry directly off of a camera overlay, or monitoring equipment condition using a fine-tuned vision-language model (VLM).
Why Does Oceaneering Use VIF For Subsea Inspections?
Every time the Oceaneering team conducts an ROV inspection, they generate continuous footage across the entire dive (which can last for hours at a time). While that dive runs, someone has to watch and write down timestamps by hand for anything that is worth revisiting for further investigation. The recording then has to ship back to shore from the ROV, where someone else builds the dive log by matching the notes against the video recording. Finding one relevant moment in a video feed means scrubbing forward through hours of footage to reach it. The log arrives well after the operation ends.
Camera coverage scales while human review time doesn’t. This is the same ceiling large traffic camera networks, law enforcement monitoring agencies, and sports broadcasters often run into. On the webinar, Will described how VIF helps ease the burden of that sequence:
“It does save the time of scrubbing through an 8-hour video to find one specific instance.”
Will Stucky, Oceaneering
How Does Oceaneering Stream Live ROV Video?
Oceaneering’s team runs Wowza Streaming Engine in a self-hosted cloud deployment for both ingest and delivery. The team uses WebRTC for low-latency viewing, as well as SRT for reliable contribution over constrained networks. Roughly 250 deployed ROVs feed video through that pipeline into operational systems.
Reviewing an inspection used to mean flying a specialist offshore and housing them for the duration of the job. Oceaneering can now take video that would otherwise sit on a vessel and deliver it to authorized viewers anywhere in the world. Even with the AI analysis layer and cloud deployment, Oceaneering maintains data sovereignty and security.
How Does Oceaneering Use VIF With Wowza Streaming Engine?
VIF runs alongside Wowza Streaming Engine. Oceaneering uses VIF to sample frames out of band, with configurable frame sampling intervals to control how much video reaches the analysis layer. Oceaneering uses the ID3 metadata from VIF to attach detections to the video itself, which is what makes a moment findable later without a separate index. All this happens isolated from streaming, so their live feeds are never compromised if a GPU gets saturated.
Analyze some streams locally and run others in the cloud. Oceaneering’s mixed environment does just that, routing a specific detection type to a small local machine. VIF is flexible and can be run on-premises against existing NVIDIA hardware, on cloud GPUs, or across both at once.
Does Oceaneering Use Custom Models In VIF?
Oceaneering’s AI and machine learning group had trained a detector for seismic nodes. These large devices sit on the ocean floor recording activity on the seabed. That model came out of earlier project work and had a labeled dataset behind it. Now, they just had to decide where inference should run, and how the right video streams got fed to it.
VIF supports an RF-DETR architecture, so all Oceaneering had to do was convert the existing training data to RF-DETR and point it at live footage inside their existing Wowza instance. Nothing left their environment, no new hardware was needed, and no workflows needed to be rebuilt.
“It was quite simple and very, very pain-free to be able to do that.”
Will Stucky, Oceaneering
How Does Oceaneering Use Vision-Language Models?
Not every question maps to a fixed class list, but that’s where vision-language models (VLMs) really shine.
Oceaneering wanted to know the status of above-deck operations, including whether a crane was moving, whether it was lifting a load, whether cargo was transferring to another vessel, or whether crew members had their hard hats on. Each of these would have required a separate model, and building them all would have taken months.
By pairing a text model with a vision model, a VLM lets Oceaneering ask questions about what’s happening in a video using plain language. They established the conditions they wanted to watch for on deck, with no training run required, and then got the answers they needed without spending unnecessary development cycles.
But Oceaneering took it a step further. They prompted a VLM to read the telemetry values burned into the ROV camera overlay using optical character recognition (OCR). Then, VIF would emit the telemetry data as ID3 timed metadata alongside the video. Oceaneering was able to build this workflow in about a day on deliberately under-resourced hardware.
“It seems like the sky is the limit. We can now, instead of needing to spend months training on something that’s very niche, this is propping us up to be able to get started much quicker.”
Will Stucky, Oceaneering
How Does The Oceaneering Workflow Apply To You?
Oceaneering’s workflow may seem unique, but it can be broadly applied to a number of use cases. Any operation with more camera coverage than review capacity can sample frames from streams it already runs, apply a model suited to its domain, and attach the results to the video to reduce time to action. That shape holds for industrial and remote operations, security and surveillance, media and entertainment, and traffic management alike. Teams start from a model that they already own, with infrastructure they already run.
Watch the Oceaneering Webinar On-Demand
You can watch the full webinar with Oceaneering today for live demonstrations of object detection, deck monitoring, telemetry extraction, and anode assessment, plus a walkthrough of the VLM configuration screens and more. Reach out with any questions.
Frequently Asked Questions
How does Oceaneering use AI on live ROV video?
Oceaneering samples frames from live ROV and deck streams already running through Wowza Streaming Engine, runs inference using its own GPUs, and returns results as ID3 timed metadata. The team runs a custom-trained seismic node detector model, a vision-language model for querying above-deck operations and telemetry, and a fine-tuned model to assess anode depletion.
How quickly can a team put an existing trained model into production?
Oceaneering moved a trained seismic node detector model into live production in roughly a day. They did so by converting the existing labeled dataset to RF-DETR and running it against live streams inside their own environment.
Can a vision-language model read telemetry text off a video overlay?
Yes, Oceaneering prompted a vision-language model in VIF to read telemetry values burned into an ROV camera overlay using optical character recognition (OCR). The output of those values was then sent as ID3 timed metadata alongside the video. It attached telemetry to the stream during the dive rather than reconciling it against a recording afterward.
How much does it cost to fine-tune a vision-language model?
Fine-tuning NVIDIA Cosmos 4B to assess anode depletion cost Oceaneering roughly $1.50 in compute and took under an hour on an A100 using 15 to 20 clips. Low-Rank Adaptation (LoRA) is a fine-tuning method that keeps that cost low by freezing the original model weights and retraining only a small fraction of them, typically between 0.5% and 1% of the model size.
How does timed metadata change how a dive log gets built?
A dive log records what happened during an inspection and when. The conventional process depends on someone noting timestamps by hand during the dive, then matching those timestamps against the recording after it has been shipped back from the ROV. Timed detection metadata, written into the stream, marks those moments as they occur so reviewers can open the log and jump directly to each event instead of waiting for files and cross-referencing timestamps manually.
Can VIF trigger a recording when it detects something of interest?
Yes, VIF can trigger a custom Java listener that receives detections and, upon a detection of interest, pulls decoded frames from the buffer before passing them into a recording flow.
Can a service provider add AI analysis without mixing customer video together?
Yes. Oceaneering keeps each client’s video isolated. This requirement applies internally as well as externally, and VIF preserves that separation. VIF runs inside the customer’s own environment and sends no frames to an external service.
