Your AI Detected Something. Now What?

A camera sees a person near a restricted gate. The model detects them, a confidence score appears, and an alert fires. The demo worked.

But who received the alert? Did they have enough context to evaluate it? Was it the first notification or the fifteenth? Was the person an authorized contractor? What happens if the notification never arrives?

Those questions are where the real deployment work lives, and they usually get answered after the demo, if they get answered at all.

New call-to-action

I’ve sat in a lot of Wowza Video Intelligence Framework (VIF)® conversations with security teams this year. The question that decides whether a deal moves forward is almost never about model accuracy. It’s some version of “who on my team is going to look at this, and when?”

Video AI helps people see what they would otherwise miss. But detecting something and doing something useful about it are two different engineering problems. If you only solve detection, you hand already busy operators another screen to watch and another queue to clear, which solves one problem by creating another.

A detection only matters if someone can act on it in time.

Why should you start with the response for a video AI project?

Consider an industrial facility that wants to identify people entering a restricted area after hours. Remote monitoring and industrial operations often have complex, nuanced detection needs.

“Detect a person” sounds like a clear requirement until you look at what it covers. A person on a public sidewalk doesn’t matter. A maintenance technician arriving during an approved work window is expected. A person lingering inside a restricted perimeter needs immediate review.

The model’s job is to supply evidence. The application’s job is to combine that evidence with the rules and context that apply. The organization’s job is to decide what response is appropriate. Before choosing a model, I ask five questions:

  1. What specific condition requires attention?
  2. Who is responsible for evaluating it?
  3. What information do they need?
  4. How quickly must they respond?
  5. What happens if they can’t?

The answers usually change the technical requirements. The team may need a better camera angle, a defined region of interest, schedule data, a short clip, or an integration with the incident management system already in place. A larger model is rarely the missing piece.

Starting with the response keeps the project focused on the problem the organization is trying to solve.

Why do operators need to be part of the video AI capacity plan?

We size compute, storage, and bandwidth as a matter of routine. Human attention deserves the same discipline, and it rarely receives it.

Take a modest example: 100 cameras, each producing two unnecessary alerts a day. At 30 seconds per review, that’s 100 minutes of operator time every day.

Scale that to a real security operation with 1,000 cameras and the same rate costs about 16.7 hours a day, or roughly two full-time operators doing nothing but clearing alerts that never mattered. 30 seconds is also optimistic, because it ignores the cost of pulling someone away from whatever they were doing when the alert landed. At that volume, unnecessary alerts are a staffing cost and should be budgeted like one.

Those interruptions also compete with real incidents, existing responsibilities, and every other system asking for attention, which is how alert fatigue sets in. Model accuracy alone can’t tell you whether a deployment is working. You also need to know how often the system interrupts someone and whether the interruption was justified.

Repeated detections of the same event shouldn’t become repeated incidents, and a brief, uncertain observation shouldn’t get the same treatment as a sustained condition. Thresholds will also need to differ by location and operating hours.

The opposite risk matters just as much. Raise thresholds until the complaints stop, and you’ll also suppress the events the organization needed to catch. What you want is a defensible balance between missed events (false negatives), unnecessary interruptions (false positives), and the consequences of each, tested in the environment where the system will actually run.

What should an AI video alert give the operator?

A useful notification establishes what happened, where, when, and why it reached this person. Depending on the use case, that means a camera location, timestamp, event duration, the relevant image or clip, and a direct path to the live feed. The operator shouldn’t have to open three applications and scrub through footage to decide whether a notification deserves attention.

Model chaining, which shipped in VIF 1.2, can add to that context. A detector flags a person, and a vision-language model (VLM) chained behind it describes what that person is doing, so the operator starts with more than a label and a score. That gets them to a decision faster, but the decision is still theirs.

Model uncertainty should stay visible. A confidence score is an estimate, and a generated description still needs to be verified by a person.

This is where integration earns its value. Sending a detection to a webhook is a technical handoff. It isn’t proof that the right person received it, understood it, or acted on it.

The receiving workflow needs its own engineering: duplicate handling, delivery monitoring, ownership, acknowledgment, and escalation. If nobody owns the next step, the alert is just another record.

How much authority should a video AI system have?

Every vendor says their system keeps a human in the loop. The question I ask is which decisions that human is actually making. If the answer is all of them, the system hasn’t saved anyone any time. If the answer is none of them, nobody is accountable when it gets one wrong.

The right answer depends on what’s at stake. Opening a review item is different from locking a door, stopping equipment, or accusing someone of wrongdoing. Actions at the high end need deliberately designed safeguards, permissions, and validation, and they should never follow simply because a model returned a high score.

A useful system also makes its limits explicit and tells operators when it can’t do its job. A camera disconnect, a stalled inference service, or a failed notification path must never look like an uneventful afternoon.

Why is Time to Action the metric that matters?

Inference speed matters, but it’s only one part of the clock. Time passes while video is captured, transported, sampled, analyzed, routed, and reviewed, and a fast model can’t make up for a notification sitting in an unmonitored queue.

For a production deployment, measure five things:

  • Detection quality: Relevant events found and missed, using representative footage and agreed validation methods such as precision and recall.
  • Operator burden: Unnecessary alerts, duplicate incidents, and review time.
  • Time to Action (or TTA): The time from the event to a usable notification.
  • Workflow reliability: Delivery failures, unavailable cameras, and periods when analysis wasn’t running.
  • Operational outcome: Time to resolution and effort per incident, against the current baseline.

Compare each of these against the existing process. A technically impressive deployment that doesn’t improve that process hasn’t demonstrated its value yet.

So start with one operational use case. Define the response, establish the baseline, and agree on what success and failure look like before anyone expands the scope. Then grow it based on evidence.

Where do Wowza Streaming Engine and VIF fit?

This is how Wowza approaches Wowza Streaming Engine and the Wowza Video Intelligence Framework (VIF)®.

VIF runs on Wowza Streaming Engine, against the cameras and GPUs an organization already operates. It can run on-premises and air-gapped. Nobody has to move their video into someone else’s platform to analyze it, and the security boundary stays where the customer drew it.

One of our first VIF customers, Oceaneering, analyzes subsea ROV footage on the GPUs it already owns, in its own cloud environment and on NVIDIA Jetson devices at the edge. Its internal ML team trained its own detection models, and the deployment runs at 3 to 5 frames per second because that’s what the footage requires. Nothing about the workflow had to move to fit the product.

Several of the problems above are built into how VIF works. Every detection is a timestamped record tied to its stream, and the VIF tab in Wowza Streaming Engine Manager shows detection overlays on live stream thumbnails, so an operator can see what the model saw. System health and per-stream analysis status sit in the same view as the detections, so a stalled analysis path shows up as a problem instead of as quiet. Detections go out by webhook to the systems the organization already runs, such as a VMS, SIEM, or dispatch platform.

Model chaining lets one model’s result feed the next, so a detection can arrive with a description of what moved and what it’s doing. It works in reverse too: detecting a lack of motion tells a broadcaster a stream has frozen.

What VIF doesn’t do is decide what counts as acceptable risk, who owns an incident, or what action an organization should authorize. No streaming platform or AI model does. Those decisions belong in the design from day one.

Ask them to keep going

The next time someone shows you video AI detecting an object, ask them to keep going.

Show me what reaches the operator and what happens if the result is wrong. Show me how you know the notification arrived, and what happens when the system goes quiet.

Then show me how the operation improved.

Until then, all you’ve seen is a demo.

Frequently Asked Questions

How do you calculate the operator cost of unnecessary alerts?

Multiply the number of cameras by the unnecessary alerts each one produces per day, then by the time it takes to review one. At 1,000 cameras, two unnecessary alerts per camera per day, and 30 seconds per review, that’s about 16.7 hours of operator time every day, or roughly two full-time operators. That figure doesn’t include the cost of pulling someone off whatever they were doing.

What should be defined before choosing a video AI model?

Define the response before choosing a video AI model. That means defining the specific condition that requires attention, who is responsible for evaluating it, what information they need, how quickly they must respond, and what happens if they can’t. Those answers usually change the camera placement, the region of interest, the data the alert needs, and the integrations required.

What is time to action?

Time to Action (TTA) can be thought of as the end-to-end delay from when something is captured in a video frame to a usable alert in an operator dashboard, after which the operator can coordinate a response. That encompasses the full pipeline spanning video ingest, analysis, and alert delivery into the systems that teams use to organize and manage a response. The Wowza Video Intelligence Framework supports a per-frame inference time of as low as 10 milliseconds on Turing-class hardware, and a Time To Action of under half a second.

Does webhook delivery mean someone acted on the alert?

No. Webhook delivery only confirms that a receiving system got the event. Acknowledgment confirms that a person took ownership of it. Track both and measure the time from notification to acknowledgment alongside the time from event to notification.

How should a video AI deployment handle camera or inference failures?

Treat them as events that need an owner. A disconnected camera, a stalled inference service, or a failed notification path should alert someone the same way a detection would. Track the periods when analysis wasn’t running, because a quiet system and a broken system look the same from the operator’s chair.

Is human review still required with model chaining?

Yes, human review is still required. A vision-language model (VLM) chained behind a detector can describe what the detected person or object is doing, which gets the operator to a decision faster. That description is still a model output. Someone has to confirm it before anyone acts on it, and the higher the stakes of the action, the more that confirmation matters.

Wowza Streaming Engine: Flexible, Extensible, & Reliable Streaming

About Jon Corley

Jon Corley is the Chief Technology Officer at Wowza, where he oversees the strategy and execution of the world’s most reliable streaming engine. A pioneer in cloud media processing, Jon previously founded the Elastic Transcoder service at Amazon Web Services (AWS) and architected the massive-scale delivery infrastructure for Comcast’s Peacock. With deep roots in the video ecosystem (including tenures at Brightcove and Limelight), Jon is currently focused on the intersection of video streaming and AI, helping developers move from passive video delivery to intelligent, “agentic” media pipelines and transformations.
View More

FREE TRIAL

Live stream and Video On Demand for the web, apps, and onto any device. Get started in minutes.

START STREAMING!
  • Stream with WebRTC, HLS and MPEG-DASH
  • Fully customizable with REST and Java APIs
  • Integrate and embed into your apps

Search Wowza Resources


Subscribe


Follow Us


Categories

Blog

Back to All Posts