Wowza Video Intelligence Framework

Wowza’s Video Intelligence Framework (VIF) is a flexible intelligence layer for Wowza Streaming Engine that extracts and operationalizes data from video feeds. VIF takes frames from live video streams, routes them to AI models for inference, and converts the results into structured outputs that your existing operational systems can act on. It works with the cameras and infrastructure you already have, with no cloud dependency.

VIF works with any video source that can push a stream to Wowza Streaming Engine, including IP cameras, PTZ cameras, drones, body cameras, dash cameras, mobile devices, IoT sensors, and broadcast encoders. Supported ingest protocols include RTMP, RTSP, SRT, and WebRTC. Because VIF operates at the streaming layer, it’s camera-agnostic, so there’s no need to replace existing hardware or standardize on a specific manufacturer.

Yes. VIF is fully deployable on-premises with zero cloud dependency. All inference runs locally on your hardware, and no video data needs to leave your network. This includes support for air-gapped environments common in defense, corrections, critical infrastructure, and government deployments. Docker images can be pre-loaded, and all five output channels remain operational without internet connectivity.

VIF ships with support for RF-DETR, a transformer-based object detection model that recognizes 80 common object classes (people, vehicles, equipment, etc.) with inference latency starting at 10 milliseconds. It also includes an experimental scene analysis capability using a CLIP-based model for natural-language scene matching. Beyond the built-in models, VIF also supports custom-trained RF-DETR models. You can train a model on your own dataset and deploy it alongside the default model without changing your pipeline configuration.

VIF connects to external platforms via standard webhooks and in-band metadata without requiring any proprietary connectors. Confirmed integrations include Datadog, Splunk, Elastic, PagerDuty, Opsgenie, and ServiceNow. VIF can also deliver alerts to traffic management systems, VMS platforms, SIEM tools, and any system that accepts HTTPS webhook payloads. The JSON event payload includes timestamp, object class, confidence score, bounding box coordinates, and tracking ID.

Yes. Wowza provides a guided training workflow using Google Colab notebooks. You supply a labeled dataset (your own images or publicly available datasets via Roboflow), configure training parameters, and the notebook produces a custom model weights file you can deploy directly into VIF. A typical training cycle takes roughly half a day of active effort, with approximately five to six hours of GPU training time. You don’t need a massive dataset to start. A small, clean, well-labeled set can produce strong initial results using transfer learning from the pre-trained base model. It’s important to note that model performance is entirely dependent on the quality and quantity of your training data. Wowza provides the tooling and infrastructure for training and inference, but the accuracy of custom model outputs is the customer’s responsibility.

Detection results are routed simultaneously across five independent output channels: timed metadata embedded in the video stream (ID3 tags), visual bounding-box overlays burned into a companion output stream, structured JSON log files written locally, webhook alerts dispatched to external systems via HTTP POST, and a custom Java listener API for building tailored integrations inside Wowza Streaming Engine. This means a single detection can simultaneously trigger an operator alert in your incident management platform, annotate the live video feed, and log the event for compliance without any additional configuration per channel.

VIF provides per-stream tuning controls for managing false positive rates. You can adjust confidence thresholds, set minimum consecutive frames before an object is tracked, configure tracking persistence, and control how detection events are batched or aggregated before delivery. These settings are accessible through the built-in management dashboard or via REST API and can be changed at runtime without restarting the streaming engine. This allows operators to calibrate sensitivity per camera based on real-world conditions. A high-traffic freeway camera and a fixed perimeter camera can run at different thresholds simultaneously.

Wowza provides the inference infrastructure, the training tooling, and a reference detection model, but the accuracy of any AI model is determined by the data it was trained on, the conditions it operates in, and how it has been tuned for a given environment. For custom-trained models, the customer owns the training data, controls the training process, and is responsible for validating that the model performs at an acceptable level before putting it into production. Wowza is not responsible for the accuracy or outputs of customer-trained models. We recommend running the built-in validation and inference test steps in the training notebook against real-world footage from your environment before deploying any custom model into a live workflow. Use VIF’s per-stream confidence thresholds and false positive controls to calibrate detection sensitivity for your operational requirements.

VIF processes frames in memory during inference and does not retain video frames by default. The system produces structured event metadata (timestamp, camera ID, detection type, confidence score, and bounding box coordinates) which can be logged and audited independently. This architecture supports environments with strict data retention requirements where frame storage creates legal or compliance exposure.

Yes, VIF requires an NVIDIA GPU with CUDA 12.8+ and Turing architecture (SM 7.5) or newer. The minimum validated configuration is an NVIDIA T4 (16 GB), with an L4 (24 GB) recommended for production workloads. A baseline deployment supports approximately eight concurrent 720p streams at up to 15 frames per second. Additional hardware specs include 8 vCPU minimum (16 recommended), 32 GB RAM, and 32 GB of free storage (NVMe recommended).

VIF supports four deployment topologies: an all-in-one Docker container (ideal for evaluation and lower-volume production), a split-server configuration with a dedicated GPU host (for independent scaling of inference), multi-instance deployments (where multiple streaming engines share inference resources or vice versa), and fully air-gapped on-premises deployments. You can start with the all-in-one Docker approach and upgrade to split-server without changing your stream configuration.

No. VIF is non-blocking by design and operates asynchronously. If the inference service becomes unavailable, your live streams continue uninterrupted. Detection coverage pauses until the inference service is restored, but video ingest, processing, and delivery are never impacted.

VIF plugs directly into the Wowza Streaming Engine processing pipeline. Your existing ingest and delivery configurations remain untouched. Stream configuration for VIF, including model selection, confidence thresholds, output channels, and sampling rates, is managed through a dedicated panel in the Streaming Engine Manager or via REST API.

The same protocol and codec flexibility of Wowza Streaming Engine also applies to VIF. So, any camera or device, across any deployment, can be modernized with VIF. There are suggested minimum hardware requirements for running VIF locally, on-prem. You can view detailed technical specifications on our documentation page.