Wowza Video Intelligence Framework®
Wowza Video Intelligence Framework Demos & Walkthroughs
See how to use VIF to detect objects, understand scenes, apply vision-language models, and trigger custom workflows on the video you already stream.
Wowza’s Video Intelligence Framework (VIF) is the intelligence layer inside your video pipeline, and emits detections through five simultaneous output channels that feed into your VMS, dispatch console, control center, and observability stacks. Act on events in seconds using the cameras and systems you already operate.
- 01Enriched HLSIn-band ID3 timed metadata for ad insertion, telemetry, and chain of custody.
- 02Burned-in overlaysAnnotations drawn on live video, with bounding boxes and confidence scores.
- 03JSONL logsStructured and formatted logging for tools like Splunk and Elasticsearch.
- 04WebhooksReal-time alerts in dispatch systems, PagerDuty, ServiceNow, and Datadog.
- 05Custom Java ListenersYour business logic inside Wowza Streaming Engine.
Reduce time to action with Wowza Streaming Engine and VIF, while staying in control of your infrastructure, workflows, and costs.
Detect Objects with VIF
RF-DETR, trained on 80 classes out of the box
VIF ships with RF-DETR pre-trained on 80 common COCO classes, so you can detect people, vehicles, and everyday objects on a live stream with no training required. When you need classes the default model does not cover, use a guided Google Colab notebook to train and import a custom model.
- Detect 80 COCO classes out of the box with the bundled RF-DETR model
- Train custom RF-DETR models on your own data in a Google Colab notebook and export to ONNX
- Run custom and default models on the same stream at the same time
- Persist object identities across frames with ByteTrack-compatible tracking
- Get object class, bounding box, confidence score, and tracking ID for every detection
Understand Scenes with VIF
Match scenes against natural-language descriptions
VIF includes a CLIP-based model that scores live scenes against natural-language descriptions instead of a fixed class list. VIF matches each frame against a plain language description for open-ended situations a fixed 80-class detector cannot express.
- Describe detections in plain language instead of maintaining a fixed class list
- Supports open prompts and cases the default 80-class detector doesn’t express
Use VLMs with VIF
Add a vision-language reasoning layer on top of detection
Vision-language models add a reasoning layer on top of object detection. VIF routes selected frames to a VLM that returns descriptions of a scene rather than a single fixed label. VLM inference runs much slower than object detection, so the effective pattern uses object detection as the trigger and the VLM as the second-stage analyst you call once something fires.
- Route frames to a vision-language model for classification using open prompts
- Generate descriptions of what is happening in a frame
- Trigger VLM analysis from an object detection event instead of running it on every frame
- Run heavier models out-of-band on their own GPU so the stream never waits on inference
- Score frames for AI-generated content with the NVIDIA Synthetic Video Detector, running as an NVIDIA NIM inside VIF
Trigger Custom Workflows with VIF
Turn detections into actions in the systems you already run
VIF turns detections into actions by emitting every event through five output channels, so results land in the tools teams use today. The same detection can annotate the video, notify an external system, write to a log, and trigger custom code at the same time.
Deliver in-band ID3 timed metadata in enriched HLS, and burn overlays directly onto the video.
Write JSONL logs to disk for Splunk or Elastic, and POST webhooks to PagerDuty, Datadog, ServiceNow, or a dispatch console.
Run custom business logic natively in Wowza Streaming Engine, for rules like line-crossing, zone monitoring, or clip generation on a trigger.
Orchestrate the engine from AI agents and IDEs through the Wowza Streaming Engine REST API and the Model Context Protocol (MCP) server.
Deploy VIF alongside Wowza Streaming Engine without re-architecting your pipeline
Add detection to the cameras and feeds you already run, while keeping ingest and delivery configurations as is.
Stand up a pilot on your existing infrastructure then expand once the numbers hold. Run inference on hardware you own instead of paying per-minute cloud AI rates, and tune frame sampling to control GPU spend per stream. Deploy on-prem, at the edge, in air-gapped networks, or in the cloud, with no external calls required during inference. VIF supports compliance-sensitive architectures by keeping video and inference inside environments you approve, subject to your own legal and security validation, and you keep your model weights and your data.
Walkthroughs
Set it up, then make it yours
How to set up and use Wowza Video Intelligence Framework
Get up and running with Wowza Video Intelligence Framework, connect your live stream, and push data into tools like Datadog.
Watch VideoHow to train custom models for Wowza Video Intelligence Framework
Train, integrate, and validate custom AI models in Wowza Video Intelligence Framework through a guided workflow.
Watch VideoSee it on your streams
Wowza Streaming Engine captures your feeds. Video Intelligence Framework extracts the data.
VIF sits in the streaming pipeline underneath the applications your teams already run, and delivers detections into your VMS, dispatch, SIEM, and observability platforms through webhooks, in-band ID3 timed metadata, burned-in overlays, JSONL logs, and Java Listeners. Get a closer look today.