Configure VIF in Wowza Streaming Engine Manager

Wowza Streaming Engine Manager includes a dedicated VIF tab in the top navigation bar. Selecting it opens the VIF interface, which is organized into three tabs:

• Live Streams — real-time system health and per-stream analysis metrics.
• On-Demand — analysis jobs run against stored media files (VOD), with a results viewer.
• Configs — configuration, split into Stream Configs (per-stream rules), Stream Config Defaults (values every stream inherits), and On-Demand Configs (settings for On-Demand jobs).

A Test Player for previewing an analyzed stream opens when selecting a stream's thumbnail on the Live Streams tab (it plays the stream's -vi overlay output).

---

Live Streams

http://<WSE_HOST>:8088/Home.htm#plugin/server/vif/shm.html

The Live Streams tab is the default landing view when you select the VIF tab. It shows real-time system health across the top and a row (or thumbnail card) for each active stream below. Each tab label shows a live count (for example, Live Streams 1, On-Demand 51). An Auto-refresh 1s indicator at the top right shows the refresh cadence.

System-level stats (cards across the top)

Card	Description
WSE Host	Wowza Streaming Engine version (for example, 4.11.3+11) and current CPU utilization of the WSE host (for example, CPU 4%).
WSE GPU	GPU utilization of the WSE host, with GPU model and CUDA version (for example, Tesla T4 · CUDA 12.8) and a Decode % · Encode % · Mem bus % line. The ? tooltip reads: The Wowza Streaming Engine (WSE) host machine's GPU: utilization, video-engine (decode/encode) load, and memory-bus busy share (not VRAM in use). Driver 570.133.20. The tooltip also lists Card, Driver, and CUDA details and a breakdown of GPU Utilization, Mem Bandwidth, Encode, and Decode (Mem Bandwidth = Percent of time the GPU's memory controller was busy — not VRAM in use.).
VIF Module	VIF plugin module version (for example, 1.1.0).
Inference Host	The Video Intelligence Service (VIS) host and its version and CPU (for example, video-intelligence-service.docker / 1.1.0 · CPU 7%).
Inference GPU	GPU utilization of the inference GPU, shown per device (for example, cuda:0). The ? tooltip reads: How busy the GPU running the models is. Low values mean headroom for more streams or higher concurrent executions. Sustained values near 100% mean analysis may fall behind: lower the Inference FPS on busy streams or add GPU capacity.
Inference GPU Memory	VRAM in use on the inference GPU, as a percentage and absolute figure (for example, 86% / 12.9 GiB / 15.0 GiB). The ? tooltip reads: VRAM in use on the inference GPU. Models keep their memory while loaded, so a high value even with idle streams is normal. It limits how many different models fit at once, not how busy they are.

Note: The card layout consolidates the health data that earlier releases spread across separate WSE, CPU, Card, Driver, CUDA, GPU, Mem Bandwidth, Encode, and Decode cards, plus a separate Inference panel. The same values are now grouped into the six cards above, with the GPU detail moved into the WSE GPU card's tooltip.

Active Live Streams

The Active Live Streams header shows the stream count alongside average processing and detection timing, for example: Active Live Streams: 1 · Processing avg 11 ms · Detect avg 12 ms. 

A Thumbnails toggle at the right of the header switches the layout between the detailed table view (default) and a thumbnail grid of live preview cards. In the grid view, each card shows the application/stream name and resolution (for example, live / object.stream (1280x704 @ 30fps)), a live preview with detection overlays and on-frame stats, and the analyzed percentage.

In the table view, each active stream displays as a row with the following columns:

Column	Description
Model Type	Detector-type icon and model name (for example, Model: medium). Scene and VLM streams also show an analysis-window line (Window).
Stream Name	Stream identifier (for example, live/object.stream) with an inline Active toggle for runtime on/off control.
Stream Info	Resolution and source frame rate (for example, 1280x704 @ 30fps), an Inference fps slider for adjusting analysis frequency in real time, and the VIS host, version, and connection status (for example, video-intelligence-service.docker (1.1.0) with connected shown in green).
Ping (RTT)	Round-trip latency to VIS in milliseconds. Tooltip: Round-trip time between Wowza Streaming Engine (WSE) and the Video Intelligence Service (VIS).
Processing Avg	VIS processing time per request. Tooltip: Video Intelligence Service (VIS) processing time per analysis request. (Renamed from Ttl Processing Avg in earlier releases.)
Frame Detect Avg	Detection round-trip time. Tooltip: Time from Wowza Streaming Engine (WSE) sending the frames to receiving the detections back. Includes network time and Video Intelligence Service (VIS) processing.
Frames Analyzed	Share of captured frames that were analyzed, as a percentage. Tooltip: Share of the frames Wowza Streaming Engine (WSE) captured from the stream that were analyzed by the Video Intelligence Service (VIS), over the last 10 seconds.
Thumbnail	Latest captured frame with overlay rendering. Select the thumbnail to open the Test Player for the stream's -vi output.

---

On-Demand tab

http://<WSE_HOST>:8088/Home.htm#plugin/server/vif/vod.html

The On-Demand tab runs VIF analysis against stored media files (VOD) rather than live streams. It opens on the On-Demand Jobs list.

On-Demand Jobs list

The list shows previously submitted jobs. Controls above the list:

• New Analysis — starts a new On-Demand job (see New Analysis below).
• State — filter by job state: All states, Active, Running, Queued, Completed, Failed, or Cancelled.
• Tag — filter by an exact job tag.
• Filter / Clear — apply or reset the filters.
• Refresh and Auto-refresh (5s, 15s, or off) — control list refresh.
• Pagination — Showing 1–25 of N with ‹ Prev and Next ›.

A status line summarizes the job subsystem, for example: Workers 2 · Retention 100 jobs · no TTL · Global webhook none · Auto-resume on.

Each job is a row with the following columns:

Column	Description
Media	Thumbnail and source filename (for example, nvdia-demo.mp4). A NO FRAME or NO FRAME (CLIP PATH) badge appears when no preview frame is available; an optional tag badge appears when the job has a tag.
Analysis	Analysis type and icon (VLM Analysis, Object Detection, or Synthetic Detection) and the config rule applied (for example, live / vlm_describe.*, od / od-object, live / svd.*).
State	Job state (for example, COMPLETED, CANCELLED) with a request count (for example, 402 requests). Cancelled but resumable jobs show, for example, 8 / 38 requests · partial results kept · resumable.
Timing	Completion and duration (for example, finished 3h ago / analyzed in 40m 49s), or cancelled 5h ago.
Actions	Resume (resumable jobs only), JSONL (download stored results), View (open the results viewer), and Delete.

New Analysis

Selecting New Analysis opens a three-step page. A ← On-Demand Jobs breadcrumb returns to the list.

1. Source — choose the media to analyze. Search the file list, then select a file, or select Upload to add one. Files can also be drag-and-dropped: Drag & drop video files here to upload them — they land in the Engine's content directory and appear in this list. Each listed file shows its size, modified date, and a Delete control.
2. Analysis — choose a Config from the dropdown (each entry is a stream rule plus its detector type, for example, live / vlm_describe.* · vlm). Select Edit to tune settings for this job only; the job then runs against an inline config (shown as Config: inline on the resulting job) rather than a saved stream rule. A preview card summarizes the selected config's detector, model, output fields, and which listeners apply—for example, a VLM config shows a vision-language model reviews sampled frames against your prompt and notes that stream listeners are skipped for file jobs; detections still land in the stored results.
3. Outputs & Options — set output and lifecycle options: Store results (JSONL) — Every analysis response kept raw and media-time-stamped; pageable, downloadable, readable while running. Lifecycle webhook — Inherit global, Custom, or Off (No global webhook is configured — inherit means none). Auto-resume — Inherit global, Custom, or Off (Transient failures retry on a backoff, up to 3 attempts; the counter resets on progress. 'Custom' forces it on for this job). Tag — Free-form label exact-match filter on the jobs list.

Select Start analysis to submit the job, or View request JSON to inspect the request payload first.

Job results viewer (View)

Selecting View on a job opens its results viewer. The viewer adapts to the job's analysis type, but the following elements are common to all types:

• Job metadata — Source, Status, Job ID (with copy control), Config (a stream-rule name or inline for a job-specific config), and Detection type with the model used (for example, VLM Analysis · Qwen3-VL-4B-Instruct-FP8, Object Detection · large, Synthetic Detection · SVD). An optional Tag is shown. A note reports any stream listeners skipped for file jobs.
• Actions — Download JSONL and Delete.
• Summary cards — the first card is always Duration; the remaining cards vary by analysis type (see below).
• Score Strip — a binned activity timeline across the media. Selecting a cell jumps the player and browses that cell's rows. The legend/coloring varies by type.
• A results pane — What the model saw for VLM jobs, or Windows for Object and Synthetic jobs (see below).
• Player — an embedded player for the analyzed media.
• Detection Details — an expandable section showing the raw rows behind the summary (also reachable by selecting a Score Strip cell).

The type-specific elements are:

Analysis type	Summary cards (after Duration)	Score Strip	Results pane
Object Detection	Detections (raw, across stored windows), Windows (one analysis request each), Classes Seen (of N configured), Requests (answered / expected)	color = leading class · darker is higher confidence · gray = empty, with a class-count legend (for example, scissors ×21, stop sign ×16)	Windows — one row per window: detection count, timestamp, the detected class(es) with ×N counts, and a window ID.
Synthetic Detection	Verdict (for example, No synthetic content detected with real … · synthetic … split), Peak Score (with mean), Requests (answered / expected)	darker red = more synthetic · gray = not flagged	Windows — one verdict per window: score, timestamp, real/synthetic label relative to the threshold, and a window ID. A note may explain when window length was adjusted to source keyframes.
VLM — Describe	Windows Answered, Mode (Describe, with model), Requests	darker is higher activity · gray = empty	What the model saw — a free-text description per window.
VLM — Custom	Windows Answered, Mode (Custom, with model), Requests	darker is higher activity · gray = empty	What the model saw — the custom output schema per window (for example, verdict: real / ai_generated plus custom fields).

Configs tab

The Configs tab holds all VIF configuration, split into three sub-tabs: Stream Configs, Stream Config Defaults, and On-Demand Configs.

Note: In earlier releases these were reached from Default Config and Stream Configs buttons on the dashboard. Those buttons no longer exist; configuration now lives entirely under the Configs tab.

Stream Configs

http://<WSE_HOST>:8088/Home.htm#plugin/server/vif/stream-config.html

The Stream Configs sub-tab manages per-stream overrides. Each stream rule corresponds to a per-stream configuration file (<applicationName>_<streamName>.json) in conf.modules/vif/. It lets you view and edit VIF settings per stream rule without editing config files or using the REST API. Defaults that are not overridden here come from Stream Config Defaults.

Use the Incoming Stream dropdown to choose a rule to configure, or + New Stream Config… to create one. Help text: Choose which incoming streams VIF should analyze. Application must match exactly. Stream Name can be an exact name or a regex pattern — e.g. application live and stream name traffic.* matches rtmp://…/live/traffic_cam1. Fields you leave unset inherit from the Stream Config Defaults. Buttons at the top right:

• Save all changes — save the current rule.
• Clone Config — create a new rule based on the current one. (New in this release.)
• Delete Config — remove the current rule.

The panel is laid out in two columns: Video Processing and the detector-specific section (plus Advanced Options) on the left, and Event Listeners on the right.

Note:  The UI displays confidence, overlap, and other threshold values as percentages (for example, 30, 50, 20). The corresponding values in the VIF configuration files under conf.modules/vif/ (Default.json or a per-stream file) are stored as decimals (0.3, 0.5, 0.2), and the UI converts automatically on save.

Video processing section

Which fields appear depends on the selected Detector Type. Fields common to all types are marked below; type-specific behavior is noted.

Setting	Description
Active	Turns VIF processing on or off for streams matching this rule.
Detector Type	Object — Tracks objects like person/car as bounding boxes across frames. Scene — Classifies the whole frame against your target scene phrases. VLM — Sends frames to a Vision Language Model endpoint for free-text analysis. Synthetic — Checks each video window for AI-generated content (a Real-vs-Fake verdict).
Grayscaled	Convert frames to grayscale before analysis to reduce bandwidth and processing overhead.
Use Transcoder	When enabled, frames are pulled from the transcoder output instead of the incoming stream.
Inference FPS	Frames per second sent for analysis. Lower values reduce load but may miss short events. Set to -1 to match the stream frame rate. (Not shown for Synthetic.) For VLM: one request is sent per window carrying Duration × Inference FPS images; if that exceeds the endpoint's image cap, the service evenly subsamples the window down to the limit.
Inference Video Height	Controls the frame size sent for analysis. DEFAULT uses the built-in sizing behavior. SOURCE sends frames at the original stream height. MODEL uses the model's native input size. CUSTOM lets you specify an exact pixel height.
Duration (seconds)	Length of each analysis window. (Scene, VLM, and Synthetic only.) VLM: one request per window carrying Duration × Inference-FPS images… (a computed Images per request hint is shown). Scene: Length of each analysis window (seconds) sent to the Video Intelligence Service (VIS). Synthetic: Target length of each analysis window (seconds). Windows snap to the next keyframe — set Duration ≥ your source's GOP.
Auto Frame Throttle	Automatically lowers Inference FPS when inference can't keep up. (Not shown for Synthetic.) For Object: This is object detection's latency control — frames stay near-live and contiguous so tracking is preserved. For Scene/VLM: throttling first means fewer coverage gaps before Catch Up To Live has to skip ahead.
Catch Up To Live	When inference falls persistently behind real time, automatically skip a batch of old frames to catch up with the live edge. (Scene and VLM only.)
Catch-Up Max Behind (seconds)	How far behind live detections may fall before catch-up skips ahead. Leave blank for auto (~2s). (Shown with Catch Up To Live.)

Object analysis section

Visible when Detector Type is Object. Configure object detection and optional ByteTrack tracking.

Setting	Description
Model Name	Choose a built-in model size — Nano, Small, Medium, Large — or Custom to load custom object detection models / model weights from the checkpoint path.
Checkpoint Path	Visible when Model Name is Custom. A dropdown listing available custom models / model weights (for example, models/fire_smoke.pth). Required when using a custom object model.
Class Names	A multi-select list of the model's available classes. Select one or more to allowlist. Leave nothing selected to pass all classes.
Confidence Threshold %	Minimum object score to keep before listener filtering or tracking logic runs. Default 30 (stored as 0.3).
Tracking Method	ByteTrack links detections across video frames so the same moving object keeps a consistent ID over time. Use it when you need consistent IDs for regions of interest, counts, direction, or time-in-area logic. Use None for raw detections only.

ByteTrack properties

Appear when Tracking Method is ByteTrack. These settings control when tracks start, how long they persist, and how strict matching is between frames.

Setting	Description
Ignore Untracked Objects	When enabled, objects that are not being tracked are excluded from event listener results.
Min Track Confidence %	Minimum score needed to create a new track. Default 50 (stored as 0.5).
Max Lost Track Frames	How many missed frames a track survives before removal. Default 30.
Min Consecutive Overlap %	How much two detection boxes must overlap to be treated as the same tracked object. Default 20 (stored as 0.2).
Min Consecutive Frames	How many consistent frames are needed before a new track becomes visible. Default 3.

Scene analysis section

Visible when Detector Type is Scene. Configure scene classification for clips rather than object boxes.

Setting	Description
Class Names	One row per target scene phrase or label to classify against (for example, person walking). Use + Add class to add rows and the × control to remove them.
Sensitivity	Higher values make scene triggers more sensitive and likely to fire.
Confidence Threshold %	Minimum scene score to keep before listener filtering.

VLM analysis section

Visible when Detector Type is VLM. Configure Vision Language Model (VLM) analysis using an external AI endpoint.

Setting	Description
Model Name	Model served by the VLM endpoint. Choose a supported model — Qwen3-VL-4B (Qwen), Nemotron Nano 12B VL (NVIDIA), Gemma 3 4B (Google), Cosmos3 Edge (NVIDIA), Cosmos3 Nano (NVIDIA) — or Other… to enter a custom name. Must match a model the endpoint actually serves.
VLM Server	Address of the VLM server (for example, vlm.docker:8000). If omitted, http:// and the /v1 path are added on save; an explicit scheme or path is kept as typed. Leave empty to use the default. Use Verify to test the address. (Replaces the earlier Endpoint URL field name.)
API Key	API key for authenticating with the VLM endpoint. Use Show to reveal the stored value.
Request Timeout (seconds)	Timeout in seconds for each VLM API request.
Analysis Mode	Detect — list what to look for, get a per-item verdict. Describe — a written description of each window, no fixed class list or prompt configuration. Custom — your own prompts and, optionally, your own output schema.

Advanced (VLM)

Setting	Description
Temperature	Controls randomness in model output. Lower values are more deterministic.
Max Tokens	Maximum number of tokens in the VLM response.

Synthetic (AI Generated) detection section

Visible when Detector Type is Synthetic. Relay each source H.264 window to the NVIDIA Synthetic Video Detector NIM (over gRPC) for a per-window Real-vs-Fake verdict. Opt-in / bring-your-own endpoint; H.264-only; no re-encode.

Setting	Description
Endpoint	SVD NIM as a gRPC host:port (not an HTTP URL). Local sidecar: svd.docker:8001; NVCF hosted dev: grpc.nvcf.nvidia.com:443.
Transport Security (TLS)	Force TLS (recommended), Auto-detect, or Force plaintext. Force TLS for any endpoint reached over the network (pair it with the certificate paths under TLS Certificates; TLS uses the system trust store otherwise). Auto-detect picks TLS when any certificate path is set or the port is :443 (for example, NVCF), and plaintext otherwise — plaintext is only appropriate for an in-network sidecar.
API Key	Bearer token for NVIDIA's hosted endpoint — use an invocation-scoped key (an NGC pull key is not authorized to invoke). Use Show to reveal the stored value. A banner notes that NVIDIA's hosted endpoint is for evaluation only, not production.
Function ID	NVCF function id (sent as function-id metadata). Required by NVIDIA's hosted endpoint.
Classification Threshold	Score above which a window is classified synthetic (0.0–1.0). NIM reference default 0.3 (conservative); 0.5 balanced.

Advanced options

Per-stream connection overrides and debug settings. Leave any field blank to inherit the defaults from Stream Config Defaults.

Service Connection

Setting	Description
VIS URL	Override the Video Intelligence Service (VIS) WebSocket URL for this stream only. Supports ${ENV_VAR} substitution.
VIS API Key	Override the X-API-KEY header sent to VIS for this stream only. Use Show to reveal the stored value.

Debug options

Setting	Description
Log Max Messages	Maximum VIF/WebSocket messages written to the WSE access log. Use 0 to disable.
Log Timing	Delay in seconds between writing logging details/timing metrics to /tmp/vif/<stream_name>. 0 is disabled.

Event listeners section

Event listeners are triggered by detections and control how detection data is delivered. Pick one from the Configured Listeners list to edit it, or Create New Listener to add another one. Listener type stays fixed after creation.

The panel includes:

• Rollup/Batch Time — time in seconds used by listeners whose method is rollup or batch. Corresponds to rollup_batch_interval in the VIF configuration files under conf.modules/vif/ (Default.json or a per-stream file).
• Configured Listeners — a list of the listeners on this rule, each shown as Name (Type): Method (for example, Id3Event (Id3Event): Rollup, Overlays (OverlayEvent): Immediate, Webhook (WebhookEvent2): Disabled). Select a listener to open its edit form.
• Create New Listener — add a listener, choosing from the built-in types: Id3Event, LogFileEvent, ObjectTracking, OverlayEvent, and WebhookEvent2. The type is fixed after creation.

Each listener edit form has these common settings:

Setting	Description
Listener type	Read-only, showing the event class and version (for example, OverlayEvent v1.1.0). Fixed once created — remove and recreate the listener to change the type.
Method	Delivery mode. The available options depend on the listener type (the form states, for example, Available methods for this listener type: Immediate). Modes include Immediate, Batch, Rollup, and Disabled.
Confidence threshold %	Optional listener-side filter. Results below this score are not passed to this listener, even if the detector kept them.
Suppress Empty Detections	When enabled, frames with no detections are not passed to this listener. (New in this release.)

Below the common settings, each listener shows a contextual description and may expose custom properties.

Overlays (OverlayEvent) — Draws boxes, labels, and optional stats back onto the Wowza video output. Immediate is best when downstream systems need each detection as it happens.

Property	Description
Fade Step	Number of frames over which overlays fade out.
Jitter	Don't draw if the pixel change is less than this amount. Reduces flickering in static scenes.
Show Stats	Overlay system stats on the video.
Replace Video	Replace the source rendition with the overlay rendering. Only applies to Object detection.
Debug String	Title overlay using a date-format pattern (for example, 'Cam-01:' yyyy-MM-dd HH:mm:ss). Also supports the variables {{app_name}}, {{stream_name}}, and {{detector_type}}.

Stream Config Defaults

http://<WSE_HOST>:8088/Home.htm#plugin/server/vif/default.html

Sets the values that apply to all streams unless a per-stream rule overrides them. These correspond to the top-level keys of video-intelligence.json. Environment variable placeholders (for example, ${VIS_API_KEY}) are resolved at runtime.

Service Connection

Setting	Description
VIS URL	WebSocket endpoint for the Video Intelligence Service (VIS) (for example, ${VIS_PROTOCOL}://${VIS_HOST}:${VIS_PORT}/ws/stream/). Supports ${ENV_VAR} substitution.
VIS API Key	Sent as the X-API-KEY header. Use Show to reveal the stored value.

Video Processing Defaults

Setting	Description
Inference FPS	Default frames per second sent for analysis. Set to -1 to match each stream's frame rate. Per-stream configs can override this value. Default -1.
Use Transcoder	When enabled, frames are pulled from the transcoder output instead of the incoming stream.
Frame Grab Interval	Seconds between frame grabs when not using the transcoder. Set this to the GOP of the incoming stream. Default 1.
Duration (seconds)	Scene/VLM only. Length of the clip sent to VIS per analysis request. Default 2.
Concurrent Executions — Object Detection	Maximum simultaneous object detection inferences. Increase this if GPU usage stays below 50% at full load. Default 10.
Concurrent Executions — Scene Detection	Maximum simultaneous scene detection inferences. Default 1.

Select Save all changes to apply.

On-Demand Configs

http://<WSE_HOST>:8088/Home.htm#plugin/server/vif/vod.html

Settings for the On-Demand job subsystem. A ← On-Demand Jobs link returns to the jobs list.

VOD Settings (the vod-settings document)

Setting	Description
Concurrent jobs	Worker pool size; applies immediately. Blank keeps the current value.
Jobs kept	Retention cap — the oldest finished jobs are evicted past the cap. Blank keeps the current value.
Job TTL (seconds)	Forget finished jobs this long after they end. Empty or 0 = off.
Max upload size (bytes)	A larger upload is refused with 413. Blank keeps the current value (for example, 10737418240 = 10 GiB).
Global lifecycle webhook	Default status-webhook destination for every job.
Webhook Authorization secret	A webhook secret (defined under Webhook Secrets) sent only to the global destination above — per-job URLs name their own at submit.
Auto-resume transiently failed jobs	Retry transient failures up to 3 attempts; the counter resets on progress.

Select Save vod settings to apply.

Restart-only

These directory settings apply at the next Engine start (moving them under running jobs would strand those jobs). Edit them by patching the settings document or its file.

Setting	Description
content_dir	Source media directory (default: Engine content/).
jobs_dir	Job storage directory (default: Engine vif-vod-jobs/).

Webhook Secrets (the secrets document)

Named Authorization values a job references at submit (lifecycle_webhook_secret) to authenticate its own webhook destination — the job record keeps the name only. File webhook credentials here and nothing else. Use Add secret and Save secrets to manage entries.