• How to configure closed captioning for live streaming

    Wowza Media Server includes support for closed captioning for live streams. It can ingest caption data and convert it to the appropriate caption format for streaming using the Apple HTTP Live Streaming (Apple HLS), Adobe HTTP Dynamic Streaming (Adobe HDS), and RTMP protocols. Many player technologies, including Apple iOS devices, VideoLAN VLC player, and many set-top boxes, can display CEA-608 closed captioning data embedded in live streams. Other player technologies, such as JW Player and Flowplayer, can only display AMF onTextData captioning data.

    Note: Wowza Media Server™ 3.6.0 or later is required.

    Contents



    Introduction

    Ingesting CEA-608 captions

    Ingesting AMF onTextData captions

    Ingesting AMF onCaption captions

    Playback

    Legacy closed captioning functionality

    More resources

    Introduction



    There are many types of closed caption sources and each streaming standard may use a different format for embedding captions on the output side. The closed captioning feature in Wowza Media Server for live streaming supports the following input sources and output formats.

    Supported input sources

    • CEA-608 captions embedded in live streams.
    • Captions embedded as Action Message Format (AMF) onTextData in input streams.
    • Captions embedded as Action Message Format (AMF) onCaption and onCaptionInfo in input streams (Wowza Streaming Engine™ 4.0 and later).
    • An API is provided to insert onTextData into a stream.


    Supported outputs

    The following outputs can be translated from any of the supported input sources:

    • onTextData events in Adobe HDS (San Jose) and RTMP live streams.
    • CEA-608 formatted SEI data within the video track in Apple HLS (Cupertino) live streams.
    • WebVTT text tracks in Apple HLS live streams (since 3.6.x).


    Supported caption translations


    The following table summarizes the input-to-output caption translations that are supported by Wowza Media Server:

    Outputs >

    Inputs ?
    Adobe HDS (San Jose)
    (onTextData)
    RTMP
    (onTextData)
    Apple HLS (Cupertino)
    (CEA-608 in video)
    Apple HLS (Cupertino)
    (WebVTT)
    AMF onTextData Yes Yes Yes Yes
    AMF onCaption Yes Yes Yes Yes
    CEA-608 Yes Yes Yes Yes
    via API Yes Yes Yes Yes

    Future releases of Wowza Media Server may support additional input and output sources/formats.

    Ingesting CEA-608 captions



    This section describes how to configure the ModuleCEA608ToOnTextData module, which monitors live streams for CEA-608 captions, decodes the captions, and converts them to onTextData events.

    Note: For use with Wowza Media Server 3.6.

    Configuring Application.xml


    1. Set up an application for live streaming using one of our live streaming Tutorials. For playback of onTextData closed captions in the live stream, it's best to set up the application for Adobe HDS (San Jose) so that you can easily test the captions. To do this, open [install-dir]/conf/[application]/Application.xml in a text editor and set the Streams/LiveStreamPacketizers property to:
      <LiveStreamPacketizers>cupertinostreamingpacketizer,sanjosestreamingpacketizer</LiveStreamPacketizers>
    2. In the [install-dir]/conf/[application]/Application.xml file, make the following additional modifications:

      1. Add the following module to the end of the <Modules> container:
        <Module>
        	<Name>ModuleCEA608ToOnTextData</Name>
        	<Description>ModuleCEA608ToOnTextData</Description>
        	<Class>com.wowza.wms.timedtext.live.module.ModuleCEA608ToOnTextData</Class>
        </Module>
      2. Add the following properties to the application-level <Properties> container at the bottom of the [install-dir]/conf/[application]/Application.xml file (be sure to get the correct <Properties> container - there are several in the file:
        <Property>
              	<Name>ccIngestCEA608LanguageID</Name>
               	<Value>eng</Value>
               	<Type>String</Type>
        </Property>
        <Property>
               	<Name>ccIngestCEA608TrackNumber</Name>
               	<Value>99</Value>
               	<Type>Integer</Type>
        </Property>
        <Property>
              	<Name>ccIngestCEA608EnableField1</Name>
               	<Value>true</Value>
               	<Type>Boolean</Type>
        </Property>
        <Property>
               	<Name>ccIngestCEA608EnableField2</Name>
               	<Value>false</Value>
               	<Type>Boolean</Type>
        </Property>
        <Property>
                <Name>ccIngestCEA608LogCaptions</Name>
        	<Value>true</Value>
        	<Type>Boolean</Type>
        </Property>
        <Property>
               	<Name>ccIngestCEA608LogAsSCC</Name>
               	<Value>false</Value>
               	<Type>Boolean</Type>
        </Property>
        <Property>
               	<Name>ccIngestCEA608VerboseLogging</Name>
               	<Value>false</Value>
               	<Type>Boolean</Type>
        </Property>
        For more information about the available properties, see Property details.

    3. Start Wowza Media Server and publish a live stream to it from an encoder. For this example, we'll use the stream name myStream.


    Property details


    • ccIngestCEA608LanguageID: The language code to use for the caption. If not specified, defaults to the 3-letter locale of the server.

    • ccIngestCEA608TrackNumber: The track number of the caption text track. If not specified, defaults to 99.

    • ccIngestCEA608EnableField1: Specifies if CEA-608 field 1 is enabled. The default value is true (enabled). Typically, either field 1 or field 2 is enabled, not both.

    • ccIngestCEA608EnableField2: Specifies if CEA-608 field 2 is enabled. The default value is false (disabled).

    • ccIngestCEA608LogCaptions: Specifies if the parsed captions are logged. The default value is true (enabled).

    • ccIngestCEA608LogAsSCC: Enables logging of all CEA-608 commands as they're sent to the encoder in Scenarist Closed Caption (SCC) format. The default value is false (disabled).

    • ccIngestCEA608VerboseLogging: Enables verbose logging of the CEA-608 parser. The default value is false (disabled).


    Practical considerations


    Due to the differing nature of CEA-608 captions versus captions in a streaming server, some caption modes are decoded better than others. Pop-on captions (captions that appear anywhere on the screen as a whole, followed by another caption or no captions) are best. Other caption modes (such as scroll-up captions that draw directly to the screen) may exhibit some delay. This is due to the fact that CEA-608 draws directly to the device, while Wowza Media Server must detect the end of each caption in order to capture it and send it with the outgoing stream.

    When considering how to send captions to Wowza Media Server in live streams:

    • Use pop-on captions where possible.

    • If using other caption modes, if possible, minimize the amount of time between the start of caption drawing and the end of the caption.


    Ingesting AMF onTextData captions



    This section describes how to configure the ModuleOnTextDataToCEA608 module, which monitors live streams for AMF onTextData captions, decodes the captions, and converts them to CEA-608 captions.

    Note: For use with Wowza Media Server™ 3.6.

    Background



    An onTextData event is an Action Message Format (AMF) event that's traditionally associated with the Adobe HDS and RTMP streaming protocols and is used to carry closed caption or subtitle information. An onTextData event usually includes the following fields:

    • text. Carries the UTF-8 encoded text for the event.

    • lang. The three-letter language ID for the event (for example, eng).

    • trackid. (Optional) For video on-demand streaming, this is the ID of the text track in the video file. For live streaming, this field is either omitted or set to 99.

    The ModuleOnTextDataToCEA608 module monitors a live stream for onTextData events, extracts the text field, formats it as CEA-608 closed caption data, and injects it into the video stream as SEI NAL units. Many player technologies, including Apple iOS devices, VideoLAN VLC player, and many set-top boxes can then display the embedded closed captioning data.

    You can use the Wowza Media Server server-side API to inject onTextData events into a live stream using the IMediaStream.sendDirect() API call (there are details below). This enables a live source that passes closed caption data in the form of timed text events or SMPTE events to use the Wowza Media Server server-side API to inject onTextData into a live stream. When using the ModuleOnTextDataToCEA608 module, the onTextData is intercepted and injected into the stream as CEA-608 events.

    Configuring Application.xml


    1. Set up an application for live streaming using one of our live streaming Tutorials. For playback of CEA-608 closed captions in the live stream, it's best to set up the application for Apple HLS (Cupertino) so that you can easily test the captions. To do this, open the [install-dir]/conf/[application]/Application.xml file in a text editor and set the Streams/LiveStreamPacketizers property to:
      <LiveStreamPacketizers>cupertinostreamingpacketizer,sanjosestreamingpacketizer</LiveStreamPacketizers>
    2. In the [install-dir]/conf/[application]/Application.xml file, make the following additional modifications:

      1. Add the following module to the end of the <Modules> container:
        <Module>
        	<Name>ModuleOnTextDataToCEA608</Name>
        	<Description>ModuleOnTextDataToCEA608</Description>
        	<Class>com.wowza.wms.timedtext.live.module.ModuleOnTextDataToCEA608</Class>
        </Module>
      2. Add the following properties to the application-level <Properties> container at the bottom of the [install-dir]/conf/[application]/Application.xml file (be sure to get the correct <Properties> container - there are several in the file:
        <Property>
        	<Name>closedCaptionLiveMaxDisplayTime</Name>
        	<Value>10000</Value>
        	<Type>Integer</Type>
        </Property>
        <Property>
        	<Name>closedCaptionLiveCommandsPerFrame</Name>
        	<Value>30</Value>
        	<Type>Integer</Type>
        </Property>
        <Property>
        	<Name>closedCaptionLiveChannel</Name>
        	<Value>0</Value>
        	<Type>Integer</Type>
        </Property>
        <!-- white:0, green:2, blue:4, cyan:6, red:8, yellow:10, magenta:12 -->
        <Property>
        	<Name>closedCaptionLiveColor</Name>
        	<Value>0</Value>
        	<Type>Integer</Type>
        </Property>
        <Property>
        	<Name>closedCaptionLiveCharacterSet</Name>
        	<Value>UTF-8</Value>
        </Property>
        <Property>
        	<Name>closedCaptionLiveLogOnTextDataEvents</Name>
        	<Value>true</Value>
        	<Type>Boolean</Type>
        </Property>
        <Property>
        	<Name>closedCaptionLiveEnableTranscoderResults</Name>
        	<Value>true</Value>
        	<Type>Boolean</Type>
        </Property>
        For more information about the available properties, see Property details.

    3. Start Wowza Media Server and publish a live stream to it from an encoder. For this example, we'll use the stream name myStream.


    Property details


    • closedCaptionLiveMaxDisplayTime: Maximum time, in milliseconds, that a closed caption is displayed on the screen.

    • closedCaptionLiveCommandsPerFrame: Maximum number of closed caption commands to send per-frame (maximum is 31).

    • closedCaptionLiveChannel: Close captioning channel. Valid values are 0 (CC1) and 1 (CC2). (Note: Only CC1 seems to work with Apple iOS devices.)

    • closedCaptionLiveColor: Color of closed caption text. Valid values are 0 (white), 2 (green), 4 (blue), 6 (cyan), 8 (red), 10 (yellow), and 12 (magenta).

    • closedCaptionLiveCharaterSet. Closed caption character set to use when converting the onTextData text data to caption data.

    • closedCaptionLiveLogOnTextDataEvents: Controls logging of the individual onTextData events. If true, all onTextData events are logged.

    • closedCaptionLiveEnableTranscoderResults: Controls whether CEA-608 captions are passed through Wowza Transcoder. If true, all CEA-608 captions are passed through.


    Publishing onTextData



    You can download a package that includes an example module (ModulePublishOnTextData) that injects onTextData events into a live stream. It's useful for testing a closed captioning implementation based on the ModuleOnTextDataToCEA608 module. This module takes caption data from a flat text file and injects an onTextData event every 6.5 seconds into the live stream. Again, this module is only for testing purposes and is an example of how to inject onTextData events into a live stream.


    Configuring Application.xml


    1. In the [install-dir]/conf/[application]/Application.xml file, make the following additional modifications:

    1. Add the following module to the end of the <Modules> container:
      <Module>
      	<Name>ModulePublishOnTextData</Name>
      	<Description>ModulePublishOnTextData</Description>
      	<Class>com.wowza.wms.plugin.closedcaption.test.ModulePublishOnTextData</Class>
      </Module>
    2. Add the following properties to the application-level <Properties> container at the bottom of the [install-dir]/conf/[application]/Application.xml file (be sure to get the correct <Properties> container - there are several in the file:
      <Property>
      	<Name>publishOnTextDataFile</Name>
      	<Value>${com.wowza.wms.context.VHostConfigHome}/content/ontextdata.txt</Value>
      </Property>
      <Property>
      	<Name>publishOnTextDataPublishInterval</Name>
      	<Value>6500</Value>
      	<Type>Integer</Type>
      </Property>
      <Property>
      	<Name>publishOnTextCharsetTest</Name>
      	<Value>false</Value>
      	<Type>Boolean</Type>
      </Property>
      <Property> 
      	<Name>publishOnTextDataLanguageID</Name> 
      	<Value>eng</Value> 
      	<Type>String</Type> 
      </Property>
      <Property> 
      	<Name>publishOnTextDataTrackNumber</Name> 
      	<Value>99</Value> 
      	<Type>Integer</Type> 
      </Property>
      The available properties are:

      • publishOnTextDataFile: Path to the ontextdata.txt text file.

      • publishOnTextDataPublishInterval: Specifies how often events are injected, in milliseconds.

      • publishOnTextCharsetTest: If set to true, the module will publish test onTextData events that cover the full UTF-8 character set from 0x20-0xFF.

      • publishOnTextDataLanguageID: Specifies which language is used based on the 3-character ISO-639 code ISO-639-2 language code. Default: eng.

      • publishOnTextDataTrackNumber: Specifies which track number is used. Any number can be used but 99 (the default value) is common.

    3. Start Wowza Media Server and publish a live stream to it from an encoder. For this example, we'll use the stream name myStream.

    4. Play the stream. See the Playback section below for more information.



    Ingesting AMF onCaption captions


    This section describes how to configure the server for ingestion of AMF onCaption captions.

    Note: For use with Wowza Streaming Engine™ 4.0.

    Background

    There are two kinds of Action Message Format (AMF) onCaption events that are supported.

    The first type contains:

    • onCaptionInfo event that defines the speakers and track information of captions. This may contain an array of speakers and an array of tracks.
    • onCaption events that contain the actual captions. This includes an array of captions and an optional speaker index, which refers back to the onCaptionInfo's speaker array.


    The second type contains:

    • onCaptionInfo with a type of "708". This caption type contains Base-64 encoded CEA-608 or CEA-708 caption data. Wowza Streaming Engine will ingest only the CEA-608 style captions (paint-on, roll-up, or pop-on styles).


    Configuring Application.xml

    To enable onCaption ingestion, in the [install-dir]/conf/[application]/Application.xml file, make the following modifications:

    • Under TimedText/Properties, add the following property:
      <Property>
          	<Name>captionLiveIngestType</Name>
              <Value>onCaption</Value>
              <Type>String</Type>
      </Property>
    • Start Wowza Media Server and publish a live stream to it from an encoder containing onCaption captions.


    For more information about the available properties, see Property details.

    Property details

    • captionLiveIngestOnCaptionCEA608LanguageID: The language code to use for the caption if unspecified in ingest stream. If not specified, defaults to the 3-letter locale of the server.

    • captionLiveIngestOnCaptionCEA608TrackNumber: The track number of the caption text track. If not specified, defaults to 99.

    • captionLiveIngestOnCaptionCEA608EnableField1: Specifies if CEA-608 field 1 is enabled. The default value is true (enabled). Typically, either field 1 or field 2 is enabled, not both.

    • captionLiveIngestOnCaptionCEA608EnableField2: Specifies if CEA-608 field 2 is enabled. The default value is false (disabled).

    • debugCaptionLiveOnCaptionIngest: Specifies if the parsed captions are logged. The default value is false (enabled).


    Playback


    To play on an Apple iOS device

    Apple iOS players can display captions from CEA-608 messages in Apple HLS (Cupertino) streams.

    • Enable closed-captioning in the iOS device. To do this, go to Settings > Video, and then set Closed Captioning to ON.

    • Enter the following URL into the Safari web browser on the device:

      URL: http://[wowza-ip-address]:1935/[application]/myStream/playlist.m3u8


    To play using JW Player (RTMP)

    JW Player can display captions in RTMP streams. Use Version 5.x or 6.5 or later.

    You must download JW Player libraries and deploy an HTML page to a web server. Modify the JW Player playback instructions in How to configure closed captioning for video on demand streaming to use your application and stream name.

    For example, if you were playing an application called cclive and a stream named myStream, your webpage code would look like this for JWPlayer 5.x:
    <div id="sample" ></div>
    <script type="text/javascript">
    jwplayer("sample").setup({
        file: 'myStream',
        flashplayer: 'jwplayer5/player.swf',
        height: 300,
        plugins: {
            'jwplayer5/captions.swf': {}
        },
        streamer: 'rtmp://10.0.0.10:1935/cclive',
        width: 400
    });
    </script>
    And like this for JWPlayer 6:
    <div id="sample" ></div>
    
    <script type="text/javascript">
        jwplayer("sample").setup({
            playlist: [{
                file: 'rtmp://10.0.0.10:1935/cclive/myStream4',
                title: 'sample',
                description: 'sample'
    	}],
            height: 300,
            width: 400
    });
    </script>

    To play using Flowplayer (RTMP and Adobe HDS/San Jose)

    Flowplayer can display captions in RTMP and Adobe HDS (San Jose) streams when the appropriate captioning and streaming plugins are used.

    You must download Flowplayer libraries and deploy an HTML page to a web server. Modify the Flowplayer playback instructions in How to configure closed captioning for video on demand streaming to use your application and stream name.

    For example, if you were playing an application called cclive and a stream named myStream, your webpage code would look like this:
    <a href="" style="display:block;width:400;height:300px" id="sample"> </a>
    
    <script type="text/javascript">
    	flowplayer("sample", "flowplayer/flowplayer-3.2.15.swf", {
    		clip: {
    		url: 'http://10.0.0.10:1935/cclive/myStream/manifest.f4m',
                    urlResolvers: ['f4m'],
    		provider: 'httpstreaming',
    		autoPlay: false,
    		autoBuffering: false,
    		scaling: 'fit',
    		},
    	plugins: {
           		// streaming plugin configuration
    		f4m: { 
    			url: "flowplayer/flowplayer.f4m-3.2.9.swf" 
    	  	},
     
            	httpstreaming: {
                		url: "flowplayer/flowplayer.httpstreaming-3.2.9.swf"
            	},
    
    		captions: {
    			url: 'flowplayer/flowplayer.captions-3.2.9.swf',
    			captionTarget: 'content'
    			},
    		content: {
    			url:'flowplayer/flowplayer.content-3.2.8.swf',
    			bottom: '15%',
    			backgroundColor: 'transparent',
    			backgroundGradient: 'none',
    			border: 0,
    			opacity: .90,
    			textDecoration: 'outline',
    			style: {
    				'body': {
    					fontSize: '18',
    					fontFamily: 'Verdana, Arial, Helvetica',
    					fontWeight: 'bold',
    					textAlign: 'center',
    					color: '#ffff00'
    					}
    				}
    			}
    		},
    	}
    );
    </script>

    Legacy closed captioning functionality


    In Wowza Media Server versions prior to 3.6, live closed captioning is done with a separate module that converts AMF onTextData data events to CEA-608 closed captioning data in live video streams.

    For use with Wowza Media Server version 3.1.1.08 through Wowza Media Server version 3.5.x.

    Overview

    You can download a package that includes a simple module (ModuleClosedCaptionLive) for converting onTextData events into CEA-608 closed captioning data in a live video stream. This package only works for live streaming. The package includes the source code so that you can extend the solution as needed.

    The package also includes an example module (ModulePublishOnTextData) that injects onTextData events into a live stream. It's useful for testing a closed captioning implementation based on the ModuleClosedCaptionLive module. This module takes caption data from a flat text file and injects an onTextData event every 6.5 seconds into the live stream. Again, this module is only for testing purposes and is an example of how to inject onTextData events into a live stream.

    Download Information

    Download ClosedCaptionLive.zip.

    Review the README.html file that's included in the package. This file describes how to set up the implementation.

    For more information about how to set up players to play closed captions, see How to configure closed captioning for video on demand streaming.

    Originally Published: 11-08-2012.
    Updated: For Wowza Streaming Engine on 02-11-2014.

    If you're having problems or want to discuss this article, post in our forum.