Closed Caption Live Module and Code

This package includes a simple module, ModuleClosedCaptionLive, that converts onTextData data events to CEA-608/CEA-708 closed caption data in a live video stream. This package only works for live streaming. We believe the closed caption data that's injected to be in conformance with the ANSI/CEA-608-E specification. The full UTF-8 single byte character set is supported where it overlaps with the CEA-608 character set. The implementation is quite simple but useful. The source code is included so the solution can be extended as needed.

Also included is a simple module, ModulePublishOnTextData, that injects onTextData events into a live stream. It's useful for testing a closed caption implementation based on the ModuleClosedCaptionLive module. This module takes caption data from a flat text file and injects an onTextData event into the stream every 6.5 seconds. Again, this module is only for testing purposes and is an example of how to inject onTextData events into a live stream.

Note: Wowza Media Server 3.1.1.08 or later is required.

Background

An onTextData event is an AMF (Action Message Format) event that's traditionally associated with Real Time Messaging Protocol (RTMP) and Adobe® HTTP Dynamic Streaming (HDS). These events are used to carry closed caption or subtitle information. An onTextData event usually includes the following fields:

• text. Carries the UTF-8 encoded text for the event.
• lang. Three-letter language ID for the event. (Example: eng)
• trackid (optional). When doing video on demand streaming, this is the ID of the text track in the video file. For live streaming, this field is either omitted or set to 99.

The ModuleClosedCaptionLive module monitors a live stream for onTextData events, extracts the text field, formats it as CEA-608 closed caption data, and injects it into the video stream as SEI NAL units. Many player technologies, including Apple® iOS devices, VideoLAN VLC, and many set-top boxes, can display the embedded closed captioning data.

Using the Wowza Media Server® server-side API, it's possible to inject onTextData events into a live stream using the IMediaStream.sendDirect() API call (there are details below). This enables a publisher that passes closed caption data in the form of timed text events or SMPTE events to use the Wowza Media Server server-side API to inject onTextData into a live stream. Using the ModuleClosedCaptionLive module, the onTextData is intercepted and injected into the stream as CEA-608 events.

In the future, we plan to take this functionality further by providing a means to extract closed caption data from an MPEG-TS stream and re-inject it into the video portion of the stream. At this time, it's not possible to do this with the current Wowza Media Server server-side API.

To install

1. Copy [package]/lib/wms-plugin-closedcaption.jar to the [install-dir]/lib folder.


2. Copy [package]/content/ontextdata.txt to the [install-dir]/content folder.


3. Set up an application for live streaming. It's best to set up the application for Apple HTTP Live Streaming (Cupertino) so that you can easily test the closed captions using an iOS-based device. To do this, add cupertinostreamingpacketizer to the <LiveStreamPacketizers> list in [install-dir]/conf/[application]/Application.xml.


4. Add and configure ModulePublishOnTextData in [install-dir]/conf/[application]/Application.xml:
   
   
   1. Add the ModulePublishOnTextData module to the end of the <Modules> list:

      <Module>
      	<Name>ModulePublishOnTextData</Name>
      	<Description>ModulePublishOnTextData</Description>
      	<Class>com.wowza.wms.plugin.closedcaption.test.ModulePublishOnTextData</Class>
      </Module>
      

   2. Add the following properties to the application-level <Properties> container at the bottom of the file (be sure to get the correct <Properties> container - there are several in the Application.xml file):

      <Property>
      	<Name>publishOnTextDataFile</Name>
      	<Value>${com.wowza.wms.context.VHostConfigHome}/content/ontextdata.txt</Value>
      </Property>
      <Property>
      	<Name>publishOnTextDataPublishInterval</Name>
      	<Value>6500</Value>
      	<Type>Integer</Type>
      </Property>
      <Property>
      	<Name>publishOnTextCharsetTest</Name>
      	<Value>false</Value>
      	<Type>Boolean</Type>
      </Property>
      

      Where:

      • publishOnTextDataFile: Path to the ontextdata.txt text file.
      • publishOnTextDataPublishInterval: Specifies how often events are injected, in milliseconds.
      • publishOnTextCharsetTest: If set to true, the module will publish test onTextData events that cover the full UTF-8 character set from 0x20-0xFF.

   
   
5. Add and configure ModuleClosedCaptionLive in [install-dir]/conf/[application]/Application.xml:
   
   
   1. Add the ModuleClosedCaptionLive module to the end of the <Modules> list:

      <Module>
      	<Name>ModuleClosedCaptionLive</Name>
      	<Description>ModuleClosedCaptionLive</Description>
      	<Class>com.wowza.wms.plugin.closedcaption.live.ModuleClosedCaptionLive</Class>
      </Module>
      

   2. Add the following properties to the application-level <Properties> container at the bottom of the file (be sure to get the correct <Properties> container - there are several in the Application.xml file):

      <Property>
      	<Name>closedCaptionLiveMaxDisplayTime</Name>
      	<Value>10000</Value>
      	<Type>Integer</Type>
      </Property>
      <Property>
      	<Name>closedCaptionLiveCommandsPerFrame</Name>
      	<Value>30</Value>
      	<Type>Integer</Type>
      </Property>
      <Property>
      	<Name>closedCaptionLiveChannel</Name>
      	<Value>0</Value>
      	<Type>Integer</Type>
      </Property>
      <!-- white:0, green:2, blue:4, cyan:6, red:8, yellow:10, magenta:12 -->
      <Property>
      	<Name>closedCaptionLiveColor</Name>
      	<Value>0</Value>
      	<Type>Integer</Type>
      </Property>
      <Property>
      	<Name>closedCaptionLiveCharacterSet</Name>
      	<Value>UTF-8</Value>
      </Property>
      <Property>
      	<Name>closedCaptionLiveLogOnTextDataEvents</Name>
      	<Value>true</Value>
      	<Type>Boolean</Type>
      </Property>
      <Property>
      	<Name>closedCaptionLiveRemoveExistingCEA608</Name>
      	<Value>false</Value>
      	<Type>Boolean</Type>
      </Property>
      

      Where:

      • closedCaptionLiveMaxDisplayTime: Maximum time, in milliseconds, that a closed caption will stay on the screen.
      • closedCaptionLiveCommandsPerFrame: Maximum number of closed caption commands to send per-frame (max is 31).
      • closedCaptionLiveChannel: Close captioning channel (valid values: CC1:0, CC2:1 - only CC1 seems to work with Apple iOS devices).
      • closedCaptionLiveColor: Color of closed caption text (valid values: white:0, green:2, blue:4, cyan:6, red:8, yellow:10, magenta:12)
      • closedCaptionLiveCharacterSet: Closed caption character set to use when converting the onTextData text data to caption data.
      • closedCaptionLiveLogOnTextDataEvents: Controls logging of the individual onTextData events. If true, all onTextData events are logged.
      • closedCaptionLiveRemoveExistingCEA608: Controls if CEA-608 already in live stream gets removed. Off by default. Requires Wowza 3.5.2.04 or later

6. Start Wowza Media Server and publish a live stream to the server. For this example, we'll use the stream name myStream.


7. Play the live stream on an Apple iOS device using the following URL:

   http://[wowza-ip-address]:1935/[application]/myStream/playlist.m3u8

   Note: Captions will only display if closed captioning is turned on. To turn on closed captioning on an iOS device, go to Settings > Video and turn on the Closed Captioning option.

Notes

If you plan to extend the source code, it's best to read the ANSI/CEA-608-E specification. (It can be purchased here: http://standards.ce.org.) For a good overview of CEA-608/CEA-708 data, see "Closed Captions and the SCC Format" (http://www.theneitherworld.com/mcpoodle/SCC_TOOLS/DOCS/SCC_FORMAT.HTML). Wowza Media Server 3.1.1.08 or later includes the ClosedCaptionCEA608Utils utility class, which includes most of the closed captioning commands that are needed to extend the code. The Javadocs for this class are included in the Wowza Media Server Server-Side API documentation.

The source code is included so that you can extend it as needed. The charToByte(String text, int pos, String characterSet, int channel) method is used to convert characters to closed captioning character codes. It's this method that must be extended to add support for additional character sets.

Several special characters are mapped to UTF-8 control codes that weren't being used so that the full CEA-608 character set was covered. Below are these mappings:

To add onTextData events to a live stream, use the sendDirect(handler, param...) method. The code to add an onTextData event to a stream looks like this:

AMFDataObj amfData = new AMFDataObj();

amfData.put("text", new AMFDataItem(onTextData.text));
amfData.put("language", new AMFDataItem("eng"));
amfData.put("trackid", new AMFDataItem(99));

stream.sendDirect("onTextData", amfData);