Generate thumbnail images of live streams using the Wowza Streaming Engine Java API

This article describes code for an HTTP provider that enables generation of a thumbnail image from a live stream without the need for the live stream to be the source of a Transcoder session. The HTTP provider uses an API that enables decoding and scaling of key frames using the decoder portion of the Transcoder.

Note: Wowza Streaming Engine™ 4.5.0 or later is required.

This code example uses the API:

ThumbnailerRequest request = new ThumbnailerRequest(stream, width, height, fitMode);
request.addPacket(keyFrame);

ThumbnailerResponse response = ThumbnailerUtils.generateThumbnail(vhost, request);

The thumbnail image is returned as an RGBA byte array. This byte array can then be converted to a Java BufferedImage using the ThumbnailerUtils.nativeImageToBufferedImage(response); utility method. The following example code uses this API to convert the result to JPEG or PNG data:

BufferedImage image = ThumbnailerUtils.nativeImageToBufferedImage(response);
if (image != null)
{
	WMSLoggerFactory.getLogger(CLASS).info(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"]: Image result: format:"+formatStr+" size:"+image.getWidth()+"x"+image.getHeight());
	try
	{
		ByteArrayOutputStream baos = new ByteArrayOutputStream();
		ImageIO.write(image, formatStr.equals("jpeg")?"jpg":formatStr, baos);
		synchronized(this)
		{
			outBytes = baos.toByteArray();
		}
	}
	catch(Exception e)
	{
		WMSLoggerFactory.getLogger(CLASS).error(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"] ", e);
	}
}

The following is the full source for an example HTTP provider that used this API:

import java.awt.image.*;
import java.io.*;
import java.util.*;

import javax.imageio.*;

import com.wowza.util.*;
import com.wowza.wms.amf.*;
import com.wowza.wms.application.*;
import com.wowza.wms.http.*;
import com.wowza.wms.logging.*;
import com.wowza.wms.stream.*;
import com.wowza.wms.transcoder.httpprovider.*;
import com.wowza.wms.transcoder.model.*;
import com.wowza.wms.transcoder.thumbnailer.*;
import com.wowza.wms.transcoder.util.*;
import com.wowza.wms.vhost.*;

public class HTTPProviderGenerateThumbnail extends HTTProvider2Base
{
    private static final Class<HTTPProviderGenerateThumbnail> CLASS = HTTPProviderGenerateThumbnail.class;
    private static final String CLASSNAME = "HTTPProviderGenerateThumbnail";

    public void onHTTPRequest(IVHost vhost, IHTTPRequest req, IHTTPResponse resp)
    {
        if (!doHTTPAuthentication(vhost, req, resp))
            return;

        byte[] outBytes = null;
        String formatStr = "jpeg";
        String fitModeStr = "letterbox";
        String cropStr = null;
        int[] crop = null;

        try
        {
            while(true)
            {
                String queryStr = req.getQueryString();
                if (queryStr == null)
                {
                    WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest: Query string missing");
                    break;
                }

                Map<String, String> queryMap = HTTPUtils.splitQueryStr(queryStr);

                String streamName = queryMap.get("streamname");
                if (streamName == null)
                {
                    WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest: Streamname not specified");
                    break;
                }

                String comboAppStr = queryMap.get("application");
                if (comboAppStr == null)
                {
                    WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest: Application name not specified");
                    break;
                }

                if (queryMap.containsKey("fitmode"))
                    fitModeStr = queryMap.get("fitmode");

                if (queryMap.containsKey("fitMode"))
                    fitModeStr = queryMap.get("fitmode");

                if (queryMap.containsKey("format"))
                    formatStr = queryMap.get("format");

                if (formatStr == null)
                    formatStr = "jpeg";

                formatStr = formatStr.trim().toLowerCase();
                if (formatStr.equals("jpg"))
                    formatStr = "jpeg";

                String sizeStr = queryMap.get("size");
                int width = 0;
                int height = 0;
                if (sizeStr != null)
                {
                    int cindex = sizeStr.indexOf("x");
                    try
                    {
                        width = Integer.parseInt(sizeStr.substring(0, cindex));
                        height = Integer.parseInt(sizeStr.substring(cindex+1));
                    }
                    catch(Exception e)
                    {

                    }
                }

                if (queryMap.containsKey("crop"))
                    cropStr = queryMap.get("crop");

                if (cropStr != null)
                {
                    String[] parts = cropStr.split(",");
                    if (parts.length >= 4)
                    {
                        crop = new int[4];

                        for(int i=0;i<4;i++)
                        {
                            int cropVal = -1;
                            try
                            {
                                cropVal = Integer.parseInt(parts[i].trim());
                            }
                            catch(Exception e)
                            {
                            }
                            if (cropVal > 0)
                            {
                                crop[i] = cropVal;
                            }
                        }
                    }
                }

                String applicationStr = comboAppStr;
                String appInstanceStr = IApplicationInstance.DEFAULT_APPINSTANCE_NAME;

                int cindex = applicationStr.indexOf("/");
                if (cindex > 0)
                {
                    appInstanceStr = applicationStr.substring(cindex+1);
                    applicationStr = applicationStr.substring(0, cindex);
                }

                IApplication applicationLoc = vhost.getApplication(applicationStr);
                if (applicationLoc == null)
                {
                    WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest: Application could not be loaded: "+applicationStr);
                    break;
                }

                IApplicationInstance appInstance = applicationLoc.getAppInstance(appInstanceStr);
                if (appInstance == null)
                {
                    WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest: Application instance could not be loaded: "+applicationStr+"/"+appInstanceStr);
                    break;
                }

                IMediaStream stream = appInstance.getStreams().getStream(streamName);
                if (stream == null)
                {
                    WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"]: Stream not found: "+streamName);
                    break;
                }

                AMFPacket keyFrame = stream.getLastKeyFrame();
                if (keyFrame == null)
                {
                    WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"]: No key frame");
                    break;
                }

                int fitmode = TranscoderStreamUtils.frameSizeFitModeToId(fitModeStr);
                if (fitmode <= 0)
                {
                    WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"]: Fit mode not recognized defaulting to letterbox: "+fitModeStr);
                    fitmode = TranscoderStream.FRAMESIZE_FITMODE_LETTERBOX;
                }

                ThumbnailerRequest request = new ThumbnailerRequest(stream, width, height, fitmode);

                request.addPacket(keyFrame);
                if (crop != null)
                    request.setCrop(crop);

                ThumbnailerResponse response = ThumbnailerUtils.generateThumbnail(vhost, request);
                if (response == null)
                {
                    WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"]: Thumbnail generation failed");
                    break;
                }

                BufferedImage image = ThumbnailerUtils.nativeImageToBufferedImage(response);
                if (image != null)
                {
                    WMSLoggerFactory.getLogger(CLASS).info(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"]: Image result: format:"+formatStr+" size:"+image.getWidth()+"x"+image.getHeight());
                    try
                    {
                        ByteArrayOutputStream baos = new ByteArrayOutputStream();
                        ImageIO.write(image, formatStr.equals("jpeg")?"jpg":formatStr, baos);
                        synchronized(this)
                        {
                            outBytes = baos.toByteArray();
                        }
                    }
                    catch(Exception e)
                    {
                        WMSLoggerFactory.getLogger(CLASS).error(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"] ", e);
                    }
                }
                break;
            }
        }
        catch (Exception e)
        {
            WMSLoggerFactory.getLogger(HTTPTranscoderThumbnail.class).error("HTTPTranscoderThumbnail ", e);
        }

        try
        {
            if (outBytes != null)
            {
                resp.setHeader("Content-Type", "image/"+formatStr);
                OutputStream out = resp.getOutputStream();
                out.write(outBytes);
            }
            else
                resp.setResponseCode(404);
        }
        catch (Exception e)
        {
            WMSLoggerFactory.getLogger(HTTPTranscoderThumbnail.class).error("HTTPTranscoderThumbnail ", e);
        }
    }
}

The above HTTP provider is built into Wowza Streaming Engine and can be used by adding the following HTTP provider definition to the [install-dir]/conf/VHost.xml file. Add it as the second-to-last entry in the HostPort configuration for port 8086 (admin) port:

<HTTPProvider>
	<BaseClass>com.wowza.wms.transcoder.thumbnailer.HTTPProviderGenerateThumbnail</BaseClass>
	<RequestFilters>thumbnail*</RequestFilters>
	<AuthenticationMethod>none</AuthenticationMethod>
</HTTPProvider>

The URL format is:

http://[wowza-ip-address]:8086/thumbnail?application=[application-name]&streamname=[stream-name]&size=[width]x[height]&fitmode=[fitmode]&crop=[left],[right],[top],[bottom]&format=[png,jpg]

  • application: Application name (required).
  • streamname: Stream name (required).
  • size: Thumbnail size in pixels (required).
  • fitmode: Fit mode (how video frame is fit to frame size). Valid values are letterbox, stretch, fit-height, fit-width, match-source, crop (default is letterbox).
  • crop: Number of pixels to crop from source image.
  • format: Format of the image. Valid values are jpeg and png (default is jpeg).

Example:

http://[wowza-ip-address]:8086/thumbnail?application=live&streamname=myStream&size=640x360&fitmode=letterbox