How to generate thumbnail images of live streams

This article describes code for an HTTP Provider that enables generation of a thumbnail image from a live stream without the need for the live stream to be the source of a Wowza Transcoder session. The HTTP Provider uses a new API that enables decoding and scaling of key frames using the decoder portion of the Transcoder.

Note: Wowza Streaming Engine™ 4.5.0 or later is required.

The code sample that uses the API is:

ThumbnailerRequest request = new ThumbnailerRequest(stream, width, height, fitMode);
request.addPacket(keyFrame);

ThumbnailerResponse response = ThumbnailerUtils.generateThumbnail(vhost, request);

The thumbnail image is returned as an RGBA byte array. This byte array can then be converted to a Java BufferedImage using the ThumbnailerUtils.nativeImageToBufferedImage(response); utility method. The following example code uses this API to convert the result to JPEG or PNG data:

BufferedImage image = ThumbnailerUtils.nativeImageToBufferedImage(response);
if (image != null)
{
	WMSLoggerFactory.getLogger(CLASS).info(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"]: Image result: format:"+formatStr+" size:"+image.getWidth()+"x"+image.getHeight());
	try
	{
		ByteArrayOutputStream baos = new ByteArrayOutputStream();
		ImageIO.write(image, formatStr.equals("jpeg")?"jpg":formatStr, baos);
		synchronized(this)
		{
			outBytes = baos.toByteArray();
		}
	}
	catch(Exception e)
	{
		WMSLoggerFactory.getLogger(CLASS).error(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"] ", e);
	}
}

The following is the full source for an example HTTP Provider that used this new API:

import java.awt.image.*;
import java.io.*;
import java.util.*;

import javax.imageio.*;

import com.wowza.util.*;
import com.wowza.wms.amf.*;
import com.wowza.wms.application.*;
import com.wowza.wms.http.*;
import com.wowza.wms.logging.*;
import com.wowza.wms.stream.*;
import com.wowza.wms.transcoder.httpprovider.*;
import com.wowza.wms.transcoder.model.*;
import com.wowza.wms.transcoder.util.*;
import com.wowza.wms.vhost.*;

public class HTTPProviderGenerateThumbnail extends HTTProvider2Base
{
	private static final Class<HTTPProviderGenerateThumbnail> CLASS = HTTPProviderGenerateThumbnail.class;
	private static final String CLASSNAME = "HTTPProviderGenerateThumbnail";

	public void onHTTPRequest(IVHost vhost, IHTTPRequest req, IHTTPResponse resp)
	{
		if (!doHTTPAuthentication(vhost, req, resp))
			return;

		byte[] outBytes = null;
		String formatStr = "jpeg";
		String fitModeStr = "letterbox";
		String cropStr = null;
		int[] crop = null;

		try
		{
			while(true)
			{
				String queryStr = req.getQueryString();
				if (queryStr == null)
				{
					WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest: Query string missing");
					break;
				}

		        Map<String, String> queryMap = HTTPUtils.splitQueryStr(queryStr);

		        String streamName = queryMap.get("streamname");
		        if (streamName == null)
				{
					WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest: Streamname not specified");
					break;
				}

		        String comboAppStr = queryMap.get("application");
		        if (comboAppStr == null)
				{
					WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest: Application name not specified");
					break;
				}

		        if (queryMap.containsKey("fitmode"))
		        	fitModeStr = queryMap.get("fitmode");

		        if (queryMap.containsKey("fitMode"))
		        	fitModeStr = queryMap.get("fitmode");

		        if (queryMap.containsKey("format"))
		        	formatStr = queryMap.get("format");

		        if (formatStr == null)
		        	formatStr = "jpeg";

		        formatStr = formatStr.trim().toLowerCase();
		        if (formatStr.equals("jpg"))
		        	formatStr = "jpeg";

		        String sizeStr = queryMap.get("size");
		        int width = 0;
		        int height = 0;
		        if (sizeStr != null)
		        {
			        int cindex = sizeStr.indexOf("x");
			        try
			        {
				        width = Integer.parseInt(sizeStr.substring(0, cindex));
				        height = Integer.parseInt(sizeStr.substring(cindex+1));
			        }
			        catch(Exception e)
			        {

			        }
		        }

		        if (queryMap.containsKey("crop"))
		        	cropStr = queryMap.get("crop");

		        if (cropStr != null)
		        {
		        	String[] parts = cropStr.split(",");
		        	if (parts.length >= 4)
		        	{
		        		crop = new int[4];

		        		for(int i=0;i<4;i++)
		        		{
		        			int cropVal = -1;
		        			try
		        			{
		        				cropVal = Integer.parseInt(parts[i].trim());
		        			}
		        			catch(Exception e)
		        			{
		        			}
		        			if (cropVal > 0)
		        			{
		        				crop[i] = cropVal;
		        			}
		        		}
		        	}
		        }

		        String applicationStr = comboAppStr;
		        String appInstanceStr = IApplicationInstance.DEFAULT_APPINSTANCE_NAME;

		        int cindex = applicationStr.indexOf("/");
		        if (cindex > 0)
		        {
		        	appInstanceStr = applicationStr.substring(cindex+1);
		        	applicationStr = applicationStr.substring(0, cindex);
		        }

				IApplication applicationLoc = vhost.getApplication(applicationStr);
				if (applicationLoc == null)
				{
					WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest: Application could not be loaded: "+applicationStr);
					break;
				}

				IApplicationInstance appInstance = applicationLoc.getAppInstance(appInstanceStr);
				if (appInstance == null)
				{
					WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest: Application instance could not be loaded: "+applicationStr+"/"+appInstanceStr);
					break;
				}

				IMediaStream stream = appInstance.getStreams().getStream(streamName);
				if (stream == null)
				{
					WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"]: Stream not found: "+streamName);
					break;
				}

				AMFPacket keyFrame = stream.getLastKeyFrame();
				if (keyFrame == null)
				{
					WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"]: No key frame");
					break;
				}

				int fitmode = TranscoderStreamUtils.frameSizeFitModeToId(fitModeStr);
				if (fitmode <= 0)
				{
					WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"]: Fit mode not recognized defaulting to letterbox: "+fitModeStr);
					fitmode = TranscoderStream.FRAMESIZE_FITMODE_LETTERBOX;
				}

				ThumbnailerRequest request = new ThumbnailerRequest(stream, width, height, fitmode);

				request.addPacket(keyFrame);
				if (crop != null)
					request.setCrop(crop);

				ThumbnailerResponse response = ThumbnailerUtils.generateThumbnail(vhost, request);
				if (response == null)
				{
					WMSLoggerFactory.getLogger(CLASS).warn(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"]: Thumbnail generation failed");
					break;
				}

				BufferedImage image = ThumbnailerUtils.nativeImageToBufferedImage(response);
				if (image != null)
				{
					WMSLoggerFactory.getLogger(CLASS).info(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"]: Image result: format:"+formatStr+" size:"+image.getWidth()+"x"+image.getHeight());
					try
					{
						ByteArrayOutputStream baos = new ByteArrayOutputStream();
						ImageIO.write(image, formatStr.equals("jpeg")?"jpg":formatStr, baos);
						synchronized(this)
						{
							outBytes = baos.toByteArray();
						}
					}
					catch(Exception e)
					{
						WMSLoggerFactory.getLogger(CLASS).error(CLASSNAME+".onHTTPRequest["+applicationStr+"/"+appInstanceStr+"/"+streamName+"] ", e);
					}
				}
				break;
			}
		}
		catch (Exception e)
		{
			WMSLoggerFactory.getLogger(HTTPTranscoderThumbnail.class).error("HTTPTranscoderThumbnail ", e);
		}

		try
		{
			if (outBytes != null)
			{
				resp.setHeader("Content-Type", "image/"+formatStr);
				OutputStream out = resp.getOutputStream();
				out.write(outBytes);
			}
			else
				resp.setResponseCode(404);
		}
		catch (Exception e)
		{
			WMSLoggerFactory.getLogger(HTTPTranscoderThumbnail.class).error("HTTPTranscoderThumbnail ", e);
		}
	}
}

The above HTTP Provider is built into the Wowza Streaming Engine media server software and can be used by adding the following HTTP Provider definition to your [install-dir]/conf/VHost.xml file. Add it as the second-to-last entry in the HostPort configuration for port 8086 (admin) port:

<HTTPProvider>
	<BaseClass>com.wowza.wms.transcoder.thumbnailer.HTTPProviderGenerateThumbnail</BaseClass>
	<RequestFilters>thumbnail*</RequestFilters>
	<AuthenticationMethod>none</AuthenticationMethod>
</HTTPProvider>

The URL format is:

http://[wowza-ip-address]:8086/thumbnail?application=[application-name]&streamname=[stream-name]&size=[width]x[height]&fitmode=[fitmode]&crop=[left],[right],[top],[bottom]&format=[png,jpg]
  • application: Application name (required).
  • streamname: Stream name (required).
  • size: Thumbnail size in pixels (required).
  • fitmode: Fit mode (how video frame is fit to frame size). Valid values are letterbox, stretch, fit-height, fit-width, match-source, crop (default is letterbox).
  • crop: Number of pixels to crop from source image.
  • format: Format of the image. Valid values are jpeg and png (default is jpeg).

Example:

http://[wowza-ip-address]:8086/thumbnail?application=live&streamname=myStream&size=640x360&fitmode=letterbox

Originally Published: For Wowza Streaming Engine 4.5.0 on 06-23-2016.

If you're having problems or want to discuss this article, post in our forum.