Low Latency Architecture for live auction streaming

[system]

I am looking to set up an audio/video streamer with a live auction and allow the clients to bid remotely. Latency needs to be very low. Less than 3 seconds for the video. Less than 1 second for the audio preferred. The audio and video can be decoupled into separate streams. I am willing to spend up to $800/stream source in order to set up a decent quality stream 320X240@8fps and up to 300kbps/sream. Low latency is more important than quality. On low-bandwidth client connections I would be fine with seeing video frames drop as long as the audio stays in-tact and low latency. If latency gets above a few seconds I would expect the player to drop frames to catch up. I would like to know any tips that people can give out to help me achieve this. Here is the direction we are starting. Any suggestions would be helpful.

• We intend to use high-quality analog cameras connected to osprey 100 cards. We intend to use commercial grade microphones and audio inputs.

• We would prefer to use the flash client to encode if possible because that means as little configuration as possible for our clients who need to stream. We could have up to 100 viewers attending a live stream. We are going to start testing with the following set in the flash streaming client.

  camera.setMode(320, 240, 8, false); //320 width, 240 height, 8 fps

  camera.setQuality(300 * 125, 0); //300 * 1kb, prefer bandwidth over quality

  camera.setKeyFrameInterval(40); //40/8 = 5 second keyframe interval (particularly long)

• We are going to use Wowza through EC2. Because we want the low latency we are planning on using the live-low-latency stream. I noticed the system says this stream is optimized for chat applications. Is that simply a because the latency is so low, or are there other factors? How different is the live stream?

• I saw a post that talked about modifying maxliveaudiolatency, behindDropDFrames, behindDropPFrames, and behindDropKFrames. What do each of these settings mean? Where can I find more information?.

• We are planning to use flash as the viewer with bufferTime set to .1 and live=true.

  Are there any other settings / technologies / etc that you can recommend for me? I know that flash is built ontop of TCP which is nerve-racking to me because it means the protocol is preoccupied with making sure it does not lose packets. Can I get better performance / lower latency / less latency degredation over time by using a UDP-based protocol like RTP? What are my options that way?

[Richard_Lanham]

I wonder if you are aiming higher then necessary on the hardware side if you are just going to use Flash plugin for encoding. I’m not sure, but the benefit might not be there compared to a good web cam. If you use another software or hardware encoder you would get more out of a high-end camera.

Assuming you do you use Flash for encoding, the settings are good, conservative. But might reduce even further utilization of the camera.

For keyframe try 16. Rule of thumb: keyframes = (fps *2) or lower. Setting keyframe too high will cause delay in play start and seek time.

You can compare stream types by looking at /conf/Streams.xml. You can modify the available stream types, cautiously of course. I think you can create your own stream type in Streams.xml, experiment that way.

We are planning to use flash as the viewer with bufferTime set to .1 and live=true.

That buffer might be okay, and try a buffer that is a little higher too.

In Wowza you set the stream type on the server in the Application.xml /StreamType. You can change that from the client with a netconnection call:

netconnection.call("setStreamType","live")
// then create a new netstream that will have a live streamtype:
ns = new NetStream(netconnection);
ns.play("somelivestream");

maxliveaudiolatency, behindDropDFrames, behindDropPFrames, and behindDropKFrames

I don’t have detailed info on those settings. Which post?

Richard