Recording and playback with audio on S3

[ange52]

Hi folks,

I am trying to build an application that will allow me to record an audio FLV stream to an S3 bucket and then play the stream back any time. I've been trying to do a lot of reading on the site and on the forum, and I understand a few key elements:

- I know that I need to record the stream to EC2 then copy the saved stream over to S3 after the recording is complete
- I know that I can mount an S3 bucket from EC2 to stream the FLV's for playback

I basically have the playback part of the equation sorted out by using the vods3 default Wowza application. I'm having a much harder time trying to get the recording part of the application up and running, however. I have a number of questions:

- Is recording available on one of the default applications? I know I should use the vods3 application for playback... are there default options available for recording, too?

- Immediately after I record a stream, how should I play it back? Is it possible to have the same application record a stream to EC2, copy it to S3, and be able to play back whichever stream is available? (Meaning after I record, can it play the stream from S3 by default, or check EC2 if it is not available on S3 yet?)

Thanks so much,
Ange52

[charlie]

You can repord using the record-live stream type. It is covered in the User's Guide. Or you can control recording through Java using this package:

http://www.wowzamedia.com/forums/showthread.php?t=2733

The steps to then automatically copy to S3 are here:

http://www.wowzamedia.com/forums/showthread.php?t=5849

It is probably best to set this up then ask specific questions if you have problems.

Charlie

[ange52]

Thanks for your reply, Charlie.

I have done a bit more reading and experimenting, and I can make my question much more specific. For right now, the defaults are good enough to help me get things started, so my initial setup is as basic as can be:

I fire up a new EC2 instance, create the /mnt/s3 directory, and tie it to my s3 bucket. Next I add the module and modify the properties of conf/vods3/Application.xml
(both steps following http://www.wowzamedia.com/forums/showthread.php?t=5849)

Next I open up videorecording/client/videorecording.html on my local machine, enter in my EC2 address and connect. Everything is great so far. I click record, say hi to the camera, then click stop.

The app automatically tries to play the video back to me, but says that it can't find the stream. Fail.

I still want to confirm that the video recorded successfully, so next I wait a minute (maybe the file is being copied to S3 at this point), and I open up simplevideostreaming/client/simplevideostreaming.html. I add in the server and the stream:

rtmp://[ec2-address].amazonaws.com/vods3
flv:amazons3/[my-bucket]/videorecording.flv

And it works! I can play it back at this point.
My question is how to correct this situation so that I can record and playback with vods3 without the point of failure and without having to open up simplevideostreaming to be able to play my stream back.

Do you have any recommendations for me? Thanks so much!

[charlie]

The VideoRecording example is trying to play it from the content directory on the local instance. After we copy it to S3 we delete it. So the local copy is not there.

To see this work you will need to modify the VideoRecording example and change the stream name it uses to play the stream after recording. I think you will find it works after doing this.

Charlie

[ange52]

Ok -- this turned out to be much simpler than I had expected! Gotta love that. Thanks for your patience, Charlie. Things are really moving along now.

A few follow-up questions:

1) Is there much of a delay when the FLV is being copied from EC2 to S3? As soon as the recording is completed, I now update the stream url to point to the file in my S3 bucket. Could this be a problem? For example, what if a user records a 10-minute audio clip then tries to play it back right away?

2) I'm encountering problems with my recordings getting clipped. For example, I hit the "record" button, then wait for a few seconds, then start talking about the weather for 10 seconds, then hit stop. When I play it back, the seconds of silence are gone, as well as half of the first word. Sometimes the end of the recording gets cut off, too. Do you have any ideas why this could be happening? Is there an effective way to change this? Does it have something to do with recording quality settings in vods3?

3) Related to quality settings: since I am only trying to record audio, are there settings I could be changing in vods3 to improve recording and playback performance?

Thanks so much! I really appreciate it.
Ange52

[charlie]

I suggest you carefully study the VideoRecording example. The steps that are needed to properly record are all contained within this example. The sequence of events and timing of these events is very import. This example includes the right checks to make it work properly.

1) Is there much of a delay when the FLV is being copied from EC2 to S3? As soon as the recording is completed, I now update the stream url to point to the file in my S3 bucket. Could this be a problem? For example, what if a user records a 10-minute audio clip then tries to play it back right away?

Once you receive the NetStream.Unpublish.Success NetStream event the file should be there.

2) I'm encountering problems with my recordings getting clipped. For example, I hit the "record" button, then wait for a few seconds, then start talking about the weather for 10 seconds, then hit stop. When I play it back, the seconds of silence are gone, as well as half of the first word. Sometimes the end of the recording gets cut off, too. Do you have any ideas why this could be happening? Is there an effective way to change this? Does it have something to do with recording quality settings in vods3?

Be sure the Microphone.setSilenceLevel is set to zero and again study the VideoRecording example. You need to be sure you wait for the NetStream.bufferLength to be zero before you unpublish the stream.

3) Related to quality settings: since I am only trying to record audio, are there settings I could be changing in vods3 to improve recording and playback performance?

All the setting are part of the Microphone object.

Charlie

[ange52]

You're the best! Thanks so much.

[slegay]

1) Is there much of a delay when the FLV is being copied from EC2 to S3?

From my experience, using a small instance, we get about 50-60 Mbps (about 7MB / sec) upload speed on a single S3 connection. At 150 kbps, a 15 minute recording is about 130 MB, which takes around 20 seconds to upload.

Now, keep in mind you may have more than one user concurrently saving a recording to S3. When that happens, right now I don't know whether Amazon throttles your EC2 / S3 speed per connection or per machine. If it's per machine, your 60 Mbps will be divided by the number of concurrent connections.

Also, it is possible that you'd get a better throughput using a Large instance. I'm not sure about it though, you may want to test that first.

When saving lengthy recordings, I strongly recommend building a custom module that will offload the S3 upload to a separate thread pool to have better control over what's going on behind the scenes. It'll also allow you to monitor upload progress and push that progress down to your Flash clients through the NetConnection. That's what we did, and it makes a 30 second wait much more tolerable on the client.

[ange52]

Thanks for the details, slegay. And your custom module sounds brilliant. Do you have any tips on how to get started writing that kind of an implementation?

[slegay]

You'll need a decent understanding of Java. If you've never developed a Wowza custom module, downloading the Wowza IDE will save you some set up time. Read the Wowza doc. You basically need to create a class that extends ModuleBase and package it in a jar, it's very straightforward.

I can't put the source code here, but here's the idea. For our code I've been using the jets3t S3 library for Java. By digging through the source code I found that that library supports progress tracking if you feed it with a ProgressMonitoredInputStream, which wraps around your file stream.

You'll need 3 classes:

- An UploadStreamWorker, which implements Runnable and therefore can be started in a separate thread. The "work" method will take care of starting the S3 upload and kick off a progress watcher (see below).

- A SendClientProgressUpdateWorker, which implements Runnable as well as IModuleCallResult. This worker will also be run a separate thread, watching the UploadStreamWorker and sending progress back to the client by using client.call().

- Your custom module, which will expose a "saveRecording" method your client can call to start the save cycle. This method will simply start the upload thread, which will start the progress update thread, and return immediately.

Once the upload is complete, call an uploadComplete method on your client from the ProgressUpdateWorker.

The whole thing is only about 150 lines of code.