Record stream directly to S3 with AWS Multipart Upload?

[rrlanham]

Copy the command to ssh console and see it works. Double check everything. Did you add the startup.xml RunScript tag to run init.sh? It did work when I tested it.

Richard

[davidlgood]

I'm using the ami-5453a53d Amazon EC2 Amazon Machine Image ID.

In the startup package (based off the 'startup_2.2' provided example) I modify the "startup.xml" file:

- - - - - - - - - - - - - - - - - - - -

<Startup>
<Commands>
<Install>
<Folder>wowza</Folder>
</Install>
<RunScript>
<Script>tuning/tune.sh</Script>
</RunScript>
<RunScript>
<Script>init.sh</Script>
</RunScript>
</Commands>
</Startup>

- - - - - - - - - - - - - - - - - - - -

I then create an "init.sh" file and place it at the same level as the "startup.xml" file and "wowza" and "tuning" directories. The "init.sh" file looks like this (with the XXXXXXXXXX being replaced with my Amazon AWS keys:

- - - - - - - - - - - - - - - - - - - -

#!/bin/sh
mkdir /mnt/s3
/usr/bin/s3fs cogwareplayhouston.s3.amazonaws.com -o accessKeyId=XXXXXXXXXX -o secretAccessKey=XXXXXXXXXX -o default_acl=public-read /mnt/s3

- - - - - - - - - - - - - - - - - - - -

I have an S3 bucket called "cogwareplayhouston" that sits at the top level of my S3 account. I'm assuming that "cogwareplayhouston.s3.amazonaws.com" is the correct way to point the script to this bucket, right?

- - - - - - - - - - - - - - - - - - - -

I then used the instructions, as noted above by acropolis, found here: http://www.wowzamedia.com/forums/content.php?72

I didn't do the mkdir command or mount the S3 bucket from these instructions, since I assume those are handled by the "init.sh"

What I did was modify the Wowza application that records video. In my case, I'm using "rtplive" to stream live video.

I then added the following, exactly, to the end of the "Modules" list and the "Properties" list in the "Application.xml that sits in "conf/rtplive/. Since there were three "<Properties>" areas I assumed it was the was the one right below the Modules list I just added to (just before "</Application>"). So, the final portion of the "Application.xml" looks like this:

- - - - - - - - - - - - - - - - - - - -

<Module>
<Name>ModuleMediaWriterFileMover</Name>
<Description>ModuleMediaWriterFileMover</Description>
<Class>com.wowza.wms.module.ModuleMediaWriterFileM over</Class>
</Module>
</Modules>
<Properties>
<Property>
<Name>fileMoverDestinationPath</Name>
<Value>/mnt/s3</Value>
</Property>
<Property>
<Name>fileMoverDeleteOriginal</Name>
<Value>false</Value>
<Type>Boolean</Type>
</Property>
<Property>
<Name>fileMoverVersionFile</Name>
<Value>true</Value>
<Type>Boolean</Type>
</Property>
</Properties>
</Application>
</Root>

- - - - - - - - - - - - - - - - - - - -

I use the LiveStreamRecord example Flash player to "Start Recording" and then "Pause Recording". When I FTP into the server I can see the "myStream.sdp.flv" file that is recorded, as well as the temporary file being written if I am still 'recording'. When I'm done recording the temporary file goes away as it becomes part of the "myStream.sdp.flv" file that resides in the "content" folder.

So, the file is correctly being recorded and saved to the "content" folder… but it's not being moved to the s3 folder -- in fact, when I FTP into the server I don't see a "s3" folder anywhere, not even in the "mnt" directory.

In the "mnt" directory I see "lost+found", "mediacache" and "WowzaMediaServer" as the only directories.


I'm not sure what I'm missing here. Any help would be greatly appreciated.

[rrlanham]

I think I just used the simple bucket name:

/usr/bin/s3fs cogwareplayhouston -o accessKeyId=XXXXXXXXXX -o secretAccessKey=XXXXXXXXXX -o default_acl=public-read /mnt/s3

Your credentials provide the location, i.e., a bucket belonging to you.

Richard

[davidlgood]

Richard

That was the issue. I changed the s3 location to "cogwareplayhouston" and everything worked like magic. Thanks for your help on this!!!

[rrlanham]

You're welcome, thanks for the update

Richard

[ertu22]

Should I be able to see s3 in FileZilla?

[ianbeyer]

If it's mounted with s3fs, within the FTP path, yes, you should be able to see it.

[ertu22]

It's mounted with s3fs. Everything is working fine except I cant see the 's3' directory in FileZilla.
What do you mean within FTP path?

[savvas]

Richard,

unfortunately it doesn't work on our case.

We use EC2 with live-record and the module:

<Module>
<Name>ModuleMediaWriterFileMover</Name>
<Description>ModuleMediaWriterFileMover</Description>
<Class>com.wowza.wms.module.ModuleMediaWriterFileM over</Class>
</Module>

and the properties:

<Property>
<Name>fileMoverDestinationPath</Name>
<Value>/mnt/s3</Value>
</Property>
<Property>
<Name>fileMoverDeleteOriginal</Name>
<Value>false</Value>
<Type>Boolean</Type>
</Property>
<Property>
<Name>fileMoverVersionFile</Name>
<Value>true</Value>
<Type>Boolean</Type>
</Property>

It saves perfectly well the files on /mnt/s3,

Now, if we "/usr/bin/s3fs mybucket -o accessKeyId=XXXXX -o secretAccessKey=XXXXXX -o default_acl=public-read /mnt/s3", and try to navigate to the /mnt/s3 folder it doesn't allow us to even open the folder.
Because of this, we are not able to use FME to stream. We have to reboot the machine for everything to work.

Is there a chance for this be a permission issue or something else?

Thank you Richard!

[savvas]

I spent all day trying to find what was the problem...! No EU buckets!