Send Apple HLS content to Amazon S3 using Wowza REST APIs

The Stream Targets feature in Wowza Streaming Engine™ media server software allows you to send live streams to widely distributed destinations. 

This article explains how to create a Fastly stream target with a push connection to distribute your lives streams using Wowza CDN on Fastly. You will need an Amazon AWS account and a Wowza Streaming Cloud account for this workflow.

Alternatively, you may configure your connection to for a pull-based workflow where the destination pulls the stream data from the server. 

Unless there is a specific need for a push-based workflow, it's strongly recommended to use a pull-based workflow instead. See Migrate to Wowza CDN on Fastly to learn more about the advantages and disadvantages of the push and pull workflows. 

To configure your stream target using a pull connection, see Send a stream from Wowza Streaming Engine to a Wowza CDN HLS target using Wowza REST APIs.

Before you start


You should have access to the following items:

  • A valid Wowza Streaming Engine license and a Wowza CDN subscription. Contact sales@wowza.com for more information.
  • An Amazon Web Service account with create, read, and write access to S3 storage using access key security credentials. See AWS Free Tier to create a free account.

1. Create a bucket in AWS S3

See Creating a bucket for instructions on how to create a bucket in AWS.

You'll need to configure some settings to allow streams to be played back from your S3 bucket.

Configure the following:

  • ACLS must be enabled. For best results, the Object Ownership should be set to the Object Writer.

Graphical user interface, application

Description automatically generated

  • Public Access must be enabled for objects in the bucket.

Graphical user interface, text, application

Description automatically generated
 

  • For testing direct playback from the bucket, you may need to enable Cross-origin resource sharing (CORS). After the bucket is created, select the Permissions tab for the bucket and add the following CORS configuration.
     
    Note: CORS is only required for direct playback from the bucket. It's not required for playback via the Fastly Custom Stream Target.
[ 
    { 
        "AllowedHeaders": [], 
        "AllowedMethods": [ 
            "GET" 
        ], 
        "AllowedOrigins": [ 
            "*" 
        ], 
        "ExposeHeaders": [] 
    } 
] 

Once your bucket is created, you'll need to record the following information from your amazon account:

  • Bucket name, for example my-s3-bucket
  • Bucket Region, for example us-west-1
  • AWS Bucket URL. This will be used with the Fastly Custom Stream Target. The format for the AWS Bucket URL is https://[bucket-name].s3.[region].amazonaws.com/ 

    For example, https://my-s3-bucket.s3.us-west-1.amazonaws.com/
     
    For buckets in the us-east-1 region, AWS doesn't require the region in the URL (https://[bucket-name].s3.amazonasw.com/). This is a legacy format AWS URL. It won't work with Fastly Custom Stream Targets. Instead, you need to use the regionalised S3 URL for us-east-1, which is https://[bucket-name].s3.us-east-1.amazonaws.com

You'll also need to record your IAM credentials. These are set per user in AWS. See Create an IAM admin and user group in AWS for more information.

Record the following:

  • Access ID, for example AKIAI6234VXXREN3KWJQ
  • Secret Access Key, for example y1PFFPOEwSrUfvvvdalA1qs9sFDM7+QzQTMHoqP7

2. Enable the Wowza Streaming Engine stream target

Before you stream, you must enable the Stream Targets feature for the Wowza Streaming Engine application or the individual stream target you created.

In the contents panel of your application, if you don't see a checkmark next to Stream Targets, click Stream Targets in the contents panel and then click Enable Stream Targets.

When Stream Targets is enabled, a checkmark appears next to Stream Targets in the contents panel and the Stream Targets page shows Status is Enabled.

3. Create a stream target in Wowza Streaming Engine to send the stream to AWS S3

  1. Once you have created your AWS S3 bucket and recorded your credentials, you'll need to create a stream target in Wowza Streaming Engine by sending a POST request to the /v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/live/pushpublish/mapentries/WSC_via_S3 endpoint.


    You can use the following sample request, making sure to:
  2. Set http.relativePlaylists to true. It is set to false by default.
  3. Set the>stream_name[_xxx]to match the incoming stream name or transcoder rendition for each stream target.
  4. Set entryName to a unique name for your stream target.
  5. For adaptive bitrate entries, set profile to group-manager and group.name to a unique name for your stream target.
  6. Set cloudstorage.bucketName to the Bucket Name provided by AWS in Step 1.
  7. Set cloudstorage.region to the Bucket Region provided by AWS in Step 1.
  8. Set cloudstorage.accessID to the Access ID provided by AWS in Step 1.
  9. Set cloudstorage.secretAccessKey to the Secret Access Key provided by AWS in Step 1.
  10. Be sure to include the extraOptions object in the post request, as shown below.
  11. Change any additional values to be unique to your stream target.
  12. Before saving the file, use a JSON Validator, such as https://jsonlint.com, to validate the map entries. Copy the complete text after the = sign into the validator to check the syntax.
Note: For more information on map entries, see Use CDNs and services to distribute live streams from Wowza Streaming Engine.

Sample code — Single bitrate map entries

curl -X POST \ 
-H 'Accept:application/json; charset=utf-8' \ 
-H 'Content-Type:application/json; charset=utf-8' \ 
http://localhost:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/live/pushpublish/mapentries/WSC_via_S3_720p \ 
-d ' 
{ 
    "entryName": "WSC_via_S3_720p", 
    "sourceStreamName": "stream_name", 
    "streamName": "stream_name", 
    "profile": "cupertino-cloudstorage", 
    "debugLog": false, 
    "extraOptions": { 
        "cloudstorage.provider": "S3",         
        "cloudstorage.bucketName": "my-s3-bucket", 
        "cloudstorage.region": "us-west-1", 
        "cloudstorage.accessID": "AKIAI6IO5VXX-EXAMPLE", 
        "cloudstorage.secretAccessKey": "y1PFFPOEwSrUfWyhalA1qs9sFDM7+QzQ-example", 
        "cloudstorage.transportDebug": false, 
        "http.relativePlaylists": true 
    } 
} 

Sample code — Adaptive bitrate map entries

curl --location --request POST 'http://127.0.0.1:8087/v2/servers/_defaultServer_/vhosts/_defaultVHost_/applications/live/pushpublish/mapentries/WSC_via_S3' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data-raw '{
    "profile": "group-manager",
    "childProfile": "cupertino-cloudstorage",
    "sourceStreamName": "stream_name",
    "group.streamMatcher": "namePrefix",
    "group.name": "stream_name_ABR",
    "debugLog": false,
    "extraOptions": {
        "cloudstorage.provider": "S3",
        "cloudstorage.bucketName": "my-s3-bucket",
        "cloudstorage.region": "us-west-1",
        "cloudstorage.accessID": "AKIAI6IO5VXX-EXAMPLE",
        "cloudstorage.secretAccessKey": "y1PFFPOEwSrUfWyhalA1qs9sFDM7+QzQ-example",
        "cloudstorage.transportDebug": false,
        "http.relativePlaylists": true
    }
}
'


Sample Response

{
    "success": true,
    "message": "Entry (WSC via S3 - source) saved successfully",
    "data": null
}
  1. Copy the URL for the playlist from your S3 bucket, for example https://my-s3-bucket.s3-us-west-1.amazonaws.com/myStream/playlist.m3u8
     
    Note: For buckets in the us-east-1 region, this URL will be the legacy format and needs to be changed to include the region. 
  2. Test the playback using the URL for your playlist in your player.

     

    Note: Some players will require CORS headers to be enabled for direct playback from the bucket.

4. Create a Fastly Custom Stream Target in Wowza Streaming Cloud
 

Note: Only one Fastly Custom Stream Target needs to be created for each S3 bucket.

You can use the following sample request, making sure to:

  • Set the custom_origin_region to the region where you Wowza Streaming instance is located.
    Valid values:
    • us - United States
    • eu - Europe
    • asia - Asia
    • sa - South America
    • india - India
       
      Note: custom_origin_region is available in version 1.6 and later of the REST API.
  • Set custom_origin_url to the AWS Bucket URL provided by AWS in step 1.
  • Change any values unique to your broadcast, using the API reference documentation as a resource. See the Endpoint Reference button below.

Sample request

Endpoint Reference

curl -X POST \
-H "Content-Type: application/json" \
-H "wsc-api-key: ${WSC_API_KEY}" \
-H "wsc-access-key: ${WSC_ACCESS_KEY}" \
-d '{
   "stream_target_fastly": {
     "name": "My Wowza CDN on Fastly target"
     "custom_origin_region": "us"
     "custom_origin_url": "https://cdn3.wowza.com/5/QkxIcU9SMnh6d3JX/[application-name]/[stream-name]"
   } 
}' "${WSC_HOST}/api/${WSC_VERSION}/stream_targets/fastly"

Sample response

The response includes:

  • A custom_origin_url you'll use for playback. For example: https://cdn3.wowza.com/5/QkxIcU9SMnh6d3JX/[application-name]/[stream-name]
{
    "stream_target_fastly": {
        "custom_origin_url": https://cdn3.wowza.com/5/QkxIcU9SMnh6d3JX/[application-name]/[stream-name]
        "id": "lwzgrj9r",
        "name": "My first Wowza CDN on Fastly target",
        "state": "activated",
        "stream_name": "ZmYxSXRrTERrUlk9",          
        "playback_urls": {
          "hls": [
            {
	      "name": "default",
              "url": "https://[subdomain].wowza.com/1/[stream_id]/[stream_name]/hls/live/playlist.m3u8"
            }
          ]
        },
        "token_auth_enabled": false,
        "token_auth_playlist_only": false,
        "geoblock_enabled": false,
        "geoblock_by_location": "disabled",
        "geoblock_ip_override": "disabled",
        "force_ssl_playback": false,
        "created_at": "2021-07-22T16:38:21.000Z",
        "updated_at": "2021-07-22T16:38:21.000Z"
    }
}

5. Test playback 

Now that you have retrieved your custom_origin_url, you'll need to modify it slightly to use for playback.

You can modify the root address of the playlist URL you copied from your S3 bucket in Step 3 with the Custom Origin URL excluding the [application-name]/[stream-name] to create a .m3u8 file for playback.

Examples

Custom Origin URL - https://cdn3.wowza.com/5/QkxIcU9S-example/[application-name]/[stream-name] 

Single bitrate (uses streamName in the URL) 

S3 URL -  https://my-s3-bucket.s3.us-west-1.amazonaws.com/myStream/playlist.m3u8 

Wowza CDN URL - https://cdn3.wowza.com/5/QkxIcU9S-example/myStream/playlist.m3u8 

Adaptive bitrate (uses groupName in the URL) 

S3 URL - https://my-s3-bucket.s3.us-west-1.amazonaws.com/myStream-ABR/playlist.m3u8 

Wowza CDN URL - https://cdn3.wowza.com/5/QkxIcU9S-example/myStream-ABR/playlist.m3u8

More resources