How to transport data to Google Cloud Platform services

Wowza Streaming Engine™ media server software (4.6.0.03 and later) can be used to transport data to external services. This article describes how to transport data to Google Cloud Platform.

Contents


Requirements
Create a Google Cloud Platform transport provider Configure the MediaTransport listeners Configure a Google Cloud Platform bucket Configure the Google Cloud Platform transport object

Requirements


To use the TransportMedia packages to transport data to Google Cloud Platform, you must have the following:

  • Wowza Streaming Engine 4.6.0.03 or later
  • Knowledge of programming in Java
  • Wowza IDE
  • A Google Cloud Platform account with the following:
    • Google Cloud JASON private key or PKCS 12 file
    • Google service ID (this is usually an email address)
    • Google project ID
  • Depending on the functionality you use, you may also need a bucket name.

Create a Google Cloud Platform transport provider


The following example demonstrates how to create  and initialize a transport provider:

String bucketName = "";
String serviceID = "";
String secretKey = "";
String projectID = "";

MediaTransport mediaTransport = new MediaTransport();
mediaTransport.setDebug(this.debug);
mediaTransport.init();
mediaTransport.addTransportListener(new MediaTransportListener());

MediaTransportConfigurationGoogleCloud googleConfig = new MediaTransportConfigurationGoogleCloud ();
MediaTransportProviderGoogleCloud googleProvider = new MediaTransportProviderGoogleCloud ();

// Now we configure the provider configuration with our base configuration items
googleConfig.setAttributeString(MediaTransportConfigurationGoogleCloud.GC_CONFIG_BUCKET,bucketName);
googleConfig.setAttributeString(MediaTransportConfigurationGoogleCloud.GC_CONFIG_SECRETACCESSKEY,secretKey);
googleConfig.setAttributeString(MediaTransportConfigurationGoogleCloud.GC_CONFIG_SERVICEID,serviceID);
googleConfig.setAttributeString(MediaTransportConfigurationGoogleCloud.GC_CONFIG_PROJECT_ID,projectID);

googleConfig.init();
// Now we add the configuration to the transport provider
googleProvider.setTransportProviderConfiguration(googleConfig);
// Now we initialise the transport provider
// The transport provider can be done SYNC or ASYNC, ASYNC is default.
googleProvider.init();
googleProvider.setTransportProcessType(IMediaTransportProvider.TRANSPORT_PROVIDER_PROCESS_SYNC);
if ( googleProvider.isReady() )
{
	mediaTransport.addTransportProvider(googleProvider);
}

Where:

  • bucketName is the name of the bucket in Google Cloud Platform. You'll need to use the same bucketName when you Create a bucket.
  • secretaccesskey is the secret access key. For JSON private keys, this is the entire private key including -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY-----\n. For PKCS 12 files, this is the password required to access the file.
  • serviceID is the service ID (email address) associated with your Google Cloud Platform account.
  • projectID is the project number associated with your project in Google Cloud Platform. Note that this is not the same as the Project ID in Google Cloud Platform.

Google Cloud Platform transport provider parameters

The following static parameters are designed so that they can be used to easily added to a provider or, in some cases, on a per-transport object basis.

GC_CONFIG_BUCKET

The GC_CONFIG_BUCKET parameter sets the name of the bucket.

Use the following command to apply this parameter to the transport configuration:

googleConfig.setAttributeString(MediaTransportConfigurationGoogleCloud.GC_CONFIG_BUCKET,”mybucket”);

GC_CONFIG_SECRETACCESSKEY

The GC_CONFIG_SECRETACCESSKEY parameter specifies the secret access key:

  • JSON private key - Provide the entire private_key string, including -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY-----\n.
  • PKCS 12 file - Provide the password that is required to open the file.

GC_CONFIG_SERVICEID

The GC_CONFIG_SERVICEID parameter sets the service ID associated with your Google Cloud Platform account. Typically, this is an email address.

GC_CONFIG_PROJECTID

The GC_CONFIG_PROJECTID parameter specifies the project number of your Google Cloud Platform account.

Note: This is not the Project ID in Google Cloud Platform.

GC_CONFIG_AUTHENTICATION_TYPE

The GC_CONFIG_AUTHENTICATION_TYPE parameter sets the authentication method to use. The default value is GC_SERVICE_AUTHENTICATION_JSON_SECRET, which uses the JSON private key method.

Use the following command to use the PKCS 12 file method:

googleConfigObject.setAttributeString(MediaTransportConfigurationGoogleCloud.GC_CONFIG_AUTHENTICATION_TYPE, MediaTransportConfigurationGoogleCloud.GC_SERVICE_AUTHENTICATION_PKCS12);

CG_CONFIG_SECRETACCESS_FILE

The GC_CONFIG_SECRETACCESS_FILE parameter specifies the full path location of the PKCS 12 file. 

Configure the MediaTransport listeners


The transport model is asynchronous by default, so two listeners are available to determine when object transports are requested and whether they were successful: IMediaTransportListener and IMediaTransportProviderListener.

IMediaTransportListener

The IMediaTransportListener provides two callback functions:

  • public void onTransportProviderAdd (IMediaTransportProvider thisProvider)
  • public void onTranportProviderRemove (IMediaTransportProvider thisProvider)

The following example logs the addition or removal of providers:

public class MediaTransportListener implements IMediaTransportListener
{
	public void onTransportProviderAdd(IMediaTransportProvider thisProvider)
	{
		getLogger().info("Adding Provider: "+thisProvider.getTransportProviderName());
	}

	public void onTransportProviderRemove(IMediaTransportProvider thisProvider)
	{
		getLogger().info("Removing Provider: "+thisProvider.getTransportProviderName());
	}
}

Use the following command to add the IMediaTransportListener to the transport media system:

mediaTransport.addTransportListener(new MediaTransportListener());

IMediaTransportProviderListener

The IMediaTransportProviderListener providers one callback function:

public void onTransportObjectStatus(int count, IMediaTransportResponse responseObject, String transportProviderName)

Each time a transport provider attempts to send an object, the call back function is called to access the status of the transport. In the callback function:

  • count is the current attempt count for the object being transported. The transport provider will attempt to transport the requested object up to the maximum number of retries set for the object. By default, each object's maximum retry value is 5.
  • responseObject is additional information about the transport attempt.
  • transportProviderName is the name of the transport provider used in the attempt to transport the object.

The following example checks the success of the responseObject. If successful, the response data and headers are displayed. If unsuccessful, the response data is displayed.

public class MediaTransportProviderListener implements IMediaTransportProviderListener
{
    public void onTransportObjectStatus(int count, IMediaTransportResponse responseObject, String transportProviderName)
    {
  getLogger().info("onTransportObjectStatus: count: "+count);
  getLogger().info("onTransportObjectStatus: transportProviderName: "+transportProviderName);
  getLogger().info("onTransportObjectStatus: success: "+responseObject.getSuccess());
  getLogger().info("onTransportObjectStatus: basic message: "+responseObject.getBasicMessage());

  byte[] responseCode = responseObject.getSuccessData(IMediaTransportResponse.RESPONSE_INDICATOR);
  if ( responseCode != null )
  {
    String thisResponseCode = new String(responseCode);
    getLogger().info("onTransportObjectStatus: Response code: "+thisResponseCode);
  }

  if ( responseObject.getSuccess() == true )
  {
    getLogger().info("onTransportObjectStatus: Path of object: "+responseObject.getPath());
    byte[] responseData = responseObject.getSuccessData(IMediaTransportResponse.RESPONSE_DATA);
    if ( responseData != null )
    {
      getLogger().info("onTransportObjectStatus: responseData Length: "+responseData.length);
      byte[] responseHeaders = responseObject.getSuccessData(IMediaTransportResponse.RESPONSE_HEADERS);
      if ( responseHeaders != null )
      {
      getLogger().info("onTransportObjectStatus: responseHeaders Length: "+responseHeaders.length);
      getLogger().info("onTransportObjectStatus: responseHeaders :"+new String(responseHeaders));
      }
    }
  }
  else
  {
    byte[] responseData = responseObject.getSuccessData(IMediaTransportResponse.RESPONSE_DATA);
    if ( responseData != null )
    {
      getLogger().info("onTransportObjectStatus: responseData Length: "+responseData.length);
      getLogger().info("onTransportObjectStatus: responseData :"+new String(responseData));
    }
  }
}

Use the following command to add the IMediaTransportProviderListener to the transport provider:

googleProvider.addProviderListener(new MediaTransportProviderListener());

Configure a Google Cloud Platform bucket


Create a bucket

The following example creates a bucket named asimpletestbucket in your account. Note that the XML is placed as the body of the object and the setPath is set to a forward slash (/).

Note: This example assumes that the transport provider has been initialized.

String bucketCreateCommand = "<CreateBucketConfiguration>";
bucketCreateCommand+= "<StorageClass>"+MediaTransportConfigurationGoogleCloud.GC_OBJECT_SERVICE_STORAGE_CLASS_MULTIREGIONAL+"</StorageClass>";
bucketCreateCommand+= "</CreateBucketConfiguration>";
MediaTransportDataObject transportDataObject = new MediaTransportDataObject();
transportDataObject.setOption(MediaTransportConfigurationGoogleCloud.GC_OBJECT_OPTION_SCOPE, MediaTransportConfigurationGoogleCloud.GC_SERVICE_SCOPE_READ_WRITE);
transportDataObject.setOption(MediaTransportConfigurationGoogleCloud.GC_OBJECT_OPTION_MD5_ENABLE, MediaTransportConfigurationGoogleCloud.GC_MD5_ENABLED_FALSE);
// This parameter overrides the default bucket set for the transport method. You can do this for each object if you wish.
transportDataObject.setOption(MediaTransportConfigurationGoogleCloud.GC_OBJECT_OPTION_BUCKET, "asimpletestbucketagain");
// You can also override the project id with
//transportDataObject.setOption(MediaTransportConfigurationGoogleCloud.GC_OBJECT_OPTION_PROJECT_ID,"a new project id");
transportDataObject.setPath("/");
transportDataObject.setData(bucketCreateCommand.getBytes());
transportDataObject.setCommand("PUT");
mediaTransport.addTransportDataObject(transportDataObject);

Upload content to a bucket

The following example transports an object to the destination path set. 

// We get the bytes for the file and display the size of the byte array.
// We assume the file contents are provided as a byte array.
byte[] fileBytes = getBytesFromFile(uploadFile);
getLogger().info("Filename: "+filename+" File: FileByte Count: "+fileBytes.length);
if ( fileBytes != null )
{
getLogger().info("Filename: "+filename+" UploadFile:  "+uploadFile);
	MediaTransportDataObject transportDataObject = new MediaTransportDataObject();
	transportDataObject.setPath("/"+filename);
	transportDataObject.setContentType("video/mp4");
	transportDataObject.setData(fileBytes);
	transportDataObject.setCommand("PUT");
	transportDataObject.setOption(MediaTransportConfigurationGoogleCloud.GC_OBJECT_OPTION_SCOPE, MediaTransportConfigurationGoogleCloud.GC_SERVICE_SCOPE_READ_WRITE);
	transportDataObject.setHeader(MediaTransportConfigurationGoogleCloud.GC_OBJECT_HEADER_STORAGE_CLASS, MediaTransportConfigurationGoogleCloud.GC_OBJECT_SERVICE_STORAGE_CLASS_MULTIREGIONAL);
	transportDataObject.setHeader(MediaTransportConfigurationGoogleCloud.GC_OBJECT_HEADER_ACCESS_LIST, MediaTransportConfigurationGoogleCloud.GC_OBJECT_SERVICE_ACL_PUBLIC_READ);

mediaTransport.addTransportDataObject(transportDataObject);
}

Delete a bucket

The following example deletes the bucket named asimpletestbucket from your account. Note that the setPath is set to  a forward slash (/).

Note: This example assumes that the transport provider has been initialized.

MediaTransportDataObject transportDataObject = new MediaTransportDataObject();
transportDataObject.setOption(MediaTransportConfigurationGoogleCloud.GC_OBJECT_OPTION_SCOPE, MediaTransportConfigurationGoogleCloud.GC_SERVICE_SCOPE_READ_WRITE);
transportDataObject.setOption(MediaTransportConfigurationGoogleCloud.GC_OBJECT_OPTION_BUCKET, "asimpletestbucket");
transportDataObject.setPath("/");
transportDataObject.setCommand("DELETE");
mediaTransport.addTransportDataObject(transportDataObject);

Configure the Google Cloud Platform transport object


Transport objects have two parts: options and headers. The options override default values set by the transport provider configuration, and headers provide additional information in the HTTP request.

Transport object options

The following static options are available for transport objects:

  • GC_OBJECT_OPTION_BUCKET - Use this option to set the bucket's name to something other than the name configured by the transport provider. This is useful when you want objects to go to different buckets.
  • GC_OBJECT_OPTION_PROJECTID - Use this option to change the project ID for the project you are accessing.

    Note: In most cases, a project ID isn't applicable to objects because credentials typically don't apply to more than one project.

  • GC_OBJECT_OPTION_SCOPE - Use this option to change the capabilities of the request. There are three scopes you can use:
    • MediaTransportConfigurationGoogleCloud.GC_SERVICE_SCOPE_READ_WRITE
    • MediaTransportConfigurationGoogleCloud.GC_SERVICE_SCOPE_READ_ONLY
    • MediaTransportConfigurationGoogleCloud.GC_SERVICE_SCOPE_FULL_CONTROL
    The following example demonstrates how to set the scope:
  • transportDataObject.setOption(MediaTransportConfigurationGoogleCloud.GC_OBJECT_OPTION_SCOPE, MediaTransportConfigurationGoogleCloud.GC_SERVICE_SCOPE_READ_WRITE);
    
  • GC_OBJECT_OPTION_MD5_ENABLE - Use this option to include or exclude the MD5 header in the request. This can be set to your preferred security policy, and is also useful when troubleshooting.

Transport object headers

Transport object headers enable you to add HTTP headers for specific objects. The following is a list of pre-defined headers:

  • GC_OBJECT_HEADER_STORAGE_CLASS
  • GC_OBJECT_HEADER_ACCESS_LIST
  • GC_OBJECT_HEADER_COPY_SOURCE
  • GC_OBJECT_HEADER_META_DIRECTIVE
  • GC_OBJECT_HEADER_COPY_SOURCE_IF_MATCH
  • GC_OBJECT_HEADER_COPY_SOURCE_IF_NONE_MATCH
  • GC_OBJECT_HEADER_COPY_SOURCE_IF_UNMODIFIED_SINCE
  • GC_OBJECT_HEADER_COPY_SOURCE_IF_MODIFIED_SINCE
  • GC_OBJECT_HEADER_SERVER_ENCRYPT
  • AMZ_OBJECT_HEADER_SERVER_ENCRYPT_KEY_SHA256

Note: You can directly add any HTTP header that you want to, and aren't restricted to the headers listed above.

When setting these headers, you must provide the correct value. For example, the following command demonstrates how to set the access list header:

transportDataObject.setHeader(MediaTransportConfigurationGoogleCloud.GC_OBJECT_HEADER_STORAGE_CLASS, MediaTransportConfigurationGoogleCloud.GC_OBJECT_SERVICE_STORAGE_CLASS_MULTIREGIONAL);

If you're having problems or want to discuss this article, post in our forum.