Develop Mobile Live-Streaming Apps with the Wowza GoCoder SDK
Learn how the Wowza GoCoder™ SDK can simplify and speed up your development of live-streaming apps for iOS and Android.
In this session we cover the following:
- New capabilities of the GoCoder SDK
- How the Wowza GoCoder SDK enables live mobile streaming through integrations with Wowza Streaming Engine™ and Wowza Streaming Cloud™
- Installing the SDK and integrating it into your mobile application
- Common GoCoder SDK-related tasks and topics
- GoCoder SDK examples
- Answers to your questions
Good afternoon, good evening, wherever you are. I want to thank you for joining us for today's webinar reviewing the GoCoder SDK and specifically its capabilities in terms of integrating and developing applications for mobile live streaming. My name is James Jackson. I'm the director of tech marketing here at Wowza Media Systems™. I'm joined today by Barry Owen, the VP of engineering.
Barry, to start off with, the primary benefit behind the GoCoder SDK is in terms of simplification of mobile app development for live streaming, we're providing an API for cross-platform capture. Is there anything you wanted to add to that?
Hi, everybody. This is Barry. Happy you could join us today. We created the GoCoder SDK because creating video encoding from a mobile application is hard. There is a lot of moving parts from the actual capture from the devices and/or whether it be the onboard camera or even external cameras to encoding that in a clean way through H.264 into AAC and then transporting that to your streaming server. There's a lot of moving parts, it can be very complicated, and we wanted to provide people a nice, clean, cross-platform way to be able to manage that whole process and really ease the development of mobile applications.
Thanks, Barry. There's one housekeeping thing. We are monitoring the questions today and in addition to Barry, we also have Rob Gabbard, the primary architect of the GoCoder SDK on with us, so please keep those questions coming as we go through the webinar.
We mentioned a few of these, but one of the key features of the SDK is the rapid integration of live streaming into your own application. We had our GoCoder app … and we did do white-label version of that and people wanted more. They wanted the ability to really build their own application and include our encoding technology and our streaming technology in the app. The SDK, simple to use, handles obviously Android and iOS, both Swift and Objective C, and supports the most current SDKs and technologies for both devices. It's optimized to connect to Wowza Streaming Engine, whether that be your own on-premises engine, or an engine you're running on EC2 or else running in the cloud, or also the Wowza Streaming Cloud service.
It provides additional things that will … allow you to truly optimize those connections and support some really interesting stuff. I'll get into some of that in the road map later. You can do very detailed configuration of your audio and video capture settings, all the way up to 4K. There's no reason it can't go higher once the devices support that. We can utilize external microphones, external cameras as long as those cameras have appropriate drivers to get the data onto the device so we can grab it. We've actually had customers just doing that, and it's working out quite well. Then, of course, any of the on-device settings, like exposure control, focus, flashlight on and off, front and back camera, things like that.
One of the newest things we've added to both Android and iOS is dynamically variable bitrate and frame rate encoding, based on your available bandwidth. This is something that we're kind of in the walk phase of now, where it does pretty basic bitrate reduction, if it sees that it's falling behind, or you can optimize for dropping frames. In the future, we're going to make this much more intelligent and you'll see some really, really kind of high-end features as far as variable bitrate and frame rate encoding QRS goes in the future.
Okay, so we have one question that came in regarding support for Windows Mobile in terms of our plans for doing that in the future.
We currently do not have support for Windows Mobile phones on our road map, just due to the sheer volume of the devices versus the other two doesn't justify the investment at this point.
Okay, thanks, Barry.
I want to highlight a few ... The SDK shipped in February of this year, it was very successful, thankfully, with lots of customers on the SDK with some really cool use cases, and some of these, I'll demo some home-brewed versions of some of these things for you later and even give you a glance of how simple it is to do it from the code perspective. We have people that are doing Periscope clones, if you will. Somebody is doing a Periscope app for weather, called Weatherscope, and you can actually go and broadcast yourself showing the lightning in your neighborhood or whatever you would like. People are streaming games, so kind of like a mobile Twitch, where you can embed this into your gaming app and stream to as many people you wish to watch that. Remote broadcasting, or what I'll call vanity broadcasting, where you can have people doing interviews of an athlete, where they give the athlete a phone. You throw a phone in his hand, it's got the app on it, and he's up and running. People just broadcasting for fun to whomever.
I mentioned the external camera support where maybe it's an action camera, or doc cam, where you want to take that feed and maybe that camera doesn't have LTE or cellular broadcast capabilities, but we'll allow you to route that through the SDK and use your phone's radio to stream that out. I have a company who's doing tons of sports streaming and highlights, so kids' soccer games, kids' baseball games, and they can also record those into clip extraction and all kinds of cool stuff for highlight reels for the kids' sports teams.
Lots of interesting stuff going on with embedded encoding and broadcasting. A lot of these devices are really Android devices under the hood, and a lot of them support cameras and encoders. You can actually put the SDK right on these devices, and turn your device into streaming-capable, whether that be a drone, or a doorbell with a camera in it or security, things like that, like a doc cam. Another one of the fun demos we'll show that Rob cooked up was this little facial recognition, Pokémon GO kind of augmented-reality outfits. It's strangely quite simple to do, but yet really fun and really interesting to play with. That's some of the interesting use cases we've seen to date, so I look forward to hearing about more.
A real quick review of just what you need to get started, and then we're going to actually walk through a few steps on what it takes to actually get up and running. For those of you familiar with iOS, it's just an SDK. It's a framework, and that's code speak. You literally just drop the framework into your app and you have access to all of it. It really couldn't be simpler than that, and in the same way, on Android, the library is a typical Android library. Put it in your Android Studio project and add it, and you're good to go. After you've done that, what do you need to do? Well, there's a few things you need to do. The initial step is the SDK does require a license. You get a license key from us. You can actually go to Wowza.com and get a trial license right away. You can even purchase the SDK there when you're ready.
Initialize it, then what you're going to do is you're going to set up your camera preview. You're going to configure your camera, you're going to specify your settings, which is, what's my server address? What's the application name? What's the stream name? Is there authentication I need to deal with? You can optionally define some callback methods to get notified when things happen. The SDK will notify you of many events while it's going on, whether those are just informational or they could indicate errors or something you need to deal with. Then you start the broadcast.
What we'll illustrate here is how simple this is, both in Objective C or Swift, for that matter, and on Android, and that the APIs are consistent. Obviously, the syntax is different for your languages, but the things you do get started are identical. Here, we're going to take our camera preview view and attach it to our UI view. In this case, we're going to set the camera up in that we want to make this camera continuously autofocus. Obviously, you can set spot focus and other methods that the individual platforms support. Now we're going to set out preset. In this case, we're going to set it to 720p. We're going to set up our host address and our stream name. Obviously, we're missing out on a few details here, so if there was config and things like that. Actually, if there's authentication and things like that involved, you'd want to address that here.
Then these are some of the callbacks. These are just to give you an idea of some of the information you would get back for status or errors during the course of a broadcast, and reflect that back into your app and you can let the user know that something's happened, something wasn't configured right. You could also get an authentication message here that says, "Authentication failed," things like that. Then, of course, at the end, you just stop. You start streaming and you stop streaming. It's relatively simple at that point. Now it will send your broadcast to your destination.
This is kind of a quick and dirty.... I know that's small and probably not super easy to see. This kind of chains all of those things together and shows you how simple this is to get up and running. In this case, Android and Java, but you can literally ... 20, 30 lines of code, embed it in your app, get it up and running and streaming. All these examples are available on GitHub, in our GitHub account, that you can actually just go and download, free to use, tweak, enhance, change, whatever you'd like.
Now to the fun part. We'll start off here with a few demos that show you some of the capabilities, and I will apologize in advance for the video quality you're about to see via GoToWebinar, because it's obviously not going to hit you at full frame rate, so it might be a little jumpy, but trust me, in real life, it looks great. First of all, what I want to show you is a sample app. This is included in the sample app on GitHub. This is an iOS app that streams games via ... Sorry, I'm going to go back. This is an iOS streaming app that this is going to stream ... essentially a demo app that Apple created for their SceneKit, which is their 3D gaming engine. We're going to pop over here real quick.
This is a live view of my phone here. This is showing the preview of the phone. There's a game running in the background, but we've overlaid a GoCoder sample app on top. It's going to grab these frames and broadcast them. I'm going to go ahead and hit Broadcast here. See that little indicator here, it's connected. Then I'm going to go over into Wowza Streaming Engine that I have running locally on our machine as well, and we're going to see that we have the stream showed up that's coming into Wowza. I have the playlist coming up here .... Then we see we have the live stream of the video there. This was literally ... I'm going to go ahead and stop that real quick, and jump back over here. This is the kind of relevant code for that, that you would have. Obviously, there's more code to it. You're welcome to go look at it in the sample app, but this is how simple it is to do that. We've created a callback for you that's a processed frame to send a GoCoder.
We grabbed our screen snapshot from this Scene View app. We've obviously set up GoCoder destination and things like that ahead of time, and at this point, we're really just grabbing a pixel buffer and then we're calling the GoCoder capture routine and pumping that pixel buffer into GoCoder. We can set our frame rate, obviously we can set our size and our height … and things like that. Then we just jam the frames in, and GoCoder grabs them and codes them and broadcasts them. Simple as that.
We're going to show another one here. This particular demo I'm going show you now actually, you can see on either end of your iOS. It's just going to take a local .mp4 file that we have, and it could be any file on your device, whether you pre-recorded it or whatnot. Some of the interesting applications of this are that if you've printed your app and somebody wants to stream but they don't have bandwidth, you can certainly record that locally and the SDK supports that as well, and then stream that recording later. You could interrupt recording potentially as bandwidth goes away, record parts and re-stream, things like that, so there's a lot of capabilities for this and allows some kind of some interesting offline uses.
Let's go to this one real quick here, then come back over here. Here we have a little video that's some of sort of little Red Bull jet kind of thing flying around, and we're going to get that guy started. You see the plane flying here. Go back to streaming engine and do a refresh. You'll see that my file is coming into the server. Then, we're going to go back here and we should be able to see this playing in our browser. Go! All right, never mind. I messed that one up, I'm going to play it here. We have our playback there. You'll notice in this view that there's actually this little overview here, this overlay, this Wowza Media Systems overlay. That happens to just be the watermark of the trial version of the SDK, or the evaluation version of the SDK, but it also highlights another capability in that you can actually have all the ability to manipulate frames of video prior to them being encoded.
That's going to lead us into our next demo, which is one of my favorites. Oh, sorry, missed a step.
We have one other question that came in. This is asking about the approximate latency to Wowza Streaming Cloud.
That's a good question. We do get asked that a lot. I'm going to talk about that even more when we get to the road map. Obviously, it's heavily dependent on whether you're using Wi-Fi, whether you're using LTE, your resolution, your bit rate, all kinds of stuff. In general, over a good Wi-Fi network, the latency between the GoCoder SDK and the streaming engine is no more than a second or two. It can be very quick. Certainly on a Wi-Fi network to a local server, it can be even less, and then when I stream at home or when I stream in this environment I'm streaming in here, which is off of my laptop, I can regularly get 1 second or slightly higher latency from encode to playback.
Okay. Thank you.
Just real quick, I kind of skipped this real quick, but this is the .mp4 streaming example in actually Swift code here, and what we're doing here is we're leveraging the asset reader class on iOS which is giving us frames as it reads an .mp4 file, and then we're doing the very same thing we did before, which you can see here, is we're calling the GoCoder routine that takes that image, buffers that pixel buffer and sends it off to be encoded and then streamed. Again, very simple, very straightforward. Lots of the heavy lifting is done behind the scenes.
That takes us to our next demo, which is, again, that really interesting case about ... Obviously Pokémon GO is huge in the world, augmented reality and all that stuff. This shows a couple different things. It shows how easy it is to manipulate the capture buffer and add those kinds of things, like add a Pokémon. In this case, we're going to add a Wowza ninja. All the heavy lifting is done behind the scenes in the SDK by a class called WZ Bitmap in this case, and it's handling the compositing and all this other stuff for you. What this also shows is, we don't hide any of the access to the native APIs for you. You can access the native APIs, and in this case what we do is we use the native face detection app, API of Android, to actually locate a face on screen, and overlay a logo on top of it.
Let me show you that one real quick here. What we're going to see ... Flip the camera. You see me. No, you see a ninja. You can see, it will follow me around a little bit, kind of fun. We'll go ahead and stream that. Now we can go back to Wowza Streaming Engine. We're going to go back to our incoming streams page, and then we can see the live view of the same thing. This is a fun little app, and I have to say thanks to Rob's work as far as abstracting out this stuff in this SDK, it was kind of ridiculously easy to put together. We'll go back over to our code here. This is a little bit of Java code. We're going to get our face-detection stuff in here, and then over here, we're going to set the face in the bitmap object, and the GoCoder SDK is going to take care of the rest. It's going to do the compositing for you, and it's going to stream that out.
Those are just three of the use cases you can see for the current version of the SDK. I encourage you to download and play with the sample apps. There's lots of stuff going on. The Android one can do a lot more than I showed. The iOS has several different implementations, both in Swift and Objective C. You can learn a lot about the SDK by sort of running through those. That's kind of what happens currently, so I wanted to talk a little bit about what are we doing next.
Almost since the first day we released the SDK, the first question we had was, "Great, well, when can I play back in the SDK?" We've listened, and we are integrating a playback functionality within the SDK. What this will allow you to do is have a single app that can both stream live video, but also consume and play back live video, whether that be both at the same time on a screen so that you could have someone broadcasting and someone perhaps giving them direction or talking to them. It could be an audio-only feed. This should be, end to end. ... Again, it's highly dependent on your connectivity, but we're targeting sub-2-or-3-second latency between them. Certainly not Facetime light. That's not really the point yet, but it does allow the capability for you to build, in a single application, a full end-to-end app.
Some of the other things we're going to do are create APIs to do enhanced metadata back and forth, bidirectional events in the stream. Things like, you could have a button on your screen when you're broadcasting, and you could click it and it would send an event to the server, and the server would react to that and send an event back to the application, that you could then respond to. This could be cues, it could be even something as interesting as, "Hey, I'm going to hit this button and I'm going to tell you stick an ad in the stream, and you're going to do that," and the resulting stream would be split into ad breaks. You could send someone's location up to the streaming engine and the streaming engines could send something back that says, "Hey, that's great, now walk 50 yards and turn left, and take a picture of that for me, or stream that for me." There's a lot of interesting capabilities once you can send this custom meta data back and forth, and be it custom modules on the server or some of the built-in methods of the server. You can get some interesting responses to those.
We're working with different vendors to do much tighter integration for external audio and video sources, whether that be a camera on a drone or a Bluetooth microphone, or things like that, or potentially just embeddable SDKs for some of these other platforms that people are creating action cameras out of or things that I mentioned before like doorbells and drones and things like that. If you're following Apple in their WWDC announcements, you'll note that one of the big things they announced at WWDC this year was Replay Kit, which allows applications that know how to encode and stream to provide those services to other applications, so you could create a game streamer app that allows people to subscribe to it and stream from external applications via your app. That opens a lot of interesting possibilities for all kinds of things, for screen testing and game streaming and things like that. It allows the game developers to not have to worry about the encoding and streaming aspects. They can just use a service that exists on the phone already through your app. I think that's going to be pretty interesting.
Last, but not least, is how do you get it? There's a free trial available. Go to Wowza.com, you can grab it. You do need a bundle ID for an app. It doesn't have to be your final bundle ID, but it will generate a key for you. You get a download, and finally, at long last, you can actually buy it online too, if you're ready. If you choose not to talk to a sales guy, you don't have to. You can get it all done online. I do want to point out one other thing, is that we are in the middle of a contest called the GoCoder SDK App Challenge.
Absolutely, so right now we started at Streaming Media East. We will be concluding and announcing our winners at Streaming Media West. Up for grabs at this point in time. For the app are a $5,000 cash prize as well as a license for the GoCoder SDK, so obviously great and so we're really looking for some innovative things, and as Barry mentioned in some of the examples, we are seeing some innovative things in terms of screen capture, the ability to leverage a lot of the embedded services and functions within the actual devices, but also some interesting things in terms of the Internet of Things world, in terms of being able to initiate other services, being able to take those video streams to perform things in the security space as well as just in a lot of other areas. Definitely some really, really interesting things, and we're really excited about the possibilities. I heard some really great things going on.
Let's see. There's one other question that came in. Is there any integration for Ionic framework, or do you know of any resources for this?
What's the framework?
I'm not familiar with that one, per se. I do know we have people who have successfully used Titanium, and I believe someone is using Xamarin as well. We don't have adapters or wrappers for those frameworks ourselves. We have thought about it, and they are on the road map, so you can probably expect them in the future. I do not know of anybody using that particular framework at this time.
Okay, one other question that came in, this is more of a business-related question. Is there an entry fee for the SDK, or is that included as piece of the Wowza Streaming Engine licensing fees? SDK is an additional license. You can request … a trial license for the SDK. Those are available at this point in time, but in terms of actual permanent license, yes, there is an actual fee associated with that. Certainly, you can go online and request that as well. We can get you in contact with the sales team as well to get the actual license. That is something that's available ... To answer your question, yes, there is an additional fee for the SDK license.
The model for the SDK is, there's a fee. It includes an app, you can release an app for both iOS and Android and also includes maintenance and support. There is a fee, but you're getting a lot for it.
Okay. Barry, just to recap or talk a little bit, so obviously we went through some great demos there, but in terms of actually accelerating and the development life cycle and reducing your risks, Rob and the dev team here, engineering team has done a great job of packaging things up. Obviously there's a lot of flexibility in terms of what you can do, but also in terms of actually leveraging those resources to really accelerate and then also really onboard you very quickly to mobile live streaming.
Not only onboard you quickly, but you're essentially protecting your investment. We're going to continue to innovate with this SDK and keep up. We're adding features for iOS 10, we're adding features for the latest version of Android, and we'll continue to do that. As the platforms improve and give us more capability, we'll not only keep up with those but we'll keep adding enhanced and extended features. Things like that are further down the road map that I didn't really talk about, but things like being able to really control Wowza Streaming Engine or Wowza Streaming Cloud via their REST APIs from the SDK, and having wrappers and macros, and things like that. From your SDK, hey, I want to spin up a server in the cloud. Great, call this call, it'll happen. There's an awful lot of stuff we can still do, and we have a team of people dedicated to keeping this up to the latest and greatest, which is something that you shouldn't have to do to build your own application.
Very, very cool. One other question had come in … [about the] ability to use GoCoder with other media services, essentially not with Wowza Streaming Engine or Wowza Streaming Cloud?
Currently, at this time, GoCoder uses a proprietary protocol called WOWZ to communicate between Wowza Streaming Engine and/or Wowza Streaming Cloud and the app. At this point in time, it doesn't do standard RTMP, it doesn't do RTSP. We're constantly evaluating that to determine if we would enable that. We know how, we have the technology, it's really just a question of if we want to open this up to other media servers or not.
Okay. On that same point, though, obviously somebody using GoCoder as they're capturing and encoding, one of the great things we have in the Wowza platform is that we'll let you syndicate globally in terms of actually, take that stream and deliver it to multiple destinations, multiple targets, and so that's definitely one of the advantages we have there as well is to make sure that we can deliver to those sources. As you mentioned before in terms of keeping abreast, obviously we're tying in to those sources, and we'll continue to do so. Not only to the source, but also those targets to make sure that we will continue to make sure that you are able to reach out and deliver to those targets.
Not only that, but you can easily stream from Wowza Streaming Engine or Wowza Streaming Cloud, we can go to Facebook, we can go to YouTube, we can go to CDNs and many other places. One of the interesting things is some of the other companies that run these services that are live-streaming services, chances are a lot of those services are using Wowza under the hood, so depending on the service, it may be possible to stream directly to them. You're just going to have to work with them to figure out what the entry point would be.
Okay. Very, very cool. Just let me see if we have any other questions. Please, if you have any other questions, we'd love to hear those. Obviously, it's not a one-time deal, so please we want you to go on and join the developer community as well, and look out and submit those questions there as well. We'll certainly stay here for a little while longer to take in a couple of other questions if anybody has any.
We'll make this deck available for you, but there's a lot of useful links in it, but if you look on the page that's up now, obviously you can go to our GitHub accounts for sample code. You can go to our forums and ask questions. We do have a full set of documentation on the SDK with some great resources and full API documentation. You can certainly hit us up on Twitter. We have a Slack channel that you can join, and unfortunately I don't have the information on how you can join that, but James can certainly get that to you. That's a place where folks can go, it's a live streaming forum on Slack where you can have areas of interest like mobile or engineer cloud or things like that.
Yes, absolutely. We'll make that available, and that information available as well, as well as putting that up on our developers page. One more question that came in regarding the ability to support end-to-end encryption with GoCoder. Is that possible, currently?
Technically, no. I'm curious if you want to encrypt the content itself, or just the transport of the content. There is a version of the WOWZ protocol called WOWZ-S that supports encrypted transport, aka HDPS, very similar to that, that we will implement in the future. I do not know, if you encrypted your content somehow before it was encoded, and then knew how decode that after it was decoded on Wowza Streaming Engine, with a custom module, perhaps? We don't have any kind of DRM support or any kind of encryption at this point.
Okay. That question is around the video itself. Just in terms of actually encrypting that, so you're saying that if you encrypted that video ...
You have to do some lifting on your part. I'm trying to figure out a way to do that. Shoot us an email and see if we can noodle on that one a little bit and come up with something that might work. I'm kind of thinking there may be a way to use ADS 128 to do something.
Okay. Thank you. Okay, so we'll stay here for a little bit longer to see if there are any more questions that come in. Once again, we want to thank everybody for joining us today. Obviously, we're very excited about it. Please visit us and take a look, for the GoCoder SDK contest that's currently in place …, we're really looking for other people to come in with really new and innovative ideas and so please go out and take a look there at the URL we've provided there for you. We are still accepting people to join that contest as well, and we will be announcing the winners of that at Streaming Media West coming up here in November.
We're not seeing any other questions coming in at this point in time, but once again, we want to thank everybody for joining us. Thanks, Barry, we really, really appreciate it. Great demos. We will be making the recording of this webinar available to everybody who registered as well as those attending, and we will make the information available to you as well in terms of what's included in the deck and those resources that Barry talked about in terms of Git repo on GitHub.com, as well as the information here in terms of actually joining our Slack channel and visiting us on Stack Overflow as well.
Cool. Thanks, James, and thanks, everybody, for listening. I appreciate it.