HLS over HTTPS Performance

[Doug]

So we’re working to migrate our internal livestreams from RTMP to HLS. I’ve successfully configured HLS and small events seemed to go off without any issues. When we had a larger event we started getting issues with the users browsers not getting the video chunks delivered in time from the edge nodes. I believe serving the video over HTTPS is playing a role in this issue, but it’s hard to troubleshoot.

Does anyone have any suggestions on how to go about testing and tuning Wowza (4.7.2) to handle serving HLS over HTTPS efficiently? All of the HLS configs I have are default out of box settings with the exception of cupertinoCalculateChunkIDBasedOnTimecode being set to true. Thanks

[Doug]

So I finally gave up on trying to configure Wowza to do SSL/TLS directly. The CPU overhead incurred is just too much for java. I switched to installing Apache HTTPD on each node and configuring it as a reverse proxy terminating SSL.

[Rose_Power-Wowza_Com]

Hi @Doug, are you using the default 10 second chunks for HLS? You can reduce those to three chunks of 2 seconds each and this is a huge reduction in the CPU load. Let me know please and we can continue to investigate the cause.

[Rose_Power-Wowza_Com]

Hi @Doug. I’d like to first point out you may want to consider upgrading your version of Engine to at least 4.7.6. We’ve made several improvements in our cupertino HLS streaming workflows to CDN endpoints.

Sounds like you’re using the default 10 second chunks for HLS? You can reduce those to three chunks of 2 seconds each and this is a huge reduction in the CPU load and obviously a significant reduction in latency. Let me know please and we can continue to investigate the cause.

Here are some docs on how to reduce the chuck size. Next, we can look at reducing the keyframe interval.

1. https://www.wowza.com/docs/how-to-configure-apple-hls-packetization-cupertinostreaming.

2. https://www.wowza.com/docs/how-to-improve-playback-of-lower-latency-apple-hls-streams

[Rose_Power-Wowza_Com]

Here you can read through some of the changes we have made in several Engine versions in the left nav. Again, several are Cupertino HLS improvements. We are now in version 4.7.7. Thanks.

https://www.wowza.com/docs/how-to-update-your-wowza-streaming-engine-installation

[Doug]

Hi Rose,

I did upgrade to the latest version, 4.7.7, and there was no improvements with the out of box configs. Note that I’m also forcing TLS 1.2 protocol too. Latency isn’t a big concern currently as we’re just trying to get away from flash based streams.

Our current ingestion model looks like: Transcoder RTMP --> Wowza Live Single Server/Origin node --> Stream Targets --> Wowza Live Single Server/Origin nodes. Would we need to make tall the HLS config changes match on all the nodes in the chain? Would there be any benefit to converting the end node apps to Live Edge?

[Rose_Power-Wowza_Com]

Hey @Doug, tech support suggests you may be able to use this property to make sure all your chunk numbers match up on your different edges.

https://www.wowza.com/docs/how-to-configure-apple-hls-packetization-cupertinostreaming#additional-properties-for-live-streams

Otherwise, they have requested you submit a support ticket so they can view all of your config files and can help you determine whether or not to convert the end nodes apps. It’s too hard for them to advise without running some tests. Thanks!

https://www.wowza.com/support/open-ticket

[Andrew_Ramberg]

Hello Doug,

There are a couple changes that can help with SSL performance on Java 8. See the following:
https://www.wowza.com/docs/how-to-improve-ssl-performance-with-java-8

SSL will add some additional CPU utilization, so if the information in the article above doesn’t fix the issues, it may be a CPU resource limitation. You can exclude some of the more resource intensive Cipher Suites and Protocols to limit the amount of CPU used with SSL. See the following article:
https://www.wowza.com/docs/how-to-improve-ssl-configuration

Best regards,
Andrew

[Doug]

Hi @Andrew Ramberg,

I’ve gone over those links multiple times and neither represents a noticeable improvement in performance.