RTMP -> Wowza -> HLS | ABR Segments are drifting apart from one another

Brooke_McKim · September 9, 2020, 8:00pm

I have Flash Media Encoder Live setup to send 3 streams from the same source to Wowza. Wowza is then transmuxing this RTMP stream into HLS segments. However after about 48 hours the playlists of the 3 streams start to drift apart on what segment in the playlist they are on. I am currently using playlists with 8 ten second segments. This is problematic because when playing ABR HLS the player will try to switch to a higher/lower bitrate and if the segments are out of order it will error.

I have been running Wowza for about 6 months and have just started seeing this issue. We are currently sending the most streams we have ever used, but the server doesn’t seem to be bogged down.

Richard_Lanham · September 9, 2020, 8:24pm

Is this a Roku issue?

Richard

Richard_Lanham · September 9, 2020, 10:02pm

What version of Wowza are you using? What is the video source in FMLE?

If you stop then restart the stream is the problem resolved or the same?

Have you set this up?:

https://www.wowza.com/docs/how-to-debug-encoder-multi-bitrate-keyframe-alignment

Roku is involved tho, right. I am assuming from your helpful response here.

There seems to be a correlation (from your report) between this problem and increased amount of streaming, which might not be straining Wowza but may be straining the network, the connection between encoder and Wowza.

Richard

Richard_Lanham · September 9, 2020, 8:40pm

(support note: continues in ticket 11412)

Richard_Lanham · September 9, 2020, 8:42pm

Is there any problem with playback that you can see?

Richard

Ian_Beyer · September 9, 2020, 8:45pm

Thanks for the update - I’m seeing similar problems with Roku using Decklink/Kulabyte. I’ll test on latest and see if it’s fixed any.

Vels_L · September 9, 2020, 8:52pm

Robert,

Any resolutions on this issue ?

We also see audio/video sync issues (not often, but they happen) when playing streams on Roku.

RTMP streaming of the same streams is fine.

Brooke_McKim · September 9, 2020, 8:52pm

No. The segments coming out of the wowza server are the issue. The segments in the playlists should be aligned.

Brooke_McKim · September 9, 2020, 8:52pm

2.2.3 build26454

The video source is a BlackMagic card - component in. If I restart the segments will align but after a few hours will start to drift apart. When i first noticed it a few days had gone by and the one stream was over 2000 behind.

I will look into setting that up.

Yes the end device is a Roku, but I am also testing in quicktime.

Good point on the network traffic between the encoders and wowza server.

I will set that up and get back to this thread also after I take a look at the network traffic.

Brooke_McKim · September 9, 2020, 8:52pm

2011-05-24 12:08:52 EDT comment server INFO 200 - Key frame[lomalinda/definst/4-2]: dts:9949985 pts:19900004 - - - 3305.152 - -

2011-05-24 12:08:52 EDT comment server INFO 200 - Key frame[lomalinda/definst/4-1]: dts:9950019 pts:19900071 - - - 3305.202 - -

2011-05-24 12:08:52 EDT comment server INFO 200 - Key frame[lomalinda/definst/4-3]: dts:9949985 pts:19900004 - - - 3305.202 - -

2011-05-24 12:08:52 EDT comment server INFO 200 - Key frame[lomalinda/definst/4-2]: dts:9950019 pts:19900071 - - - 3305.203 - -

2011-05-24 12:08:52 EDT comment server INFO 200 - Key frame[lomalinda/definst/4-3]: dts:9950019 pts:19900071 - - - 3305.203

Does this indicate a problem or is this just a margin of error in the logger?

Brooke_McKim · September 9, 2020, 8:52pm

After upgrading to the newest Wowza version and have been running for a couple hours everything is looking good. Will update with any progress tomorrow.

Brooke_McKim · September 9, 2020, 8:52pm

Came in this morning to find that the segments have drifted apart again.

2011-05-25 09:03:06 EDT comment server INFO 200 - Key frame[lomalinda/definst/1-1]: dts:79286953 pts:91465076 - - - 78558.937 - -

2011-05-25 09:03:06 EDT comment server INFO 200 - Key frame[lomalinda/definst/1-2]: dts:79286953 pts:91465076 - - - 78558.938 - -

2011-05-25 09:03:06 EDT comment server INFO 200 - Key frame[lomalinda/definst/1-3]: dts:79286953 pts:91465076 - - - 78558.938

That looks correct, however:

1-1

#EXTM3U

#EXT-X-ALLOW-CACHE:NO

#EXT-X-TARGETDURATION:10

#EXT-X-MEDIA-SEQUENCE:15542

#EXTINF:5,

media_15542.ts?wowzasessionid=1194417792

#EXTINF:5,

media_15543.ts?wowzasessionid=1194417792

#EXTINF:5,

media_15544.ts?wowzasessionid=1194417792

#EXTINF:2,

media_15545.ts?wowzasessionid=1194417792

#EXTINF:7,

media_15546.ts?wowzasessionid=1194417792

#EXTINF:4,

media_15547.ts?wowzasessionid=1194417792

#EXTINF:5,

media_15548.ts?wowzasessionid=1194417792

#EXTINF:5,

media_15549.ts?wowzasessionid=1194417792

#EXTINF:5,

media_15550.ts?wowzasessionid=1194417792

#EXTINF:5,

media_15551.ts?wowzasessionid=1194417792

1-2

#EXTM3U

#EXT-X-ALLOW-CACHE:NO

#EXT-X-TARGETDURATION:10

#EXT-X-MEDIA-SEQUENCE:15529

#EXTINF:5,

media_15529.ts?wowzasessionid=1406374313

#EXTINF:5,

media_15530.ts?wowzasessionid=1406374313

#EXTINF:2,

media_15531.ts?wowzasessionid=1406374313

#EXTINF:7,

media_15532.ts?wowzasessionid=1406374313

#EXTINF:4,

media_15533.ts?wowzasessionid=1406374313

#EXTINF:5,

media_15534.ts?wowzasessionid=1406374313

#EXTINF:5,

media_15535.ts?wowzasessionid=1406374313

#EXTINF:5,

media_15536.ts?wowzasessionid=1406374313

#EXTINF:5,

media_15537.ts?wowzasessionid=1406374313

#EXTINF:5,

media_15538.ts?wowzasessionid=1406374313

1-3 was the same as 1-2

Brian_Osborne · September 9, 2020, 8:59pm

This lets you view segment drift of HLS streams: https://github.com/brookemckim/hlspider

I’m seeing this issue on Wowza 3 (the latest). Restarting wowza fixes the issue temporarily. Here is how bad it has become:

— Unaligned with segments : 48751, 48746, 48744 —

Are there any settings or anything I can try to fix the issue, or maybe diagnose what’s going on? If we can, we’d like to avoid transcoding.

Brian_Osborne · September 9, 2020, 8:59pm

Okay, even worse this morning.

— Unaligned with segments : 2178, 1980, 2092 —

The strange thing is, that if I play back any of the individual segments that claim to be lined up, like 2178, 1980, and 2092, they are indeed all showing the same source segments. Here’s the result of curling each of the sub playlists that are out of sync:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-ALLOW-CACHE:NO
#EXT-X-TARGETDURATION:15
#EXT-X-MEDIA-SEQUENCE:1990
#EXTINF:10,
media-b2564000_1990.ts?wowzasessionid=1049865870
#EXTINF:10,
media-b2564000_1991.ts?wowzasessionid=1049865870
#EXTINF:10,
media-b2564000_1992.ts?wowzasessionid=1049865870
#EXTINF:10,
media-b2564000_1993.ts?wowzasessionid=1049865870
#EXTINF:11,
media-b2564000_1994.ts?wowzasessionid=1049865870
#EXTINF:10,
media-b2564000_1995.ts?wowzasessionid=1049865870
#EXTINF:10,
media-b2564000_1996.ts?wowzasessionid=1049865870
#EXTINF:10,
media-b2564000_1997.ts?wowzasessionid=1049865870
#EXTINF:10,
media-b2564000_1998.ts?wowzasessionid=1049865870
#EXTINF:10,
media-b2564000_1999.ts?wowzasessionid=1049865870

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-ALLOW-CACHE:NO
#EXT-X-TARGETDURATION:15
#EXT-X-MEDIA-SEQUENCE:2102
#EXTINF:10,
media-b1064000_2102.ts?wowzasessionid=171596872
#EXTINF:10,
media-b1064000_2103.ts?wowzasessionid=171596872
#EXTINF:10,
media-b1064000_2104.ts?wowzasessionid=171596872
#EXTINF:10,
media-b1064000_2105.ts?wowzasessionid=171596872
#EXTINF:11,
media-b1064000_2106.ts?wowzasessionid=171596872
#EXTINF:10,
media-b1064000_2107.ts?wowzasessionid=171596872
#EXTINF:10,
media-b1064000_2108.ts?wowzasessionid=171596872
#EXTINF:10,
media-b1064000_2109.ts?wowzasessionid=171596872
#EXTINF:10,
media-b1064000_2110.ts?wowzasessionid=171596872
#EXTINF:10,
media-b1064000_2111.ts?wowzasessionid=171596872

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-ALLOW-CACHE:NO
#EXT-X-TARGETDURATION:15
#EXT-X-MEDIA-SEQUENCE:2188
#EXTINF:10,
media-b464000_2188.ts?wowzasessionid=748396612
#EXTINF:10,
media-b464000_2189.ts?wowzasessionid=748396612
#EXTINF:10,
media-b464000_2190.ts?wowzasessionid=748396612
#EXTINF:10,
media-b464000_2191.ts?wowzasessionid=748396612
#EXTINF:11,
media-b464000_2192.ts?wowzasessionid=748396612
#EXTINF:10,
media-b464000_2193.ts?wowzasessionid=748396612
#EXTINF:10,
media-b464000_2194.ts?wowzasessionid=748396612
#EXTINF:10,
media-b464000_2195.ts?wowzasessionid=748396612
#EXTINF:10,
media-b464000_2196.ts?wowzasessionid=748396612
#EXTINF:10,
media-b464000_2197.ts?wowzasessionid=748396612

These are the three different sub-playlists.

media-b2564000_1990.ts

media-b1064000_2102.ts

media-b464000_2188.ts

are all the same content. How is this possible? The master playlist plays out fine in Quicktime, but it doesn’t seem like it should.

Brian_Osborne · September 9, 2020, 8:59pm

No there is not any that I can see. I’m trying to get it on a player that will switch its bitrate, so I can test if the switching works or not. I can’t imagine it would based on the EXT-X-MEDIA-SEQUENCE number in each of the playlists

Brian_Osborne · September 9, 2020, 9:53pm

I tried adding the chunkBreakOnPTS value as described here: https://www.wowza.com/docs/how-to-debug-encoder-multi-bitrate-keyframe-alignment

After running for about 10 hours, there’s now a drift on one of the three playlists by 4 segments.

Any other ideas?

Wowza Community

RTMP -> Wowza -> HLS | ABR Segments are drifting apart from one another

Popular Video Topics

Video Resources

Partners

Company

Stay Connected

Stay Up to Date with the Blog