Wowza Logo Wowza Icon
Wowza Video Sign Out
Talk to an Expert Free Trial
Wowza Video Sign Out

Wowza Community

Number of Applications affect to performance or stability?

Wowza Streaming Engine
#1

Hi, I have been using Wowza for 5 years, but recently I have a big problem with the stability.

Since I installed the last Wowza update (4.7.3.01) every 24 hours I need to restart it or crash.

I have Linux Centos 7.0, 32G RAM, java version 1.8.0_91 and Wowza always work fine until now, I don´t know if number of applications can affect to stability or need of ram, if is that the 4.7.3 have problems or what.

This is the error that I can see in wowza error logs when the Wowza is not working fine.

ERRORservercomment2017-12-2218:22:30-----50287.652--------IdleWorkerUtil: run.work : java.lang.NullPointerException|at com.wowza.wms.stream.livepacketizer.LiveStreamPacketizerBase.shutdown(LiveStreamPacketizerBase.java:271)|at com.wowza.wms.stream.MediaStreamMap.removeLiveStreamPacketizer(MediaStreamMap.java:1270)|at com.wowza.wms.stream.livepacketizer.LiveStreamPacketizerBase.checkTimeout(LiveStreamPacketizerBase.java:325)|at com.wowza.wms.stream.livepacketizer.LiveStreamPacketizerBase.onIdle(LiveStreamPacketizerBase.java:261)|at com.wowza.wms.util.IdleWorkerUtil.run(IdleWorkerUtil.java:111)|

Sorry about my English

#2

Hello,

I see a ticket for this issue has been opened with support. It looks like the issue is currently being investigated further.

Regards,

Andy E

#3

We are experiencing this issue as well after moving to 4.7.3.01. The null pointer error is thrown after many error entries pertaining to missing keyframes for the Cupertino packetizer, as exemplified in the log entry below:

A key frame has not been received in this audio and video stream for: 7990.954 seconds, which is over the max chunk duration of: 30.0 seconds. Restarting packetizer for stream: xxxxxxxx

After many entries of the keyframe error, the null pointer is thrown as exemplified in the log entry below:

ERRORservercomment2018-01-1500:00:00-----98435.219--------IdleWorkerUtil: run.work : java.lang.NullPointerException|at com.wowza.wms.stream.livepacketizer.LiveStreamPacketizerBase.shutdown(LiveStreamPacketizerBase.java:271)|at com.wowza.wms.stream.MediaStreamMap.removeLiveStreamPacketizer(MediaStreamMap.java:1270)|at com.wowza.wms.stream.livepacketizer.LiveStreamPacketizerBase.checkTimeout(LiveStreamPacketizerBase.java:325)|at com.wowza.wms.stream.livepacketizer.LiveStreamPacketizerBase.onIdle(LiveStreamPacketizerBase.java:261)|at com.wowza.wms.util.IdleWorkerUtil.run(IdleWorkerUtil.java:111)|

Shortly after the server begins to fail (REST API, cupertino packetizer for all streams, then WSE becomes unresponsive. Only restarting WSE resolves the issue temporarily. Both the CPU and memory spike before the server fails (screenshots below:

While we understand that a malformed stream that is missing keyframes can cause issues, it’s strange that these streams (which have been publishing for years), are now causing the server to fail.

Once support responds with their findings, I’ll update this entry.

#4

Probably not related to your stream issue but we run the same version of WSE on Windows, and it’s been quite stable with just one crash of the monitoring service since installation. We have 9 live streaming apps of which most are active 24/7.

#5

Hi, we have the same issue. The stream run over Wirecast 7 and 8, and we try ‘Main Concept’ and ‘x264’ Encoder. The framerate and keyframes are normal… and so on. All work fine in the past.
We see this problem on 4.7.3.0 and 4.7.3.01

#6

A second WSE 4.7.3.0 (later 4.7.3.01) crashed serveral times in follow of this issue. This server stream 24x7 10 streams parallel. All are send by some hardware encoders. And over weeks the failiure appears again and again. Every time after some hours of streaming, but ever the same stream.
It spam the logs and after some hours WSE > Java crashed with out of memory (10 gig ram max for Java…the server have much more).
After restart the app, the failure is gone. But it comes back after some hours or sometimes some days of streaming.

#7

Do you have a solution for this issue?

#8

The problem seams to be corrected with WSE 4.7.4:
‘Changed Apple HLS (Cupertino) chunk overflow logic so chunks are forcibly ended when they get too long, such as when keyframes stop being received’

#9

I have again the same problem,

A key frame has not been received in this audio and video stream for: 3086.636 seconds, which is over the max chunk duration of: 50.0 seconds. Restarting packetizer for stream

After a few hours with that warning the WSE crashes.

Tonight I´ll update from 4.7.3.01 to 4.7.4 I hope that it fixes the problem.

#10

We have the same problem on version 4.7.4

Huge amount of

ERRORservercomment2018-02-0800:00:00-----190157.542--------IdleWorkerUtil: run.work : java.lang.NullPointerException|at com.wowza.wms.stream.livepacketizer.LiveStreamPacketizerBase.shutdown(LiveStreamPacketizerBase.java:271)|at com.wowza.wms.stream.MediaStreamMap.removeLiveStreamPacketizer(MediaStreamMap.java:1270)|at com.wowza.wms.stream.livepacketizer.LiveStreamPacketizerBase.checkTimeout(LiveStreamPacketizerBase.java:325)|at com.wowza.wms.stream.livepacketizer.LiveStreamPacketizerBase.onIdle(LiveStreamPacketizerBase.java:261)|at com.wowza.wms.util.IdleWorkerUtil.run(IdleWorkerUtil.java:111)|

in Error log, mixed with something like

WARNservercomment2018-02-0800:00:01-----190158.618--------MPEGDashWriterHandler.endChunk[live/definst/channel_6]: Stream start time calculation is off by 12443ms from original value. Changing stream start date from “Mon Feb 05 19:11:23 MSK 2018” to “Mon Feb 05 19:11:35 MSK 2018”

then (after some hours of repeating those strings in log) WSE hangs and doesn’t answer. Only manual Restart solves this, but problem may return in any moment.

We had it on 4.7.3.01 at 2018/02/08 and on 4.7.4 at 2018/02/14

#11

We have exactly the same behavior. Using WSE 4.7.4 with java version “1.8.0_60”

Are there any solutions till now??!

#12

Same problem here, and we’re running 4.7.7 latest version with 1.8.0_77 Java version.

Any kind of fix ??

#13

ello did anyone get a solution to this problem?

#14

Because this post is outdated and we’ve had numerous upgrades and Engine releases since then. you’d need to submit a support ticket. No debugging can be done in the forums without full review of your config and logs. Are you on the latest version of Engine or at least using Java9?

As far as the null error, that can be thrown for several reasons from a configuration error to the server needing to be tuned, so for for accurate troubleshooting, we need a support ticket. I can tell you that several customers were able to resolve this by updating the drivers for the NVIDIA cards, so we suggest that you try this first. Many users see performance improvements after updating.

UPDATE: Also be aware that we have switched from Main Concept to Beamr so you may see improvements if you upgrade to 4.8.14.

https://www.wowza.com/docs/wowza-streaming-engine-4-8-14-release-notes

© 2007–2024 Wowza Media Systems™, LLC. All rights reserved.   Security & Privacy PolicyLegalSystem Status