Hi,
Yesterday we upgraded from 3.0.5 to 3.1.1. This seemed to go fine, until I noticed Wowza started overwriting old DVR recordings for some (not all) streams (window size for all is 2592000; 30 days):
May 1 00:36 0000_00_00
May 1 00:46 0000_10_00
May 1 00:50 0000_20_00
Apr 12 09:18 0000_30_00
Apr 12 09:28 0000_40_00
Apr 12 09:38 0000_50_00
Also the server started using 100% CPU for a couple of minutes. At one time during the 100% CPU use I restarted the server to make a small configuration change that I forgot to make earlier. Might this have something to do with the corruption?
Also we’re seeing a LOT of these in the error log:
LiveStreamDvrRecorder.endChunk[livedvr/definst/oODzpwg2bQ]: Recalculating duration by estimating. Was 4153. Is now 3827.
And also quite some of these:
DvrPacketHandler.handleHolder(): Skipping holder that cannot be re-aligned. Holder: {DvrPacketHolder: type:V pt:2332595 utc:1335872439639 dur:1430 (KEY, ) codec:7 numPackets:0 dataLen:18448 enc.n:0 pkt:{{DvrAMFPacket: size:18448, type:9, src:1, seq:0, absTimecode: 2332595, timecode:1430, utcTc:1335872439639}}}
In our use case the source sometimes drops out (once every few hours it drops out for 30 secs or so; it’s a low bitrate IPcam without audio in this particular case; think 160 kbit/s). Some extra info about this case:
true
true
false
600
-1
-1
Also, I notifed that IF one DVR-stream would get corrupted, all kinds of unpredictable things will happen and even the live feed won’t play anymore. No errors are logged if this happens.
So, my questions:
-
Do you have any tips, do’s/don’ts to prevent DVR archive corruption?
-
And are you planning on extra measures to stabilize the way Wowza handles restarts/DVR in general? It seems to be very easy to corrupt the DVR archive with a simple restart or especially a wowza upgrade. I hope you can make the DVR addon more robust as we plan on running a couple 100 of these low bitrate streams per server.
-
Recordings with 2592000 window size and 600 chunk grouping means Wowza will create 4320 directories per DVR per stream, or 432000 directories for 100 streams(!). Would it be wise to increase chunk grouping? What are the pros and cons?
Hope you’ve got some answers and tips for us!