Hello,
I am not adept at java or what wowza tuning options will affect what, so please excuse any tangents or minconceptions.
I am seeing similar behavior on our server. We are running Wowza 4.0.1 on Debian 7 with OpenJDK 1.7. I noticed that on 4.0.0 the heap and the memory would exponentially grow for a few hours then suddenly the heap would drop to normal levels and the memory usage would level off. The exponential growth portion was curbed in 4.0.1, but I am seeing a slow, linear increase in what I assume to be mostly reserved memory usage. Also, I have tried both GC options and not noticed a difference in the memory usage. Unfortunately our server is used for other applications outside Wowza and we can’t have it reserving 90% of the memory and forcing other applications to use swap.
One thing to check is the media cache. I noticed on a fresh install of Wowa 4.0.* the media cache is turned on. That ate up 5+ GB in < 8 hours and is used for VOD, not live streaming.
Also, I am using OpenJDK from the debian repos. Are you building your java from the oracle source code or downloading OpenJDK from the debian repos? From the research I have done there shouldn’t be much, if any, difference between the two. That being said if there is a difference then that might cause a problem.
Have you tried the different GC options? Any difference in bahavior or memory usage?
-Michael