I write this post just to share the experience. This issue has consumed me about 2 weeks of investigation.
Some time ago we began to have many problems with Wowza freezing, and consuming too much CPU in only one core. We did not know it, but the problem is the java process running your java garbage collection routine.
I did some testing with the configuration recommended by Wowza (-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:newSize=512m), but after a few minutes the problem came back. So I did some tests with experimental option (-XX:+UnlockExperimentalVMOptions -XX:+UseG1GC) and it got better. But after a few hours the java process freezes again.
Unfortunately the documentation from oracle for garbage collection is not very complete. Or if it is, is very difficult to find. I found a very usefull video, that explains in detail how the GC process works, and also plenty of tips for tuning GC. And also why the 10GB limit makes sense for old GC algorithms.
But the coolest tip from video is: Use the G1 with the most updated java as possible, with as much memory you have available on the machine. So I did it. I configured the java process to use 100GB heap size (yes, one hundred gigabytes) and the jdk-7u5-x64. Now everything is working better
For example, one of our liverepeater-origin servers is running for 20 days without any problem. This server is origin for ~80 streams, and it is the source for ~20 server farms. This server receives ~150MB/s, and send ~800MB/s signals. Our CPU peak was 24%.
The server is a Dual Xeon 6-Core X5670@2.93GHz with 128GB of RAM.