Sounds like a Java garbage collection issue. Out of the box the Wowza configs are not tuned for JVM garbage collection and once the memory buffer fills you will have a 'stop the world' event as is purges the active memory (eden) and moved it to older memory (old) or dumps it completely.
The Wowza performance tuning guide will get you 80% of the way there. The rest of it really depends on your exact server config. For us we didn't need to use tenured memory space at all, so we shut it off and just used Eden and Old. This prevented any 'stop the world' memory scans and allowed us to run ~100 live HD cameras in extremely low latency. Your environment may be different.
To test and see if this is the issue, open up the JConsole app in your Wowza folder (Windows would be Start -> Programs -> Wowza -> JConsole). Select the instance that has the Wowza name in it. Head on over to the memory tab. Watch your video and watch the memory graph. Usually easiest if you set it to 5 minutes. If you see the stuttering around the same time that the JVM is dumping the memory, then you know it is a garbage collection issue.
For your reference, this is what we use in our setenv.bat file for Wowza:
-Xmx8000M -Xms8000M -XX:+UseConcMarkSweepGC -XX:NewRatio=1 -XX:SurvivorRatio=4 -Xss256k -XX:+AlwaysPreTouch -XX:+CMSScavengeBeforeRemark -XX:+UseParNewGC -XX:ParallelCMSThreads=4 -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=0 -XX:SurvivorRatio=4096