I've been chasing this issue and made a little progress to narrow down the scope. I originally thought it was an audio transcoding issue so I posted it in that subforum:
http://www.wowza.com/forums/showthre...ream-To-Freeze but now I don't believe that is the case.

The issue: Source streams are from browser (all types) based Flash player using H.264/Speex. If you mute the audio at the source either by detaching the microphone object from the Netstream or setting the silence level to 100, the video playing on the viewer side will freeze when there is no buffer. This occurs in both a Flash based viewer of the RTMP stream with Netstream.bufferTime = 0 or in the HTML5 video tag playing the m3u8 file (where HTML doesn't provide any means to set a buffer on a live stream). If I set NetStream.bufferTime=1 the Flash Player behavior is as expected. The test was more to show that playing a stream with no buffer available freezes the video if the audio is paused as well.

When the audio is enabled (unmuted), the video stream starts playing again. If a person were to start viewing the stream WHILE the audio was muted, it would work fine. The problem occurs when a person is already viewing the stream and the audio stops being broadcast does it seem to freeze the video.

The issue occurs on v3.6 or v4.x of Wowza with pretty much stock install. Encoding using the audioonly template. But again since it happens on both the RTMP stream and HLS stream I'm not sure where the issue lies. I don't see any errors in the log files related to transcoding the audio.

Is there some buffering setting on the server I could set up on the stream that would help this?

Any other ideas would be greatly appreciated! Thanks!