With the Flash player if you set the NetStream.setBufferTime to zero, install the most recent patch from here (http://www.wowza.com/devbuild.html) and use the rtp-live-lowlatency stream type you will see the best latency out of Flash and Wowza Pro. With the Haivision Mako hardware H.264 encoders the latency is less then 1 second. If you are seeing numbers greater than this then it is most likely on the encoding side. If you set NetStream.setBufferTime to anything greater then 0 with H.264/AAC video the latency will jump up to 2-3 seconds. So do be sure you have set this property set correctly.
Charlie