Internal NAT table should be based off of source port. UDP packets will be addressed to the destination IP/port; the router records the source IP/port in the address translation table and replaces them with router IP and random port, which is then sent to Internet client. UDP is not stateful, so the scenario you're describing shouldn't matter. It seems you are not allowing new connections through the router.
C>S: (tell server to use client UDP ports)
S>C: (tell client to use server UDP ports starting at 6790 which is the default for Wowza)