We've been running an origin/edge config successfully for two years, re-streaming IP network cameras with millions of views per year.
In the last two months, we moved our edge servers to new DC's and upgraded to v3.5 . No problems. Origin still on v3.1.0 build1410 .
Yesterday we moved our origin server to a different DC and installed v3.5.
In old and new setups, origin also runs a local edge config.
The new all-3.5 cluster would not reliably stream RTMP to Flash clients. We use Flowplayer v3.2.14 with their RTMP plug-in v3.2.11 ... nothing changed in that config.
RTMP streaming was equally unreliable whether we connected via the remote edge servers or via the edge local to the origin, tending to discount the new paths between edge and origin DC's. HLS streaming to Apple devices (from the edge on the origin box) was working fine... in fact, better than before!
Unreliable = maybe 1 in 4 connection attempts would yield video, the others would just give us a black screen, even though Flowplayer thought the stream started.
Noticed that an
all v3.5 config triggers your new WOWZ protocol ... from your 3.5 manual:
The WOWZ(tm) protocol is a new TCP-based messaging protocol in Wowza Media Server 3.5 and is used for server-to-server communication. It's enabled by default. If one of the Wowza Media Servers in the origin/edge configuration isn't running Wowza Media Server 3.5, an RTMP connection will be established between the servers instead.
The only error message of note is the one we reported two years ago but with no solution...
NetConnectionConnection.connect: Failed to connect[localhost:1935]: org.apache.mina.common.RuntimeIOException: java.net.SocketException: Invalid argument: no further information
Java+Windows socket issue....
We still see this on our new edge servers (64-bit Windows Server 2008 + 64-bit Java 1.7.0_09) and the above on our new origin (32-bit Windows Server 2008 + 32-bit Java 1.7.0_10). First time we have seen it on a 32-bit WinOS and first time on a loopback local connection.
Random connect failures could explain it, especially if your new WOWZ protocol is using different code paths, timing, call parameters or whatever else triggers this issue with Java.
I'm contemplating adding a v3.1 install to the new origin box to try and remove the number of variables in this move.
The only way we got around the SocketException errors for the last two years was to run all 32-bit WinOS + 32-bit Java ... never saw the problem again. Now we need bigger servers and greater than 4GB memory space.... hence 64-bit . Hoped that sticking with 32-bit on the new origin would dodge that bullet but apparently not.
Any other suggestions or insights given the above symptoms?