Thank you for your reply, it was really helpful. And sorry for delaying with my answer.
I've found several bugs in our code that led to a thread lock and now everything works much better.
However, I was asked to recover Wowza whenever OutOfMemoryError occurs.
So I added -XX:OnOutOfMemoryError option to setenv.bat:
And it worked fine.
set JAVA_OPTS=%JAVA_OPTS% -XX:OnOutOfMemoryError="%WMSAPP_HOME%\bin\restart.bat"
But -XX:OnOutOfMemoryError option does not work when Wowza is run in service mode.
I added this line to WowzaMediaServer-Service.conf and checked Java Args in logs:
What am I missing here?
Why isn't -XX:OnOutOfMemoryError respected in service mode?