Hmm, I’m not sure what else to try. Would you recommend a larger heap size for that size of instance? I’m not even sure that memory is the issue, I just know that the CPU usage spikes to 100% after the same amount of time and Wowza stops responding until restarted.
I’m pretty sure it can’t be 3 as I’ve identified it on three different servers, each running for the same amount of time before they stop responding and they’ve mainly been idle, so it seems likely to be 1. I’ll take a heap dump from a server that’s been running for a week.
Hi Helder,
I did fix the problem in the end, but I never exactly identified the problem, unfortunately. I fixed it by ensuring all my database connections and queries were cached, as I was lead to believe that in certain instances they were re-created each time the updates were occurring.
Kind regards
One thing you can do in testing, is increase the frequency of updates to expose any leaks more quickly. Good luck!
The code is running on Wowza 2, so I’m pretty sure if we were using one it would’ve been the ‘old one’. I haven’t looked at it in about 2 years though as it’s been stable for a long time now!
MSR123,
Thanks so much for the very quick response! We’ll go ahead and perform a thorough review of our custom module code, to see whether we can fix it.
Thanks to your input, I’m even more leaned towards believing that the problem is indeed in the custom module.
I’ll let you know how it all goes.
Thanks!
Helder
Will do that. Thanks for the tip.
If you don’t mind, there is one extra quick question I’d like to ask: are you using the default Garbage Collector or are you using the “new one” (-XX:+UnlockExperimentalVMOptions -XX:+UseG1GC)?
Not that it would actually fix anything… but could provide some extra level of optimization, after we fix the module.
Thanks for your help! I really appreciate i
Ok. Thank you. We’ll hopefully have this fixed asap as well!
MSR123,
Sorry. Me again. I was looking at the solution you described and noticed that you mentioned you “fixed it by ensuring all database connections and queries were cached”.
I have 2 questions:
-
I suppose you’re using some sort of “connection pooling” mechanism (via C3P0, DBCP, etc). Can you tell me which one you’re using? We’re thinking about using this approach and knowing what works can definitely be helpful.
-
You also mentioned that your module was recording something onto the database every 10 seconds. Was that done per user (every connected user would insert/read the database) or once per server every 10 seconds?
Thanks! I do think I’ll have no further questions to you now. Sorry for inconvenience!
Helder
