Results 1 to 10 of 10

Thread: WowzaMediaServer too much use memories.

  1. #1
    Join Date
    Dec 2012
    Posts
    7

    Default WowzaMediaServer too much use memory

    Version: Wowza Media Server 2 Perpetual 2.2.4.07 build27628

    OS: CentOS release 5.7 (Final) 64bit
    RAM: 8GB
    Java version: 1.6.0_20
    Java options (WowzaMediaSever's setenv): -Xmx5000M -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+UseParNewGC -XX:NewSize=256m -Djava.net.preferIPv4Stack=true

    The media server too much use memories. ( CPU total usage is usually under 12%)
    This media server used only live streaming and VOD server is seperated.

    I set a FMLE(windows 7 64bit) PC and FMLE sends stream(RTMP, H.264, AAC) to WowzaMediaServer.
    At first, there was no problem. But there is freezing 1 or 2 days after.(memory usage is over 75%, the concurrent users are under 50. )
    Then, decoder clients can't play any streams.(RTSP, RTMP, HLS)
    That's why I restart WowzaMediaServer almost every 2 days.

    I tested many times to solve this problem as performance tuning guide. (http://www.wowza.com/forums/content....ormance-Tuning)
    The test has changed options.
    But there still has same problem.

    Is there the solution or hints?
    If I upgrade WowzaMediaSever from 2 to 3, these problem will not appear?
    Last edited by scdmd78; 01-17-2013 at 05:28 PM.

  2. #2
    Join Date
    Dec 2007
    Posts
    22,013

    Default

    Do you have any custom modules?

    Are there any other applications on that server using substantial memory?

    Show the start up lines in the access log.

    Upgrading could help
    http://www.wowza.com/forums/content....-Upgrade-Guide

    Richard

  3. #3
    Join Date
    Dec 2012
    Posts
    7

    Default

    Richard, thank you for answer.

    This machine dedicated for WowzaMediaServer. (Not VOD, but olny live with FMLE)

    And I use only 1 module HTTPProvider, there is no other custom module.

    Here is Start up lines.
    2012-12-25	09:01:02	KST	server-start	server	INFO	200	Wowza Media Server 2 Perpetual 2.2.4.07 build27628	-	-	-	-	0.711	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Serial number: XXXXX-XXXXX-XXXXX-XXXXX-UCNRH	-	-	-	0.713	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Maximum connections: Unlimited	-	-	-	0.714	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Hardware Available Processors: 12	-	-	-	0.719	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Hardware Physical Memory: 5311MB/7000MB	-	-	-	0.72	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Hardware Swap Space: 2047MB/2047MB	-	-	-	0.721	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Max File Descriptor Count: 1024	-	-	-	0.722	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Open File Descriptor Count: 44	-	-	-	0.722	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	OS Name: Linux	-	-	-	0.723	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	OS Version: 2.6.18-274.7.1.el5xen	-	-	-	0.724	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	OS Architecture: amd64	-	-	-	0.724	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Name: Java HotSpot(TM) 64-Bit Server VM	-	-	-	0.725	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Vendor: Oracle Corporation	-	-	-	0.725	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Version: 1.7.0_03	-	-	-	0.726	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java VM Version: 22.1-b02	-	-	-	0.726	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Spec Version: 1.7	-	-	-	0.727	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Home: /usr/java/jdk1.7.0_03/jre	-	-	-	0.728	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Max Heap Size: 4916MB	-	-	-	0.728	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Architecture: 64	-	-	-	0.729	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Encoding[file.encoding]: UTF-8	-	-	-	0.729	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Args[0]: -Xmx5000M	-	-	-	0.73	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Args[1]: -XX:+UseConcMarkSweepGC	-	-	-	0.731	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Args[2]: -XX:+CMSIncrementalMode	-	-	-	0.731	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Args[3]: -XX:+UseParNewGC	-	-	-	0.732	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Args[4]: -XX:NewSize=256m	-	-	-	0.732	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Args[5]: -Djava.net.preferIPv4Stack=true	-	-	-	0.733	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Args[6]: -Dcom.sun.management.jmxremote=true	-	-	-	0.733	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Args[7]: -Dcom.wowza.wms.AppHome=/WowzaMediaServer	-	-	-	0.734	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Args[8]: -Dcom.wowza.wms.ConfigURL=	-	-	-	0.734	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java Args[9]: -Dcom.wowza.wms.ConfigHome=/WowzaMediaServer	-	-	-	0.735	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java GC[0]: ParNew	-	-	-	0.735	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Java GC[1]: ConcurrentMarkSweep	-	-	-	0.736	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:02	KST	comment	server	INFO	200	-	Server threads[h/t]: 10/10	-	-	-	0.737	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:04	KST	comment	server	INFO	200	-	JMX java.rmi.server.hostname: xxx.xxx.xxx.xxx	-	-	-	2.346	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:04	KST	comment	server	INFO	200	-	JMX bind attempt service:jmx:rmi://127.0.0.1:8084/jndi/rmi://127.0.0.1:8085/jmxrmi	-	-	-	2.346	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:04	KST	comment	server	INFO	200	-	JMX password file: /WowzaMediaServer/conf/jmxremote.password	-	-	-	2.35	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:04	KST	comment	server	INFO	200	-	JMX access file: /WowzaMediaServer/conf/jmxremote.access	-	-	-	2.35	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:04	KST	comment	server	INFO	200	-	JMX bind successful	-	-	-	2.549	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:04	KST	comment	server	INFO	200	-	CMDInterface now listening: [any]:8083	-	-	-	2.598	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:05	KST	vhost-start	vhost	INFO	200	_defaultVHost_	-	-	-	-	3.261	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:05	KST	comment	server	INFO	200	-	_defaultVHost_ threads[h/t]:360/240 home:/WowzaMediaServer	-	-	-	3.325	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:05	KST	comment	vhost	INFO	200	_defaultVHost_	Bind attempt ([any]:1935:6)	-	-	-	3.331	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:05	KST	comment	vhost	INFO	200	_defaultVHost_	Bind successful ([any]:1935)	-	-	-	3.375	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:05	KST	comment	vhost	INFO	200	_defaultVHost_	Bind attempt ([any]:554:6)	-	-	-	3.378	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:05	KST	comment	vhost	INFO	200	_defaultVHost_	Bind successful ([any]:554)	-	-	-	3.379	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:05	KST	comment	vhost	INFO	200	_defaultVHost_	Bind attempt ([any]:8086:1)	-	-	-	3.379	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    2012-12-25	09:01:05	KST	comment	vhost	INFO	200	_defaultVHost_	Bind successful ([any]:8086)	-	-	-	3.38	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
    Last edited by scdmd78; 12-26-2012 at 11:38 PM.

  4. #4
    Join Date
    Dec 2007
    Posts
    22,013

    Default

    Try reducing java heap size to 4000M or 3500M, because you only have about 5g available when the server is started.

    Hardware Physical Memory: 5311MB/7000MB
    d

    Richard

  5. #5
    Join Date
    Dec 2012
    Posts
    7

    Default

    Richard, thank you for advice.

    I reduced java heap size to 4000M as your advice. (option: -Xmx4000M )
    There was an exactly same problem after 3 days.

    Is there any other hints for this problem?

    I will try anything to solve this problem.

  6. #6
    Join Date
    Dec 2007
    Posts
    22,013

    Default

    You are using advanced GC options which might not be helping. As stated in step #5 of the tuning guide, we suggest not using these options unless there is already a problem as they might cause problems. Do you have the "-server" flag turned on following step #4?

    Have you followed tuning step #6 for a 12 core machine, which is what you have?

    It is also possible that you are just over-loading the machine, in which case you might need to setup Liverepeater (origin/edge) cluster to handle the load.

    Richard

  7. #7
    Join Date
    Dec 2012
    Posts
    7

    Default

    Richard, thanks for your advice.

    I checked and test again as your advice.

    1. Of course this WowzaMediaServer runs as "-server" mode.
    2. I removed this GC option, but it still has same problem.
    3. I set step #6 for 12 core, such as ProcessorCount, PoolSize. -> It has same problem.
    4. I increase RAM from 7G to 11G and set 8G JAVA heap size and I added "NewSize=512" option. -> It has same problem.

    5. I don't try origin/edge cluster not yet. This will be final step in this situation.

    I found suspicious option "RTSPSessionTimeout=0" that is in "Application.xml".
    I want to try test as "RTSPSessionTimeout=90000" in "Applicaiont.xml".
    But now some Android apps can't keep connection without option "RTSPSessionTimeout=0".

    Is it possible "RTSPSessionTimeout=0" can be cause of this memory problem?
    So, I tried analyzing java heap memory dump.

    Please look around the capture image.
    Last edited by scdmd78; 01-15-2013 at 07:34 PM.

  8. #8
    Join Date
    Dec 2007
    Posts
    22,013

    Default

    Do you mean the /conf/[app-name]/Application.xml /RTP /Property "streamTimeout"?
    <Property>
    	<Name>streamTimeout</Name>
    	<Value>12000</Value>
    	<Type>Integer</Type>
    </Property>
    Are there outOfMemory errors in the access and error log?

    Is it possible for you to upgrade to Wowza 3.5? The current version is 3.5.0.02, with the latest patch applied.

    http://www.wowza.com/forums/content....-Upgrade-Guide

    It might improve the issue at least, there has been much improvement since 2.2.4, though I cannot guarantee that

    Richard

  9. #9
    Join Date
    Dec 2012
    Posts
    7

    Default

    No, it means the /conf/[app-name]/Application.xml /RTP /RTSPSessionTimeout

                    <RTP>
                            <AVSyncMethod>senderreport</AVSyncMethod>
                            <MaxRTCPWaitTime>12000</MaxRTCPWaitTime>
                            <IdleFrequency>75</IdleFrequency>
                            <RTSPSessionTimeout>0</RTSPSessionTimeout>
                            <RTSPMaximumPendingWriteBytes>0</RTSPMaximumPendingWriteBytes>
                            <RTSPBindIpAddress></RTSPBindIpAddress>
                            <RTSPConnectionIpAddress>0.0.0.0</RTSPConnectionIpAddress>
                            <RTSPOriginIpAddress>127.0.0.1</RTSPOriginIpAddress>
                            <IncomingDatagramPortRanges>*</IncomingDatagramPortRanges>
                            <!-- Properties defined here will override any properties defined in conf/RTP.xml for any depacketizers loaded by this application -->
                            <Properties>
                            </Properties>
                    </RTP>
    Actually, I can't upgrade Wowza version to 3.5 right now.
    And there are no errors in access or error log.
    It just freeze.

    I'm trying test as "<RTSPSessionTimeout>7200000</RTSPSessionTimeout>".
    May be I can get the result 1 or 2 days later.
    Last edited by scdmd78; 01-16-2013 at 07:00 PM.

  10. #10
    Join Date
    Dec 2012
    Posts
    7

    Default

    Here is the result of testing with option "RTSPSessionTimeout" in /conf/[app-name]/Application.xml /RTP

    Before: <RTSPSessionTimeout>0</RTSPSessionTimeout> (The WowzaMediaServer was freezing in every 1 or 2 days. Only the solution is restarting.)
    Now: <RTSPSessionTimeout>7200000</RTSPSessionTimeout>

    The WowzaMediaServer doesn't freeze for 8 days and it works stable yet.
    It was effective setting in compare with before.

    I attached some graph of memory usage

    1. WowzaMediaServer freezing pattern.


    2. WowzaMediaServer works stable pattern.

    I'm not sure the problem has been solved yet. Because of the red box in this graph.
    Anyway, It works stable till now.

    Please let me know your opinion of this result.
    Last edited by scdmd78; 01-23-2013 at 01:58 AM.

Similar Threads

  1. the number of memories (RAM) that wowza 4.0 are required.
    By sumimasa in forum Performance and Tuning
    Replies: 1
    Last Post: 02-21-2014, 07:20 AM
  2. WowzaMediaServer logging not working
    By guilleart in forum Wowza Media Server 3 for Amazon EC2 Discussion
    Replies: 2
    Last Post: 02-23-2012, 06:05 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •