Results 1 to 6 of 6

Thread: Error when using startup package on m1.medium Wowza Amazon EC2 instance

  1. #1
    Join Date
    Jan 2012
    Posts
    21

    Question [SOLVED] Error when using startup package on m1.medium Wowza Amazon EC2 instance

    Hello,

    I have a start-up package that loads successfully on m1.small Wowza instances on Amazon EC2, but when I try to load the same package on m1.medium or m1.large, the startup package fails. Here is an output of the install log file:

    0    [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  - Installer.doAmazonInstall
    3    [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  -   userdata-url: http://169.254.169.254/2007-08-29/user-data
    3    [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  -   userdata-destination: /opt/working/userdata.zip
    36   [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  -   userdata-isbase64: false
    36   [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  -   unzip: /opt/working/userdata.zip
    94   [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  -   result: /opt/working
    160  [main] INFO  com.wowza.amazon.ec2.install.Installer  - InstallParser.parseStartup: /opt/working/startup.xml
    222  [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  - InstallCommands.downloadFile
    224  [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  -   url: http://[omitted-for-security]/live-repeater-origin-balancer.zip
    224  [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  -   destination: /opt/live-repeater-origin-balancer.zip
    418  [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  -   action: install
    418  [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  -   unzipTo: /opt
    1059 [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  -   mergeSrc: /opt
    1059 [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  -   mergeTarget: /usr/local/WowzaMediaServer
    5585 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /usr/local/WowzaMediaServer/aws/bin/ec2evio (Too many open files)
    5585 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2-delete-network-acl (Too many open files)
    5585 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2addvpc (Too many open files)
    5585 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2udi (Too many open files)
    5586 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/rds-create-db-subnet-group (Too many open files)
    5586 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2-describe-spot-datafeed-subscription (Too many open files)
    5586 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2dvolatt (Too many open files)
    5586 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2mimatt (Too many open files)
    5586 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/rds-reboot-db-instance (Too many open files)
    5587 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2-ami-tools-version (Too many open files)
    5587 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2datt (Too many open files)
    5587 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2min (Too many open files)
    5587 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/as-delete-launch-config (Too many open files)
    5587 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2ii (Too many open files)
    5588 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/elb-create-lb-listeners (Too many open files)
    5588 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2mim (Too many open files)
    5588 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2addcgw (Too many open files)
    5588 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/rds-revoke-db-security-group-ingress (Too many open files)
    5588 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2-delete-route (Too many open files)
    5589 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/iam-accountgetpasswordpolicy (Too many open files)
    5589 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2-delete-volume (Too many open files)
    5589 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2-bundle-instance (Too many open files)
    5589 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2assocrtb (Too many open files)
    5590 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/iam-instanceprofilelistforrole (Too many open files)
    5590 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/iam-groupcreate (Too many open files)
    5590 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2dsph (Too many open files)
    5590 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2-allocate-address (Too many open files)
    5590 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/as-update-auto-scaling-group (Too many open files)
    5591 [main] ERROR com.wowza.util.FileUtils  - copyFile: java.io.FileNotFoundException: /opt/aws/bin/ec2deldopt (Too many open files)
    5820 [main] ERROR com.wowza.util.FileUtils  - FileUtils.traverseDirectory: recurse: java.lang.NullPointerException
    5821 [main] ERROR com.wowza.util.FileUtils  - FileUtils.traverseDirectory: recurse: java.lang.NullPointerException
    5821 [main] ERROR com.wowza.util.FileUtils  - FileUtils.traverseDirectory: recurse: java.lang.NullPointerException
    5821 [main] INFO  com.wowza.amazon.ec2.install.InstallCommands  - InstallCommands.runScript
    5835 [main] ERROR com.wowza.util.FileUtils  - FileUtils.fixLineFeeds: (/opt/working/tuning/tune.sh)java.io.IOException: Too many open files
    5836 [main] ERROR com.wowza.amazon.ec2.install.InstallCommands  - InstallCommands.runScript: Error fixing script linefeeds: /opt/working/tuning/tune.sh
    5986 [main] ERROR com.wowza.amazon.ec2.install.InstallCommands  - Installer.doAmazonInstall: Error writing publish.password: java.io.FileNotFoundException: /usr/local/WowzaMediaServer/conf/publish.password (Too many open files)
    5986 [main] ERROR com.wowza.amazon.ec2.install.InstallCommands  - Installer.doAmazonInstall: Error writing jmxremote.access: java.io.FileNotFoundException: /usr/local/WowzaMediaServer/conf/jmxremote.access (Too many open files)
    5987 [main] ERROR com.wowza.amazon.ec2.install.InstallCommands  - Installer.doAmazonInstall: Error writing jmxremote.password: java.io.FileNotFoundException: /usr/local/WowzaMediaServer/conf/jmxremote.password (Too many open files)
    Any help would be appreciated

    Thanks
    Last edited by pobrejuanito; 07-19-2012 at 02:16 PM.

  2. #2
    Join Date
    Jan 2012
    Posts
    21

    Default

    Anyone?

  3. #3
    Join Date
    Dec 2007
    Posts
    21,962

    Default

    Have you included any tuning changes in the startup package (/bin/startup.sh or /bin/wms.sh in particular)? Exactly what is in the startup package?

    The "Too many files open" error happens in Linux when the ulimit is not set properly. See step #8 in the tuning guide:
    http://www.wowza.com/forums/content....ormance-tuning

    The EC2 pre-built AMIs are tuned, so you should not need to follow the tuning guide. But if you have included /bin/startup.sh or /bin/wms.sh in the startup package, then you are tuned for a small instance instead

    Richard

  4. #4
    Join Date
    Sep 2011
    Posts
    1,934

    Default

    Hi
    This could be many things I'm afraid, is the small instance rather old and the Medium/Large instances relatively new or vice versa?
    It could be a structure change from old to new or new to old that's giving this error.

    It could also be that the server instance isn't tuned correctly (although it should be because they are pre-tuned before being sold).
    Is the Package a Wowza package or a custom package?

    Jason

  5. #5
    Join Date
    Jan 2012
    Posts
    21

    Default

    Hi, thanks for the reply,

    I am using a pre-built Wowza package ami-c7735682 size m1.medium. I am using the startup package called liverepeater-origin.zip that is found in http://wowzamediasystems.s3.amazonaw...ckagelist.html and I modified it so that it includes the load-balancer configurations.

    startup.xml

    <Startup>
    	<Commands>
    		<Download>
    			<URL>http://[url]/live-repeater-origin-balancer.zip</URL>
    			<Destination>/opt/live-repeater-origin-balancer.zip</Destination>
    			<Action>INSTALL</Action>
    		</Download>
    		<RunScript>
    			<Script>tuning/tune.sh</Script>
    		</RunScript>
    	</Commands>
    </Startup>
    MediaCache.xml
    <Root>
    	<MediaCache>
    		<WriterThreadPool>
    			<PoolSize>8</PoolSize>
    		</WriterThreadPool>
    		<ReadAheadThreadPool>
    			<PoolSize>4</PoolSize>
    		</ReadAheadThreadPool>
    		<MaxPendingWriteRequestSize>{MEDIACACHE_MAXPENDINGWRITEREQUESTSIZE}</MaxPendingWriteRequestSize>
    		<MaxPendingReadAheadRequestSize>{MEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE}</MaxPendingReadAheadRequestSize>
    		<GCFrequency>10000</GCFrequency>
    		<ContextMapperClass></ContextMapperClass>
    		<Properties>
    		</Properties>
    	</MediaCache>
    	<MediaCacheStores>
    		<MediaCacheStore>
    			<Path>/usr/local/WowzaMediaServer/mediacache</Path>
    			<MaxSize>{MEDIACACHE_STOREMAXSIZE}</MaxSize>
    			<Level1FolderCount>16</Level1FolderCount>
    			<Level2FolderCount>16</Level2FolderCount>
    			<FileCount>1000</FileCount>
    			<WriteRate>10000000</WriteRate>
    			<WriteRateMaxBucketSize>40000000</WriteRateMaxBucketSize>
    			<WriteRateFillFrequency>100</WriteRateFillFrequency>
    			<Properties>
    			</Properties>
    		</MediaCacheStore>
    	</MediaCacheStores>
    	<MediaCacheSources>
    		<MediaCacheSource>
    			<Name>amazons3</Name>
    			<BasePath>http://s3.amazonaws.com/</BasePath>
    			<Prefix>amazons3/</Prefix>
    			<BaseClass>com.wowza.wms.plugin.mediacache.impl.MediaCacheItemHTTPImpl</BaseClass>
    			<ReaderClass></ReaderClass>
    			<DefaultBlockSize>262144</DefaultBlockSize>
    			<MaxTimeToLive>1200000</MaxTimeToLive>
    			<MinTimeToLive>600000</MinTimeToLive>
    			<ReadAhead>true</ReadAhead>
    			<ReadAheadThreshold>50</ReadAheadThreshold>
    			<IsPassThru>false</IsPassThru>
    			<Properties>
    				<Property>
    					<Name>maxPoolSize</Name>
    					<Value>25</Value>
    					<Type>Integer</Type>
    				</Property>
    				<Property>
    					<Name>httpReadTimeout</Name>
    					<Value>6000</Value>
    					<Type>Integer</Type>
    				</Property>
    				<Property>
    					<Name>httpConnectionTimeout</Name>
    					<Value>6000</Value>
    					<Type>Integer</Type>
    				</Property>
    				<Property>
    					<Name>httpReadRetries</Name>
    					<Value>3</Value>
    					<Type>Integer</Type>
    				</Property>
    				<Property>
    					<Name>httpSendBufferSize</Name>
    					<Value>8000</Value>
    					<Type>Integer</Type>
    				</Property>
    				<Property>
    					<Name>httpReceiveBufferSize</Name>
    					<Value>65000</Value>
    					<Type>Integer</Type>
    				</Property>
    				<Property>
    					<Name>isAmazonS3</Name>
    					<Value>true</Value>
    					<Type>Boolean</Type>
    				</Property>
    				<Property>
    					<Name>s3BucketNameInDomain</Name>
    					<Value>true</Value>
    					<Type>Boolean</Type>
    				</Property>
    				<!-- Uncomment and set properties if streaming from S3 authenticated -->
    				<!--
    				<Property>
    					<Name>awsAccessKeyId</Name>
    					<Value></Value>
    				</Property>
    				<Property>
    					<Name>awsSecretAccessKey</Name>
    					<Value></Value>
    				</Property>
    				-->
    			</Properties>
    		</MediaCacheSource>
    	</MediaCacheSources>
    </Root>
    setenv.sh

    #!/bin/sh
    
    _EXECJAVA=java
    if [ -f /usr/java/latest/bin/java ] ; then
    	_EXECJAVA=/usr/java/latest/bin/java
    fi
    JAVA_OPTS="-Xmx{JAVAMAXHEAPSIZE} -server"
    
    WMSAPP_HOME=/usr/local/WowzaMediaServer
    WMSCONFIG_HOME=/usr/local/WowzaMediaServer
    WMSCONFIG_URL=
    
    export WMSAPP_HOME WMSCONFIG_HOME JAVA_OPTS _EXECJAVA
    tune.sh
    #!/bin/sh
    
    VARJAVAMAXHEAPSIZE=1000M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=150000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=75000000
    VARMEDIACACHE_STOREMAXSIZE=100000000000
    
    # memory/disk
    
    if [ ${AWSEC2_METADATA_INSTANCE_TYPE} = "m1.small" ]; then
    # 154068/153899044
    VARJAVAMAXHEAPSIZE=1000M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=150000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=75000000
    VARMEDIACACHE_STOREMAXSIZE=100000000000
    elif [ ${AWSEC2_METADATA_INSTANCE_TYPE} = "m1.large" ]; then
    # 7652592/433455904
    VARJAVAMAXHEAPSIZE=5000M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=600000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=400000000
    VARMEDIACACHE_STOREMAXSIZE=300000000000
    elif [ ${AWSEC2_METADATA_INSTANCE_TYPE} = "m1.xlarge" ]; then
    # 15378216/433455904
    VARJAVAMAXHEAPSIZE=8000M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=800000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=500000000
    VARMEDIACACHE_STOREMAXSIZE=300000000000
    elif [ ${AWSEC2_METADATA_INSTANCE_TYPE} = "t1.micro" ]; then
    # 613M/???
    VARJAVAMAXHEAPSIZE=300M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=20000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=10000000
    VARMEDIACACHE_STOREMAXSIZE=100000000000
    elif [ ${AWSEC2_METADATA_INSTANCE_TYPE} = "m2.xlarge" ]; then
    # 17530704/418579200
    VARJAVAMAXHEAPSIZE=8000M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=1000000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=600000000
    VARMEDIACACHE_STOREMAXSIZE=300000000000
    elif [ ${AWSEC2_METADATA_INSTANCE_TYPE} = "m2.2xlarge" ]; then
    # 35134420/866917368
    VARJAVAMAXHEAPSIZE=8000M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=1000000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=600000000
    VARMEDIACACHE_STOREMAXSIZE=500000000000
    elif [ ${AWSEC2_METADATA_INSTANCE_TYPE} = "m2.4xlarge" ]; then
    # 70343656/866917368
    VARJAVAMAXHEAPSIZE=8000M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=1000000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=600000000
    VARMEDIACACHE_STOREMAXSIZE=500000000000
    elif [ ${AWSEC2_METADATA_INSTANCE_TYPE} = "c1.medium" ]; then
    # 1758616/350891748
    VARJAVAMAXHEAPSIZE=1000M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=150000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=75000000
    VARMEDIACACHE_STOREMAXSIZE=200000000000
    elif [ ${AWSEC2_METADATA_INSTANCE_TYPE} = "c1.xlarge" ]; then
    # 7136868/433455904
    VARJAVAMAXHEAPSIZE=5000M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=600000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=400000000
    VARMEDIACACHE_STOREMAXSIZE=300000000000
    elif [ ${AWSEC2_METADATA_INSTANCE_TYPE} = "cc1.4xlarge" ]; then
    # 23G/433455904
    VARJAVAMAXHEAPSIZE=8000M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=1000000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=600000000
    VARMEDIACACHE_STOREMAXSIZE=300000000000
    elif [ ${AWSEC2_METADATA_INSTANCE_TYPE} = "cc2.8xlarge" ]; then
    # 60.5G/433455904
    VARJAVAMAXHEAPSIZE=8000M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=1000000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=600000000
    VARMEDIACACHE_STOREMAXSIZE=300000000000
    elif [ ${AWSEC2_METADATA_INSTANCE_TYPE} = "cg1.4xlarge" ]; then
    # 22G/433455904
    VARJAVAMAXHEAPSIZE=8000M
    VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE=1000000000
    VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE=600000000
    VARMEDIACACHE_STOREMAXSIZE=300000000000
    fi
    export VARJAVAMAXHEAPSIZE
    export VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE
    export VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE
    export VARMEDIACACHE_STOREMAXSIZE
    
    echo "Tuning Wowza Media Server: ${AWSEC2_METADATA_INSTANCE_TYPE}"
    echo "Java Heap Size: ${VARJAVAMAXHEAPSIZE}"
    echo "MediaCache: MaxPendingWriteRequestSize: ${VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE}"
    echo "MediaCache: MaxPendingReadAheadRequestSize: ${VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE}"
    echo "MediaCache: MediaCacheStore/MaxSize: ${VARMEDIACACHE_STOREMAXSIZE}"
    
    WMSAPP_HOME=/usr/local/WowzaMediaServer
    cp ${WMSAPP_HOME}/bin/setenv.sh ${WMSAPP_HOME}/bin/setenv-save.sh
    sed "s/{JAVAMAXHEAPSIZE}/${VARJAVAMAXHEAPSIZE}/" "tuning/setenv.sh" > ${WMSAPP_HOME}/bin/setenv.sh
    cp ${WMSAPP_HOME}/conf/MediaCache.xml ${WMSAPP_HOME}/bin/MediaCache-save.xml
    sed "s/{MEDIACACHE_MAXPENDINGWRITEREQUESTSIZE}/${VARMEDIACACHE_MAXPENDINGWRITEREQUESTSIZE}/;s/{MEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE}/${VARMEDIACACHE_MAXPENDINGREADAHEADREQUESTSIZE}/;s/{MEDIACACHE_STOREMAXSIZE}/${VARMEDIACACHE_STOREMAXSIZE}/" "tuning/MediaCache.xml" > ${WMSAPP_HOME}/conf/MediaCache.xml
    This is happening on m1.large too
    Last edited by pobrejuanito; 07-19-2012 at 10:26 AM.

  6. #6
    Join Date
    Jan 2012
    Posts
    21

    Default

    I think I figured it out. I think it didn't like the

    <Action>INSTALL</Action>
    on startup.xml

    So I changed it to:

    <Startup>
    	<Commands>
    		<Download>
    			<URL>http://[url]/live-repeater-origin-balancer.zip</URL>
    			<Destination>wowza/live-repeater-origin-balancer.zip</Destination>
    			<Action>UNZIP</Action>
    		</Download>
    		<Install>
    		  <Folder>wowza</Folder>
    		</Install>
    		<RunScript>
    			<Script>tuning/tune.sh</Script>
    		</RunScript>
    	</Commands>
    </Startup>
    and that worked.

Similar Threads

  1. Startup Package for EC2 - cannot be found
    By ACI-joe in forum Wowza Media Server 3 for Amazon EC2 Discussion
    Replies: 2
    Last Post: 01-13-2014, 08:38 PM
  2. Very High CPU Utilization (EC2 Instance - m.medium)
    By ravjr76 in forum General Forum
    Replies: 9
    Last Post: 12-12-2012, 06:25 AM
  3. Save Wowza configuration Amazon EC2 instance
    By tekmedia in forum Wowza Media Server 3 for Amazon EC2 Discussion
    Replies: 1
    Last Post: 02-17-2012, 08:42 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •