Page 1 of 2 12 LastLast
Results 1 to 10 of 15

Thread: MediaCache not filling up

  1. #1

    Default MediaCache not filling up

    I've been running a few servers for a few days with a media cache store limit of 500Gig.. but the size of the cache doesn't fill up past about 5.6 Gig. I've checked all the configs and restarted the servers a few times but it simply refuses to fill up. As a result I'm pulling excessive data from my http source.

    Thanks for any and all help.
    Jake

    The mediacache store config is:

    <MediaCacheStores>
                    <MediaCacheStore>
                            <Name>default</Name>
                            <Description>Default Store</Description>
                            <Path>${com.wowza.wms.context.ServerConfigHome}/mediacache</Path>
                            <MaxSize>500G</MaxSize>
                            <Level1FolderCount>24</Level1FolderCount>
                            <Level2FolderCount>24</Level2FolderCount>
                            <FileCount>1000</FileCount>
                            <WriteRate>38M</WriteRate>
                            <WriteRateMaxBucketSize>152M</WriteRateMaxBucketSize>
                            <WriteRateFillFrequency>100</WriteRateFillFrequency>
                            <Properties>
                            </Properties>
                    </MediaCacheStore>
            </MediaCacheStores>

  2. #2
    Join Date
    May 2013
    Posts
    681

    Default

    It's possible that your source is timing out on files which expires them from cache at a rate faster than requests. You might try toggling the MediaCacheSource->MaxTimeToLive and MediaCacheSource->MinTimeToLive accordingly.

  3. #3

    Default

    I did forget to mention my min time to live is a week and max time to live is 2 weeks.. so I don't think we're dealing with the timing out prematurely.. especially considering I've got a flow of about 20 Mbps running into the wowza box from the source.

    Thanks,
    Jake

    Applicable section below:

    <MediaCacheSource>
                            <Name>http</Name>
                            <Type>HTTP</Type>
                            <Description></Description>
                            <BasePath>http://xxx.xxx.xxx.xxx:8080/</BasePath>
                            <Prefix></Prefix>
                            <BaseClass>com.wowza.wms.mediacache.impl.MediaCacheItemHTTPImpl</BaseClass>
                            <ReaderClass></ReaderClass>
                            <DefaultBlockSize>1M</DefaultBlockSize>
                            <MaxTimeToLive>1209600</MaxTimeToLive>
                            <MinTimeToLive>604800</MinTimeToLive>
                            <ReadAhead>true</ReadAhead>
                            <ReadAheadThreshold>30</ReadAheadThreshold>
                            <IsPassThru>false</IsPassThru>
                            <Properties>
                                    <Property>
                                            <Name>debugHTTPRequests</Name>
                                            <Value>false</Value>
                                            <Type>Boolean</Type>
                                    </Property>
                                    <Property>
                                            <Name>maxPoolSize</Name>
                                            <Value>25</Value>
                                            <Type>Integer</Type>
                                    </Property>
                                    <Property>
                                            <Name>httpReadTimeout</Name>
                                            <Value>6000</Value>
                                            <Type>Integer</Type>
                                    </Property>
                                    <Property>
                                            <Name>httpConnectionTimeout</Name>
                                            <Value>6000</Value>
                                            <Type>Integer</Type>
                                    </Property>
                                    <Property>
                                            <Name>httpReadRetries</Name>
                                            <Value>3</Value>
                                            <Type>Integer</Type>
                                    </Property>
                                    <Property>
                                            <Name>httpSendBufferSize</Name>
                                            <Value>8000</Value>
                                            <Type>Integer</Type>
                                    </Property>
                                    <Property>
                                            <Name>httpReceiveBufferSize</Name>
                                            <Value>65000</Value>
                                            <Type>Integer</Type>
                                    </Property>
                                    <Property>
                                            <Name>urlDecodeStreamName</Name>
                                            <Value>false</Value>
                                            <Type>Boolean</Type>
                                    </Property>
                                    <Property>
                                            <Name>base64DecodeStreamName</Name>
                                            <Value>false</Value>
                                            <Type>Boolean</Type>
                                    </Property>
                                    <!-- Uncomment if pulling from a proxy -->
                                    <!--
                                    <Property>
                                            <Name>proxyHost</Name>
                                            <Value></Value>
                                    </Property>
                                    <Property>
                                            <Name>proxyPort</Name>
                                            <Value></Value>
                                    </Property>
                                    <Property>
                                            <Name>requestFullURL</Name>
                                            <Value>true</Value>
                                            <Type>Boolean</Type>
                                    </Property>
                                    -->
                            </Properties>
                    </MediaCacheSource>
    Last edited by jakehilton; 02-19-2014 at 09:17 AM.

  4. #4

    Default

    I restarted the box earlier today and watched the disk usage. It started filling normally but occasionally the media cache dir would shrink in size.. then increase and shrink. It did this until it hit about 5.4 Gig and now it's fluctuating around that size.. when it should be growing and never getting smaller.. especially within only 6 hours.

    I'm starting to think something is fundamentally broken with the media cache. It doesn't seem to be honoring the values in the config.

    Jake
    Last edited by jakehilton; 02-19-2014 at 04:02 PM.

  5. #5

    Default

    I watched the directory size over time.. and here are the results with the configs mentioned above:

    root@475911a:/usr/local/WowzaStreamingEngine/mediacache# du -hs
    3.8G	.
    root@475911a:/usr/local/WowzaStreamingEngine/mediacache# du -sh
    5.4G	.
    root@475911a:/usr/local/WowzaStreamingEngine/mediacache# du -sh
    5.5G	.
    root@475911a:/usr/local/WowzaStreamingEngine/mediacache# du -sh
    5.6G	.
    root@475911a:/usr/local/WowzaStreamingEngine/mediacache# du -sh
    4.6G	.
    root@475911a:/usr/local/WowzaStreamingEngine/mediacache# du -sh
    4.8G	.
    Notice the drop.. and wowza has only been up for 7 hours.

  6. #6
    Join Date
    Dec 2007
    Posts
    22,013

    Default

    Jake,

    Are you using latest MediaCache? Make sure by re-downloading if you haven't recently. And enable mediaCacheDebugLog in Server.xml Properties

    <Property>
         <Name>mediaCacheDebugLog</Name>
         <Value>true</Value>
         <Type>Boolean</Type>
        </Property>
    Richard

  7. #7

    Default

    Richard,
    I'm on wowza 4 so it's the latest that is packaged with that.

    I'll try that debug setup and see what I can see. Where should I see the debug output?

    Jake

  8. #8

    Default

    Here is what I show when the startup happens.. just want to verify all values are correct:

    2014-02-19      17:33:29        CST     comment server  INFO    200     -       MediaCache[MediaCache]: Thread pool size: [writer/readahead]:16/8       -       -       -       1.192   -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -
    2014-02-19      17:33:29        CST     comment server  INFO    200     -       MediaCache[MediaCache]: MAX Pending size: [writer/readahead]:500M/25M   -       -       -       1.192   -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -
    2014-02-19      17:33:29        CST     comment server  INFO    200     -       MediaCache[MediaCache]: Add store: path:/usr/local/WowzaStreamingEngine/mediacache maxSize:536870912000 -       -       -       1.193   -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -
    2014-02-19      17:33:29        CST     comment server  INFO    200     -       MediaCache[MediaCache]: Add source[http]: prefix: basePath:http://xxx.xxx.xxx.xxx:8080/  -       -       -       1.206   -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -
    2014-02-19      17:33:29        CST     comment server  INFO    200     -       MediaCache[MediaCache]: Flushing cache: start   -       -       -       1.21    -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -
    2014-02-19      17:33:30        CST     comment server  INFO    200     -       MediaCache[MediaCache]: Flushing cache: done    -       -       -       1.864   -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -
    2014-02-19      17:33:30        CST     comment server  INFO    200     -       MediaCache[MediaCache]: Start MediaCache GC     -       -       -       1.864   -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -       -
    2014-02-19      17:33:30        CST     comment server  INFO    200     -       MediaCache[MediaCache]: Started -       -       -       1.865   -
    Those aren't the debug of course.. but wanted to get those in the ticket. I'll look for more interesting items now in the access log.

  9. #9

    Default

    Hi Jake,

    What you may be seeing is that the OS is reporting the actual size of the content on disk but Mediacache is seeing the size of the requested content.

    If you have a high abandonment rate (viewers only watch a very short period of a long video) then these two values will be very different.

    When a piece of content is requested, the Mediacache reserves space in the cache for that piece of content to fit. It does so by writing an empty file to disk with the size set to the media size. Because it is empty, the OS does not see it as used space. As the blocks are pulled in, they are deposited in the file and the size on disk grows.

    When the MediaCache is determining if the cache is full or needs to purge anything, it looks at it's own reserved size and not the size on disk. If it is running full (based on what MediaCache sees) then it will try to purge items based on the Minimum TTL value. This is possibly why you are seeing the size reduce on disk.

    If The mediaCache cannot purge enough old items to make space for the new content then it will not put the new content into the cache and instead, the requests will bypass the mediaCache and go directly to the storage server. This is very bad as it could overload the storage server with lots of tiny requests.

    If you do have a high abandonment rate then I suggest that you use lower TTL settings so that content will get purged quicker. It will always remain in cache if it has been watched recently.

    You can see what the MediaCache is doing by enabling JMX in the Manager(Server > Properties > JMX Remote Configuration) and then after the restart, connect to the server using jConsole and view the MediaCache mbeans.

    Roger.

  10. #10

    Default

    Also, if the mediacache store is on a separate partition, using the df command (rather than the du command) should report the same as what MediaCache is seeing.

    Roger.

Page 1 of 2 12 LastLast

Similar Threads

  1. mediacache and amlst
    By notreg in forum Server-side Modules and Code Samples Discussion
    Replies: 1
    Last Post: 07-24-2013, 09:29 AM
  2. Mediacache with Rackspace
    By cherylfoster777 in forum Video On Demand Streaming Discussion
    Replies: 2
    Last Post: 01-25-2013, 07:26 AM
  3. MediaCache setup help
    By eTiMaGo in forum AddOn: Other AddOns
    Replies: 1
    Last Post: 09-26-2012, 08:47 AM
  4. Two questions about MediaCache
    By weichung in forum General Forum
    Replies: 4
    Last Post: 09-11-2012, 06:59 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •