Log Analysis

[Leigh_Spiegel]

This is not the fix, but maybe a workaround depending on what you are doing and what level of stats you are comfortable with…

I’m using two things for stats now.

Cacti, which is awesome at giving near realtime information and can be used with Wowza via this script, Cacti is great for SNMP and graphing most things.

Example of Data Source:

<path_cacti>/scripts/jmxcl.sh -jmx service:jmx:rmi://localhost:8084/jndi/rmi://localhost:8085/jmxrmi -user yyyy -pass xxxxxxxx getConnectionCounts

<path_cacti>/scripts/jmxcl.sh -jmx service:jmx:rmi://localhost:8084/jndi/rmi://localhost:8085/jmxrmi -user yyyy -pass xxxxxxxx getIOInByteRate

It takes these options, if you know what instance/stream name you want to monitor I guess you can graph many things:

getServerVersion

startVHost [vhost]

stopVHost [vhost]

reloadVHostConfig

startAppInstance [vhost:application/appInstance]

touchAppInstance [vhost:application/appInstance]

shutdownAppInstance [vhost:application/appInstance]

startMediaCasterStream [vhost:application/appInstance] [stream-name] [mediacaster-type]

stopMediaCasterStream [vhost:application/appInstance] [stream-name]

resetMediaCasterStream [vhost:application/appInstance] [stream-name]

getConnectionCounts

getConnectionCounts [vhost:application/appInstance]

getConnectionCounts [vhost:application/appInstance] [stream-name]

getIOOutByteRate

getIOOutByteRate [vhost:application/appInstance]

getIOInByteRate

getIOInByteRate [vhost:application/appInstance]

And a solution I found when I purchased jwplayer for better information than AwStats anyway the next day is Google Analytics running on the actual streams. (you can run on the website without JwPlayer of course)

http://www.longtailvideo.com/addons/plugins/107/Google-Analytics-Pro?q=%20alt=

Lets me see how many people looked at the streams from various countries which is good to know for how I use Wowza.

Once I got these two things in operation I didn’t need awstats anymore or sawmill (which I trialled for a while). Best of all it was almost free just had to purchase jwplayer + google plugin.

[Jimmy_Story]

I have multiple LIVE streams set up as separate applications on my server and want to report on each of them individually. I have set up log4j to separate out the log files as follows:

log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_stats=org.apache.log4j.DailyRollingFileAppender
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_stats.DatePattern='.'yyyy-MM-dd
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_stats.File=${com.wowza.wms.ConfigHome}/logs/${com.wowza.wms.context.VHost}/${com.wowza.wms.context.Application}/wowzamediaserver_stats.log
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_stats.layout=com.wowza.wms.logging.ECLFPatternLayout
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_stats.layout.Fields=x-severity,x-category,x-event;date,time,c-client-id,c-ip,c-port,cs-bytes,sc-bytes,x-duration,x-sname,x-stream-id,x-spos,sc-strea
m-bytes,cs-stream-bytes,x-file-size,x-file-length,x-ctx,x-comment
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_stats.layout.OutputHeader=true
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_stats.layout.QuoteFields=false
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_stats.layout.Delimeter=tab
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_stats.layout.CategoryInclude=session,stream
log4j.appender.${com.wowza.wms.context.VHost}_${com.wowza.wms.context.Application}_stats.layout.EventExclude=comment

I have also set up my AWStats config file for one of the applications as follows:

LogFile="logresolvemerge.pl /path/to/app/log/wowzamediaserver_stats.log* |"
LogType=S
LogFormat="%other %other %extraX date time %extraX c-ip %other %extraX %extraX x-duration cs-uri-stem %extraX %extraX sc-bytes %extraX %other %other %other %other"
(and some other config changes to exclude browser specific functions that are not needed)

Every time i try to parse the log files with AWStats I get the following error:

Error: Your personalized LogFormat does not include all fields required by AWStats (Add %methodurl or %method in your LogFormat string).

If I falsely add method to one of the items in my string I get another error. So I know the format is not correct in my string, but how can I match it up with the log4j output?

All I would like to get is get the number of users, their stream times, and possibly unique user. Other things like bytes, etc would be nice but i can roughly figure that out by the stream times.

Or is Sawmill my best option?

[Valeria_Ghirimoldi]

Hello,

i’ve read all the guide and i’ve done all the modifies required, but it still doesn’t work.

Does anyone know if there are others modifies to do??

[Valeria_Ghirimoldi]

actually i do not have a specific error…it doesn’t work at all!

[ben_cheung]

I have the following setting in conf file of awstats (The LogFormat that I set in log4j is the same.

):

LogType=S

LogFormat=“date time %method c-ip %referer %code %url sc-bytes filelength x-sname sc-stream-bytes cs-stream-bytes cs-uri-stem %extra1 %virtualname”

LogSeparator="\t"

After I run the commend perl awstats.pl -config=default, I got:

PS C:\Program Files (x86)\awstats-wmc\awstats-wmc\wwwroot\cgi-bin> perl awstats.pl -config=default

Create/Update database for config “./awstats.default.conf” by AWStats version 7.0 (build 1.971)

From data in log file “C:\Program Files (x86)\Wowza Media Systems\Wowza Media Server 2.2.4\logs\wmconsulting_stats.log”.

…

Phase 1 : First bypass old records, searching new record…

Searching new records from beginning of log file…

Jumped lines in file: 0

Parsed lines in file: 11

Found 7 dropped records,

Found 4 comments,

Found 0 blank records,

Found 0 corrupted records,

Found 0 old records,

Found 0new qualified records.

It drops 7 records, is there any thing wrong? What can I do?

Please help!

[ben_cheung]

Here is an article on using AWStats and Wowza:

http://www.wmconsulting.info/awstats-mod-for-process-wowza-stats/

And there is some discussion here that might be useful that is easy to find if you search for “Wowza AWStats”

Richard

Hi Richard,

Yes, I had followed that website’s instruction and this matter occur. I also spent few days to solve it by google. Finally, I seek help here. I thank I should post this matter there.

Anyway, thank you for your advice. I apppreciate that.

[ben_cheung]

Hi Alejandro,

Thank you for your very very nice advice.

Yes, you are right. I added -showdropped and it returned the logFormat problem.

So I delete the last two flield in both log4j(,c-proto,x-appinst) and awstats.default.conf(%extra1 %virtualname):

log4j.appender.serverAWStats.layout.Fields=date,time,c-proto,c-ip,c-referrer,x-status,cs-uri-stem,sc-bytes,x-duration,x-sname,sc-stream-bytes,cs-stream-bytes,x-ctx,c-proto,x-appinst

LogFormat=“date time %method c-ip %referer %code %url sc-bytes filelength x-sname sc-stream-bytes cs-stream-bytes cs-uri-stem %extra1 %virtualname”

and it works(at least I can see some number in the figure).

However, I want to know how to change above two flields(%extra1 %virtualname) instead of deleting them?

I tested with both RTMP and RTSP.

I did’t install Apache, I only use Astats and Wowzw Media Server so I downloaded those two files and comment, modified some setting in the awstats.default.conf.

[Kekebanos_Likourgos]

I bought the WMS Log Analyzer, a very good software that helped us on the wowza media server to find those,
who were arbitrarily with connected and pulling very high bandwidth in other relaying servers, causing a problems in the dedicated server…

We propose it unreservedly, validated with five stars!

https://www.exacttrend.com/wmsloganalyzer

---