Using gstreamer-0.10 with Wowza

Question

Hi

I've begun experimenting using Wowza together with the gstreamer media framework.

Here's the cmd line i'm using:

gst-launch-0.10 -vvv videotestsrc ! queue ! x264enc byte-stream=true bitrate=300 ! rtph264pay ! udpsink port=5000 host=127.0.0.1 sync=false

It results in creating a 320x240 video test pattern encoded using x264.

Since there's no RTCP enabled with such command line, i put the systemclock sync parameter in the Application.xml config file.

Server-side (on the local host), here's the myStream.sdp file:

v=0

o=- 1188340656180883 1 IN IP4 127.0.0.1

s=Session streamed by GStreamer

i=server.sh

t=0 0

a=tool:GStreamer

a=type:broadcast

m=video 5000 RTP/AVP 96

c=IN IP4 127.0.0.1

a=rtpmap:96 H264/90000

Here's the debug log (relevant parts):

DEBUG server comment - cmd: play

INFO server comment - MediaStreamMediaCasterPlay: startPlay

INFO server comment - RTPMediaCaster.create

INFO server comment - RTPMediaCaster.init

DEBUG server comment - cmd: setBufferTime

INFO server comment - RTPMediaCaster.Reconnector: start

INFO server comment - RTPSessionDescriptionDataProviderBasic.getStreamInfo: /usr/local/WowzaMediaServerPro/content/myStream.sdp

DEBUG server comment - sdp: v=0

DEBUG server comment - sdp: o=- 1188340656180883 1 IN IP4 127.0.0.1

DEBUG server comment - sdp: s=Session streamed by GStreamer

DEBUG server comment - sdp: i=server.sh

DEBUG server comment - sdp: t=0 0

DEBUG server comment - sdp: a=tool:GStreamer

DEBUG server comment - sdp: a=type:broadcast

DEBUG server comment - sdp: m=video 5000 RTP/AVP 96

DEBUG stream setbuffertime myStream.sdp -

DEBUG server comment - sdp: c=IN IP4 127.0.0.1

DEBUG server comment - sdp: a=rtpmap:96 H264/90000

DEBUG server comment - onFlushNotifyClients: false

DEBUG server comment - flushInterval: 0

DEBUG server comment - verboseDebug: false

INFO stream create - -

DEBUG server comment - RTPDePacketizerRFC3984H264.init

INFO stream publish myStream.sdp -

INFO server comment - UDPTransport.bind: /127.0.0.1:5000

DEBUG server comment - config: session: setReuseAddress: from:false to:true

DEBUG server comment - config: session: setReceiveBufferSize: from:55296 to:65000

DEBUG server comment - config: session: setSendBufferSize: from:55296 to:55296

DEBUG server comment - config: session: setTrafficClass: from:0 to:0

INFO server comment - UDPTransport.bind: /127.0.0.1:5001

DEBUG server comment - config: session: setReuseAddress: from:false to:true

DEBUG server comment - config: session: setReceiveBufferSize: from:55296 to:65000

DEBUG server comment - config: session: setSendBufferSize: from:55296 to:55296

DEBUG server comment - config: session: setTrafficClass: from:0 to:0

INFO server comment - RTPMediaCaster.Reconnector: stop

DEBUG server comment - rtp[video:1024] {80 60 11 af 25 e9 6f d6 e3 2f 49 73 1c 89 30 00 }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:1

INFO server comment - UDPTransport.firstPacket: /127.0.0.1:5000

DEBUG server comment - rtp[video:286] {80 60 11 b0 25 e9 6f d6 e3 2f 49 73 1c 49 c7 bf }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:2

DEBUG server comment - rtp[video:1024] {80 60 11 b1 25 e9 7b 8f e3 2f 49 73 1c 89 10 00 }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:3

DEBUG server comment - rtp[video:990] {80 60 11 b2 25 e9 7b 8f e3 2f 49 73 1c 49 de 0a }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:4

DEBUG server comment - rtp[video:1024] {80 60 11 b3 25 e9 87 46 e3 2f 49 73 1c 89 30 00 }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:5

DEBUG server comment - rtp[video:148] {80 60 11 b4 25 e9 87 46 e3 2f 49 73 1c 49 49 9f }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:6

DEBUG server comment - rtp[video:1024] {80 60 11 b5 25 e9 92 fe e3 2f 49 73 1c 89 30 00 }

DEBUG server comment - myStream.sdp: 17:36:15: waitforend: dropped:7

DEBUG server comment - rtp[video:177] {80 60 11 b6 25 e9 92 fe e3 2f 49 73 1c 49 7a 2f }

Of course this won't be an easy task, yet i'm wondering what this log mean ? Why are all frames dropped ?

Cheers

Florent

Answer 1

Answer by Charlie Good · Jul 16, 2008 at 04:07 PM

Also, describe how to install gstreamer. I can't get it to work with x264enc. I am on Fedora6.

Charlie

0 Show 1 · Share

seth piezas · Jul 17, 2008 at 02:34 AM 0

Share

Thanks.

I'm on ubuntu hardy 8.0.4 right now. To install, use

apt-get install

gstreamer0.10-plugins-good gstreamer0.10-plugins-base gstreamer0.10-plugins-bad-multiverse gstreamer-tools gstreamer0.10-ffmpeg gstreamer-tools

that should get all the necessary parts. the gst plugin for x264 resides in the bad-multiverse install.

I'll check which version of the player is being used (whatever debug version came with the flash install). I just installed the most recent patch for the wowza server. And I'll send a debug log soon.

The setup has isolated the variation to the RTP stream alone. RTSP announce is handled through a separate process. And wirecast or quicktime broadcaster in unicast mode work. When the rtp stream is changed to gstreamer, it seems to zero out the packets for all but the first.

More soon.

Answer 2

Answer by Charlie Good · Aug 13, 2008 at 04:04 AM

I really don't have any new information here. The format of the sprop-parameter-sets looks OK to me. I think there is an issue with Wowza Pro not being able to recognize keys frames in the stream coming from gstreamer.

Charlie

0 Show 1 · Share

Florent Thiery · Aug 13, 2008 at 05:11 PM 0

Share

I doubt it's the codec that's faulty, since it's x264 like VLC. Would the rtp payloading be the problem ?

Do you have example h264 raw rtp sdp files for inspiration ?

Thanks

Florent

Answer 3

Answer by Charlie Good · Nov 26, 2008 at 08:19 AM

I have sent you an email directly. Wowza Pro is not receiving UDP packets from the encoder. So you need to sort this out first.

Charlie

0 Show 1 · Share

Ponomarev Mikhail · Dec 12, 2008 at 09:29 PM 0

Share

i'm trying to stream H.264 AVI file to Wowza via RTP using GStreamer.

Wowza Version: WowzaMediaServerPro-1.6.0 build9437

Operation System: Windows Vista

Gstreamer version: 0.10

Gstreamer command line string:

-------------------------------------------------------

gst-launch-0.10.exe -v filesrc location=avc.avi ! avidemux ! rtph264pay mtu=1438 ! udpsink host=192.168.3.13 port=6000

-------------------------------------------------------

Wowza .sdp file:

-------------------------------------------------------

v=0

o=- 1188340656180883 1 IN IP4 192.168.3.13

s=Session streamed by GStreamer

i=server.sh

t=0 0

a=tool:GStreamer

a=type:broadcast

m=video 6000 RTP/AVP 96

c=IN IP4 192.168.3.13

a=rtpmap:96 H264/90000

a=cliprect:0,0,240,320

a=framesize:96 320-240

-------------------------------------------------------

log file:

-------------------------------------------------------

- _definst_ rtplive/_definst_

- - 127.0.0.1 -

- 127.0.0.1 -

- - -

MediaStreamMediaCasterPlay: startPlay

RTPMediaCaster.create

RTPMediaCaster.init

RTPMediaCaster.Reconnector: start

RTPSessionDescriptionDataProviderBasic.getStreamInfo: C:/Program Files/Wowza Media Systems/Wowza Media Server Pro 1.6.0/content/test264_3.sdp

-

test264_3.sdp -

UDPTransport.bind: /192.168.3.13:6000

UDPTransport.bind: /192.168.3.13:6001

RTPMediaCaster.Reconnector: stop

UDPTransport.firstPacket: /192.168.3.13:6000

-------------------------------------------------------

and nothing happens, no video :(

Can you help me with this problem?

Answer 4

Answer by Charlie Good · Dec 22, 2008 at 08:48 AM

I don't think you are transcoding the content to H.264 in your gstreamer command. It needs to be somethine like:

gst-launch-0.10.exe -v filesrc location=videotestsrc_h264.avi ! avidemux ! x264enc byte-stream=true bitrate=300 ! rtph264pay mtu=1438 ! udpsink host=10.218.35.135 port=1234

Charlie

0 Show 1 · Share

Ponomarev Mikhail · Dec 22, 2008 at 07:53 PM 0

Share

Charlie, i don't have to transcode, my file is already in H.264, ant it streams perfectly with VLC without transcoding directly to wowza. And Gstreamer RTP stream playing fine with VLC player.

Answer 5

Answer by Charlie Good · Jan 06, 2009 at 05:01 AM

I have been looking at this and I am pretty sure the bug is with gstreamer. There is code in the rtph264pay system that is searching for H.264 start codes. It is looking for start codes in the format:

0x00000001

Start codes in H.264 are only three bytes long. Like this:

0x000001

So it is not properly chopping up the NAL units to send them over RTP. Some RTP handlers can handle this. We cannot. I am hesistant to add support for this since it does not seem to be the standard way to do it. I was planning on attempting to fix the problem in gstreamer to validate my findings but have not had time to get a working gstreamer development setup.

Charlie

0 Show 4 · Share

Charlie Good · Jan 06, 2009 at 05:23 AM 0

Share

I made a change that enabled it to work when using the test pattern. So in the most recent version the following gstreamer command line works:

gst-launch-0.10 -vvv videotestsrc ! queue ! x264enc byte-stream=true bitrate=300 ! rtph264pay ! udpsink port=5000 host=127.0.0.1 sync=false

This continues to work for me. I have tried gstreamer in other situations where it does not work due to the problems I described above.

Charlie

Florent Thiery Charlie Good · Jan 06, 2009 at 03:11 PM 0

Share

Hi charlie :)

Regarding the development environment:

http://www.pitivi.org/wiki/GStreamer_CVS_Setup_Page

What do you mean by "latest version" : 0.10.21 ?

Florent Thiery Florent Thiery · Jan 06, 2009 at 03:21 PM 0

Share

I confirm the test pattern works !!! This is awesome !

So finally you did add support for the different RTP handling ?

Florent Thiery Florent Thiery · Jan 06, 2009 at 03:23 PM 0

Share

I'm just trying to understand why this works with the test pattern but not other cases: with the test pattern + x264enc, the mark is correct ?

Answer 6

Answer by Charlie Good · Jan 06, 2009 at 07:28 AM

Thanks for the development instructions. I will try to get back to this to figure out the problem.

I did make a change that fixes the test pattern but it does not seem to universally fix the issue. I have since found other gstreamer setups that do not work even with this fix in place. Again, it has to do with the way gstreamer is breaking apart (or not breaking part) the H.264 packets. I can seem that VLC is able to stream data in this format but it just doesn't seem correct to me.

Charlie

0 Show 1 · Share

Florent Thiery · Jan 06, 2009 at 03:38 PM 0

Share

Okay;

1) do you want me to report the issue to the development list ? ( https://lists.sourceforge.net/lists/listinfo/gstreamer-devel)

2) i tested with the dev env (0.10.21), and THIS WORKS PERFECTLY !!! For reference, we use the FastVDO SmartCapture device with a dedicated gstreamer plugin

This is great :)

Thanks again !

FLorent

Answer 7

Answer by Charlie Good · Jan 06, 2009 at 09:07 AM

So I am confused. Is it all working now with the most recent version of gstreamer?

Charlie

0 Show 1 · Share

Ponomarev Mikhail · Jan 19, 2009 at 07:30 PM 0

Share

Charlie, i'll try it with new version of GStreamer too.

Can i use 1.6.0 Server Version? Do i need to install some updates or patches?

Florent, can you demonstrate working GSTreamer commnad line and SDP file please?

Thanks in advance!

Answer 8

Answer by Charlie Good · Jan 19, 2009 at 09:53 AM

Upgrade the most recent patch:

http://www.wowza.com/devbuild.html

I believe it only works with the most recent version of GStreamer.

Charlie

0 Show 5 · Share

Florent Thiery · Jan 21, 2009 at 04:51 PM 0

Share

Hi Charlie,

I am now looking for a way to send MP3 or AAC data with gstreamer as well:

AAC:

gst-launch audiotestsrc ! faac ! rtpmp4apay ! udpsink host=$server port=$audio_port

Here is the sdp audio part i've been using

m=audio 10002 RTP/AVP 96

a=rtpmap:96 mpeg4-generic/48000/2

a=control:trackID=2

a=fmtp:96 streamtype=5; profile-level-id=255; mode=AAC-hbr; config=1190; objectType=64; sizeLength=13; indexLength=3; indexDeltaLength=3

The server raises exceptions:

ERROR server comment - addDataA[this.size:513 this.dataA.length:513 this.startDataLoc:512 this.dataLoc:512 data.length:303 offset:0 size:303 missing:1 ]: java.lang.ArrayIndexOutOfBoundsException

java.lang.ArrayIndexOutOfBoundsException

at com.wowza.wms.amf.AMFPacket.addData(Unknown Source)

at com.wowza.wms.stream.live.LiveReceiver.addAudioData(Unknown Source)

at com.wowza.wms.stream.live.MediaStreamLive.addAudioData(Unknown Source)

at com.wowza.wms.rtp.depacketizer.RTPPacket.write(Unknown Source)

at com.wowza.wms.rtp.depacketizer.RTPPacket.write(Unknown Source)

at com.wowza.wms.rtp.depacketizer.RTPDePacketizerMPEG4AAC.handleRTPPacket(Unknown Source)

Given gstreamer's detailed output (below) how should the SDP file be ?

caps = application/x-rtp, media=(string)audio, clock-rate=(int)48000, encoding-name=(string)MP4A-LATM, cpresent=(string)0, config=(string)40001320, payload=(int)96, ssrc=(guint)3653097729, clock-base=(guint)2206318833, seqnum-base=(guint)48474

MP3:

gst-launch audiotestsrc ! lame ! rtpmpapay ! udpsink host=$server port=$audio_port

I do not know how to tell the server in the sdp file (if supported) that audio is MP3. Do you have sample SDP files for MP3 rtp streams ?

THanks

Florent Thiery Florent Thiery · Jan 21, 2009 at 05:05 PM 0

Share

Here's what i'm trying:

m=audio 10002 RTP/AVP 96

a=rtpmap:96 mpa-robust/90000

And the corresponding gst caps:

/GstPipeline:pipeline0/GstRtpMPAPay:rtpmpapay0.GstPad:src: caps = application/x-rtp, media=(string)audio, clock-rate=(int)90000, encoding-name=(string)MPA, payload=(int)96, ssrc=(guint)1315959326, clock-base=(guint)2051349402, seqnum-base=(guint)41023

/GstPipeline:pipeline0/GstRtpMPAPay:rtpmpapay0.GstPad:sink: caps = audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)3, channels=(int)2, rate=(int)48000

Florent Thiery Florent Thiery · Jan 21, 2009 at 05:53 PM 0

Share

From http://www.rfc-editor.org/rfc/rfc3016.txt:

5.4 SDP usage of MPEG-4 Audio

The MIME media type audio/MP4A-LATM string is mapped to fields in the

Session Description Protocol (SDP), RFC 2327, as follows:

o The MIME type (audio) goes in SDP "m=" as the media name.

o The MIME subtype (MP4A-LATM) goes in SDP "a=rtpmap" as the

encoding name.

o The required parameter "rate" goes in "a=rtpmap" as the clock

rate.

o The optional parameter "ptime" goes in SDP "a=ptime" attribute.

o The optional parameter "profile-level-id" goes in the "a=fmtp"

line to indicate the coder capability. The "object" parameter

goes in the "a=fmtp" attribute. The payload-format-specific

parameters

Here's what i am trying for AAC:

m=audio 10002 RTP/AVP 96

a=rtpmap:96 MP4A-LATM/48000/2

Where gst caps are:

caps = application/x-rtp, media=(string)audio, clock-rate=(int)48000, encoding-name=(string)MP4A-LATM, cpresent=(string)0, config=(string)40001320, payload=(int)96, ssrc=(guint)3006373333, clock-base=(guint)1009694188, seqnum-base=(guint)27071

Problem is, if i do this the video doesn't play back anymore ! (Black screen). No errors though...

Florent Thiery Florent Thiery · Jan 21, 2009 at 06:05 PM 0

Share

Here is my latest sdp audio section test:

m=audio 10002 RTP/AVP 96

a=rtpmap:96 MP4-LATM/48000/2

a=fmtp:96 profile-level-id=1; bitrate=128000; cpresent=0;config=40001320

Here are the default faac parameters:

outputformat : Format of output frames

flags: accÃ¨s en lecture, accÃ¨s en Ã©criture

Enum "GstFaacOutputFormat" Default: 1, "ADTS headers" Current: 0, "Raw AAC"

(0): Raw AAC - OUTPUTFORMAT_RAW

(1): ADTS headers - OUTPUTFORMAT_ADTS

bitrate : Bitrate in bits/sec

flags: accÃ¨s en lecture, accÃ¨s en Ã©criture

Integer. Range: 8000 - 320000 Default: 128000 Current: 128000

profile : MPEG/AAC encoding profile

flags: accÃ¨s en lecture, accÃ¨s en Ã©criture

Enum "GstFaacProfile" Default: 1, "Main profile" Current: 1, "Main profile"

(1): Main profile - MAIN

(2): Low complexity profile - LC

(3): Scalable sampling rate profile - SSR

(4): Long term prediction profile - LTP

tns : Use temporal noise shaping

flags: accÃ¨s en lecture, accÃ¨s en Ã©criture

Boolean. Default: false Current: false

midside : Allow mid/side encoding

flags: accÃ¨s en lecture, accÃ¨s en Ã©criture

Boolean. Default: true Current: true

shortctl : Block type encorcing

flags: accÃ¨s en lecture, accÃ¨s en Ã©criture

Enum "GstFaacShortCtl" Default: 1, "No short blocks" Current: 0, "Normal block type"

(0): Normal block type - SHORTCTL_NORMAL

(1): No short blocks - SHORTCTL_NOSHORT

(2): No long blocks - SHORTCTL_NOLONG

It seems to me everything should be alright here (according to RFC). Does anyone see any obvious mistake ?

Florent Thiery Florent Thiery · Jan 26, 2009 at 10:45 AM 0

Share

Hi

Just found the solution: you have to use the rtpmp4gpay plugin which reports the appropriate parameters to fill in the sdp file; and it works !!!

However, if the flash player is left open for some time, audio and video are desynced.

Why is that ?

Is there a way to avoid it (e.g. using RTSP/RTCP/mpegts) ?

Answer 9

Answer by Charlie Good · Jan 26, 2009 at 04:17 AM

How long does it take for it to go out of sync? RTCP is used to synchronize the video and audio but is only needed to initially sync the stream. MPEG-TS synchronizes the audio and video in the stream (all streams run on a 90KHz clock and are pre-synchronized). Not sure why your stream is going out of sync.

Charlie

0 Show 1 · Share

Florent Thiery · Jan 26, 2009 at 04:20 PM 0

Share

Well, it's hard to say: it's perceivable after 20 minutes or so.

1) why do i need to restart the server after stopping and restarting the (publisher) streaming process ? Is there a workaround ? If i do not, the flash player only shows a black screen

2) audio plays back faster

3) my audio tests samples (voice/claps) seem to have a direct effect on drifting speed ! If i speak longer, the desync comes faster ! :mad:

Streamer: gstreamer-0.10.21 (cvs)

Wowza: 1.5.3 build7825

Client: Flash Player 10 (Vista)

Answer 10

1) why do i need to restart the server after stopping and restarting the (publisher) streaming process ? Is there a workaround ? If i do not, the flash player only shows a black screen

It is because when the encoder starts/stop it changes the time basis which screws up Wowza Pro and Flash. There are two methods to work around this. Take a look at the sdpFileCheckFreqency and streamTimeout properties as discussed in the Native Real-time Transport Protocol (RTP) Streaming section of the User's Guide. You can setup Wowza Pro to monitor the SDP file for date or size changes ( sdpFileCheckFreqency) so the stream is restarted when the SDP file changes. You can instruct Wowza Pro monitor the stream for long breaks ( streamTimeout) and restart the stream if needed. The streamTimeout is tricky to get right since if it is too short it will restart the stream when it first tries to connect if it can't find a key frame quickly enough.

2) audio plays back faster

3) my audio tests samples (voice/claps) seem to have a direct effect on drifting speed ! If i speak longer, the desync comes faster !

This is a tough one. I just don't have the resources to dig into gstreamer and debug something like this. I know there are plenty of RTP based encoders where there is no issue with AV sync. So it is most likely not a Wowza Pro issue. There may be rounding issues in GStreamer. I have seen this problem in other encoders where they don't deal with timecodes properly and they calculate time differentials which insert small rounding errors that add up over time.

Charlie

Using gstreamer-0.10 with Wowza

30 Replies

Your answer

Follow this Question

Related Questions

Hot Topics

Product Sign-in

Under the Hood

Resellers

Company

Stay Connected

Get Monthly Newsletter

Select a Language