Wowza Community

AAC packets from packet listener needs decoding before sending to another service. Unusable byte data

I’ve been researching ways how to ingest AAC / opus packets through the packet listener and send them for ingest to another service.

The opus transcoder is containlerless, and needs to be wrapped in an ogg container to be read properly it took ages to figure out why that was failing. So after that failure ive tried passing the AAC packets from the RTMP encoder through ffmpeg to create the ogg/opus. And the incoming AAC packets are corrupted when just recording them. If I pull an rtmp stream into ffmpeg to grab frames it works. The Opus output was corrupting also and wont even play back in VLC.

There is hidden undocumented apis in wowza to work with opus and AAC that might be useful but are hidden and not useful for developers.

I now know there is 2 byte headers in the Opus packet that needs to be stripped but even doing so the service fails to read them because its containless. And trying to wrap them with a Java ogg/opus library failed.

What special treatment do I need to do with the AAC packets to get the raw samples into a ByteBuffer for processing and ingest into ffmpeg to be readable by the service ? The output required is a 16khz 24kps mono output.

The code looks like this. I need a way to detect the input samplerate and channels also. The output of this is noisy and corrupted but at least it plays.

We thought just getting the output bytes direct from the transcoder would work but they are unusable without further processing. So may aswell pass them to ffmpeg to make them usable. But doing so the raw bytes from the audio packet is still unusable without further treatment also. And there is no documentation what it needs.

public void onLivePacket(IMediaStream stream, AMFPacket packet)

{

    //WMSLoggerFactory.getLogger(CLASS).info("Packet Buffer " +  packet.getData());

   

    final Object lock = new Object();

           

    synchronized(lock)

    {

        if (packet.isAudio()) {

           

            try {

                byte[] data = packet.getData();

                                                   

                ByteBuffer buffer = ByteBuffer.wrap(data);

               

                WMSLoggerFactory.getLogger(CLASS).info("Audio Packet Buffer " + buffer);

           

                //need input samplerate and channels

                this.recorder.recordSamples(48000, 2, buffer);

                //this.recorder.recordSamples(48000, 2, packet.getDataBuffer());

            } catch (Exception e) {

                e.printStackTrace();

               

                WMSLoggerFactory.getLogger(CLASS).error("Error: " + e.getMessage());

            }

        }

    }

   

}

Hmmmm…that’s very interesting Daniel, but a quick ask here at Wowza resulted in nobody really knowing much about that. Some of the devs that wrote that are no longer here. If nobody else in the community replies, feel free to send a ticket and maybe one of the support engineers will have an idea what to try.

Might be worth asking in our public slack too- so many advanced developers and engineers in there from all different companies and backgrounds.

Its a tricky task. Its for live speech 2 text. The packets need to be sent to IBM. But none of these services can read the raw packets. They need to be decoded and wrapped properly.

One support said to take 2 bytes out of the Opus packet which is the header. But it still couldn’t be read. Because it needs to be in a webm/ogg container. Decoding through ffmpeg works. It needs something done with the bytes but Im still not sure what. If I can take in the raw AAC from RTMP FLV and encode to ogg opus that would be good. Or have a usable Opus output from the transcoder.

Wowza is doing something internally with the aac packets as I can see undocumented java utils

In my research my problem seems to be the AAC packet needs to be transcoded to PCM or converted somehow with a utility first to be usable. I will ask on Slack about the Opus problem from the transcoder its not usable as a raw packet. Maybe wowza already has a utility for decoding AAC to PCM just not documented.

I’m not getting a notification to join slack using my email… I believe in the java imports. this utility might be what I need so I can feed it through ffmpeg without problems. Its not documented what offsets is needed.

AACUtils.codecConfigToWaveFormatEX(data, 0, 0)