Results 1 to 4 of 4

Thread: "Could not append buffer.getString(decoder)" from within serveMsg()

  1. #1
    Join Date
    Dec 2009
    Posts
    17

    Default "Could not append buffer.getString(decoder)" from within serveMsg()

    Hey guys,

    I'm using an overloaded version of serviceMsg() to modify HLS playlists a they're delivered; specifically for adding program timing and for adding CORS headers. In essence I have this:

    public class ModuleVisualplatformStreamCupertinoPlaylistHandler extends HTTPStreamerAdapterCupertinoStreamer {
    public void serviceMsg(long timestamp, IoSession ioSession, RtmpRequestMessage req, RtmpResponseMessage resp) {
    super.serviceMsg(timestamp, ioSession, req, resp);
    // Add CORS header
    resp.setHeader("Access-Control-Allow-Origin", "*");
    // Modify playlist
    decorateWithProgramDate()
    }
    }

    To read out the playlist from the response, I'm doing this to decode the response body:

    List<ByteBuffer> bufferlist = resp.getBodyList();
    Charset charset = Charset.forName("UTF-8");
    CharsetDecoder decoder = charset.newDecoder();
    StringBuffer originalData = new StringBuffer();
    int i = 0;
    for (ByteBuffer buffer : bufferlist) {
    try {
    originalData.append(buffer.getString(decoder));
    i++;
    } catch (Exception e){
    log("Could not append buffer.getString(decoder) to originalData for stream, " + i + "/" + bufferlist.size());
    e.printStackTrace();
    }
    }

    This however results in an exception in a limited set of cases:
    - It always works for non-DVR request.
    - When a new stream start, it works well for DVR requests as well.
    - After "a while" (say, 30 minutes, but this is not consistent) DVR requests fire an exception (see below).
    - If I restart the same stream, requests return without exception.

    Crucially, the requests themselves return data to the client even if there's an exception on the way (a full list of all relevant chunks, though of course without the lines I'm trying to add).

    The exception is:
    2014-08-18_15:21:27.21809 INFO server comment - Playlist Handler: Could not append buffer.getString(decoder) to originalData for stream, 0/1
    2014-08-18_15:21:27.21847 java.nio.charset.MalformedInputException: Input length = 1
    2014-08-18_15:21:27.21876 at java.nio.charset.CoderResult.throwException(CoderResult.java:277)
    2014-08-18_15:21:27.21893 at org.apache.mina.common.ByteBuffer.getString(ByteBuffer.java:1022)
    2014-08-18_15:21:27.21909 at com.wowza.wms.plugin.visualplatform.ModuleVisualplatformStreamCupertinoPlaylistHandler.decorateWithP rogramDate(ModuleVisualplatformStreamCupertinoPlaylistHandler.java:77)
    2014-08-18_15:21:27.21925 at com.wowza.wms.plugin.visualplatform.ModuleVisualplatformStreamCupertinoPlaylistHandler.serviceMsg(Mo duleVisualplatformStreamCupertinoPlaylistHandler.java:57)
    2014-08-18_15:21:27.21942 at com.wowza.wms.httpstreamer.cupertinostreaming.httpstreamer.HTTPStreamerAdapterCupertinoStreamer.serv ice(HTTPStreamerAdapterCupertinoStreamer.java:422)
    2014-08-18_15:21:27.21958 at com.wowza.wms.server.ServerHandler.a(ServerHandler.java:636)
    2014-08-18_15:21:27.21974 at com.wowza.wms.server.ServerHandler.a(ServerHandler.java:375)
    2014-08-18_15:21:27.21991 at com.wowza.wms.server.ServerHandler.messageReceived(ServerHandler.java:487)
    2014-08-18_15:21:27.22007 at com.wowza.wms.server.ServerHandlerThreaded.messageReceived(ServerHandlerThreaded.java:78)
    2014-08-18_15:21:27.22023 at org.apache.mina.common.support.AbstractIoFilterChain$2.messageReceived(AbstractIoFilterChain.java:18 1)
    2014-08-18_15:21:27.22039 at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(AbstractIoFilterChain.j ava:511)
    2014-08-18_15:21:27.22055 at org.apache.mina.common.support.AbstractIoFilterChain.access$900(AbstractIoFilterChain.java:42)
    2014-08-18_15:21:27.22071 at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(AbstractIoFilterCha in.java:786)
    2014-08-18_15:21:27.22087 at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimpleProtocolDecoderOutput.j ava:60)
    2014-08-18_15:21:27.22104 at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:177)
    2014-08-18_15:21:27.22120 at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(AbstractIoFilterChain.j ava:511)
    2014-08-18_15:21:27.22137 at org.apache.mina.common.support.AbstractIoFilterChain.access$900(AbstractIoFilterChain.java:42)
    2014-08-18_15:21:27.22153 at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(AbstractIoFilterCha in.java:786)
    2014-08-18_15:21:27.22173 at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java:247)
    2014-08-18_15:21:27.22189 at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(ExecutorFilter.java:323)
    2014-08-18_15:21:27.22205 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    2014-08-18_15:21:27.22222 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    2014-08-18_15:21:27.22239 at java.lang.Thread.run(Thread.java:679)


    Hopefully, the above is enough of a description to trigger good ideas. I really do appreciate the help.

  2. #2
    Join Date
    May 2013
    Posts
    680

    Default

    The MalformedInputException would suggest that the value in the buffer may be invalid. Maybe you could put the bytes from the buffer into the string with a specific encoding (utf8 etc) and see if that resolves the issue.

  3. #3
    Join Date
    Dec 2009
    Posts
    17

    Default

    I've been digging a bit deeper into this, and it turns out to be fairly reproduceable: The problem occurs when the size of the manifest (and thus the bodyList I guess) is larger than 4000 bytes.

    You're right that the specific error occurred because the content is not being decoded correct -- but that seem like a symptom of the fact that the content changes after that 4000th byte is added.

    For debugging, I've tried simply copying the bytes into a string and logging them. Before the error occurs, this yield the full chunklist. Afterwards though, a variation of this is the output:
    2014-09-11_00:21:58.23530 INFO server comment - Live Playlist Handler: ??OK?@?????????in?]???????RQPP<???Z(?Y?????df~?f6???W???????n?X?????r}_???m???y?o?????]????y}???????mjg?.O????6?x5z{z|9_????]?????UY????;????j???(??a????a?\?????A?^?_dX?l#?mE?~??????7?????!?i?~E?#?&??6D??y???
    ????v0?l9?VQ??c????2???)??2` KFm?
    ??!)?\???S??a??2?Yub??p?h7#hA????eX/?j????37?/D??)*?F!?|?8??37???>?!e??t?F'?????~

    Struggling for ideas here so any help is appreciated.

  4. #4
    Join Date
    Dec 2009
    Posts
    17

    Default

    Problem here was gzip encoding of the response; by default dvr responses are set to be gzip-encoded exactly after the 4000th byte. This is controlled by `dvrCupertinoPlaylistGzipThreshold` and `dvrCupertinoPlaylistUseGzip`. Setting the latter to false resolved this issue.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •