We're using Adaptive HLS streaming for a VOD site and we're currently trying to optimize our encoding, testing different bit rates and frame sizes and so on. We're finding that once we drop the frame size down to a certain point, the segments/chunks become misaligned with the other encodes. For instance, in our current test almost all encodes chunk out perfectly at 3 seconds per chunk, but the chunk sizes bounce around for the encode with the smallest frame size. They aren't exactly 3 seconds like the other encodes. Is this normal? Will this cause the playback to break if/when the player switches to that file, or will it just cause the video to stutter a little?

My colleague who is doing all the encoding is using Compressor. He's tried everything he can think of to get the chunk sizes to be exactly the same across all files, but once he lowers the frame size to a certain point, they just don't align. Any insight into why, and the consequences of having different chunk sizes, is appreciated.