After the fixes to CVE-2026-90559 I have noticed that sometimes compressing with SnappyHadoopCompatibleOutputStream fails with the exception IllegalArgumentException("not enough space for output: need %,d bytes, but only %,d remaining");
I have the following unit test to prove this:
package org.xerial.snappy;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.RandomUtils;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.io.compress.CompressionInputStream;
import org.apache.hadoop.io.compress.SnappyCodec;
import org.junit.Assert;
import org.junit.Test;
import java.io.FileOutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
public class HadoopTest {
@Test
public void test() throws Exception{
SnappyCodec hadoopCodec = new SnappyCodec();
hadoopCodec.setConf(new Configuration());
RandomUtils rnd = RandomUtils.insecure();
for(int i=0; i < 99_000; i++){
int size = rnd.randomInt(5_120, 512_000); // 5kb to 500kb
byte[] input = rnd.randomBytes(size);
Path tmp = Files.createTempFile("test", "test");
FileOutputStream fileOutputStream = new FileOutputStream(tmp.toFile());
SnappyHadoopCompatibleOutputStream snappyOutput = new SnappyHadoopCompatibleOutputStream(fileOutputStream);
snappyOutput.write(input);
snappyOutput.flush();
CompressionInputStream snappyInput = hadoopCodec.createInputStream(Files.newInputStream(tmp));
byte[] output = new byte[size];
IOUtils.read(snappyInput, output);
Assert.assertArrayEquals(input,output);
Files.deleteIfExists(tmp);
}
}
}
I did find a fix to this by changing SnappyOutputStream::hasSufficientOutputBufferFor
It had the magic number of -4. I changed it to -7 and all tests passed.
private boolean hasSufficientOutputBufferFor(int inputSize)
{
int maxCompressedSize = Snappy.maxCompressedLength(inputSize);
// I changed the magic number of 4 to 7
return maxCompressedSize < outputBuffer.length - outputCursor - 7;
}
I would put a PR about this, but I did not want to until I understand what -4 & -7 means.
I suspect it's something to do with hadoop not having a header but I'm not sure.
Any help would be welcome.
After the fixes to CVE-2026-90559 I have noticed that sometimes compressing with SnappyHadoopCompatibleOutputStream fails with the exception IllegalArgumentException("not enough space for output: need %,d bytes, but only %,d remaining");
I have the following unit test to prove this:
I did find a fix to this by changing SnappyOutputStream::hasSufficientOutputBufferFor
It had the magic number of -4. I changed it to -7 and all tests passed.
I would put a PR about this, but I did not want to until I understand what -4 & -7 means.
I suspect it's something to do with hadoop not having a header but I'm not sure.
Any help would be welcome.