Class Snappy
- java.lang.Object
-
- io.netty.handler.codec.compression.Snappy
-
public final class Snappy extends java.lang.Object
Uncompresses an inputByteBuf
encoded with Snappy compression into an outputByteBuf
. See snappy format.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description private static class
Snappy.State
-
Field Summary
Fields Modifier and Type Field Description private static int
COPY_1_BYTE_OFFSET
private static int
COPY_2_BYTE_OFFSET
private static int
COPY_4_BYTE_OFFSET
private static int
LITERAL
private static int
MAX_HT_SIZE
private static int
MIN_COMPRESSIBLE_BYTES
private static int
NOT_ENOUGH_INPUT
private static int
PREAMBLE_NOT_FULL
private Snappy.State
state
private byte
tag
private int
written
-
Constructor Summary
Constructors Constructor Description Snappy()
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description private static int
bitsToEncode(int value)
Calculates the minimum number of bits required to encode a value.(package private) static int
calculateChecksum(ByteBuf data)
Computes the CRC32C checksum of the supplied data and performs the "mask" operation on the computed checksum(package private) static int
calculateChecksum(ByteBuf data, int offset, int length)
Computes the CRC32C checksum of the supplied data and performs the "mask" operation on the computed checksumvoid
decode(ByteBuf in, ByteBuf out)
private static int
decodeCopyWith1ByteOffset(byte tag, ByteBuf in, ByteBuf out, int writtenSoFar)
Reads a compressed reference offset and length from the supplied input buffer, seeks back to the appropriate place in the input buffer and writes the found data to the supplied output stream.private static int
decodeCopyWith2ByteOffset(byte tag, ByteBuf in, ByteBuf out, int writtenSoFar)
Reads a compressed reference offset and length from the supplied input buffer, seeks back to the appropriate place in the input buffer and writes the found data to the supplied output stream.private static int
decodeCopyWith4ByteOffset(byte tag, ByteBuf in, ByteBuf out, int writtenSoFar)
Reads a compressed reference offset and length from the supplied input buffer, seeks back to the appropriate place in the input buffer and writes the found data to the supplied output stream.(package private) static int
decodeLiteral(byte tag, ByteBuf in, ByteBuf out)
Reads a literal from the input buffer directly to the output buffer.void
encode(ByteBuf in, ByteBuf out, int length)
private static void
encodeCopy(ByteBuf out, int offset, int length)
Encodes a series of copies, each at most 64 bytes in length.private static void
encodeCopyWithOffset(ByteBuf out, int offset, int length)
(package private) static void
encodeLiteral(ByteBuf in, ByteBuf out, int length)
Writes a literal to the supplied output buffer by directly copying from the input buffer.private static int
findMatchingLength(ByteBuf in, int minIndex, int inIndex, int maxIndex)
Iterates over the supplied input buffer between the supplied minIndex and maxIndex to find how long our matched copy overlaps with an already-written literal value.private static short[]
getHashTable(int inputSize)
Creates an appropriately sized hashtable for the given input sizeprivate static int
hash(ByteBuf in, int index, int shift)
Hashes the 4 bytes located at index, shifting the resulting hash into the appropriate range for our hash table.(package private) static int
maskChecksum(long checksum)
From the spec: "Checksums are not stored directly, but masked, as checksumming data and then its own checksum can be problematic.private static int
readPreamble(ByteBuf in)
Reads the length varint (a series of bytes, where the lower 7 bits are data and the upper bit is a flag to indicate more bytes to be read).void
reset()
(package private) static void
validateChecksum(int expectedChecksum, ByteBuf data)
Computes the CRC32C checksum of the supplied data, performs the "mask" operation on the computed checksum, and then compares the resulting masked checksum to the supplied checksum.(package private) static void
validateChecksum(int expectedChecksum, ByteBuf data, int offset, int length)
Computes the CRC32C checksum of the supplied data, performs the "mask" operation on the computed checksum, and then compares the resulting masked checksum to the supplied checksum.private static void
validateOffset(int offset, int chunkSizeSoFar)
Validates that the offset extracted from a compressed reference is within the permissible bounds of an offset (0 < offset < Integer.MAX_VALUE), and does not exceed the length of the chunk currently read so far.
-
-
-
Field Detail
-
MAX_HT_SIZE
private static final int MAX_HT_SIZE
- See Also:
- Constant Field Values
-
MIN_COMPRESSIBLE_BYTES
private static final int MIN_COMPRESSIBLE_BYTES
- See Also:
- Constant Field Values
-
PREAMBLE_NOT_FULL
private static final int PREAMBLE_NOT_FULL
- See Also:
- Constant Field Values
-
NOT_ENOUGH_INPUT
private static final int NOT_ENOUGH_INPUT
- See Also:
- Constant Field Values
-
LITERAL
private static final int LITERAL
- See Also:
- Constant Field Values
-
COPY_1_BYTE_OFFSET
private static final int COPY_1_BYTE_OFFSET
- See Also:
- Constant Field Values
-
COPY_2_BYTE_OFFSET
private static final int COPY_2_BYTE_OFFSET
- See Also:
- Constant Field Values
-
COPY_4_BYTE_OFFSET
private static final int COPY_4_BYTE_OFFSET
- See Also:
- Constant Field Values
-
state
private Snappy.State state
-
tag
private byte tag
-
written
private int written
-
-
Method Detail
-
reset
public void reset()
-
hash
private static int hash(ByteBuf in, int index, int shift)
Hashes the 4 bytes located at index, shifting the resulting hash into the appropriate range for our hash table.- Parameters:
in
- The input buffer to read 4 bytes fromindex
- The index to read atshift
- The shift value, for ensuring that the resulting value is withing the range of our hash table size- Returns:
- A 32-bit hash of 4 bytes located at index
-
getHashTable
private static short[] getHashTable(int inputSize)
Creates an appropriately sized hashtable for the given input size- Parameters:
inputSize
- The size of our input, ie. the number of bytes we need to encode- Returns:
- An appropriately sized empty hashtable
-
findMatchingLength
private static int findMatchingLength(ByteBuf in, int minIndex, int inIndex, int maxIndex)
Iterates over the supplied input buffer between the supplied minIndex and maxIndex to find how long our matched copy overlaps with an already-written literal value.- Parameters:
in
- The input buffer to scan overminIndex
- The index in the input buffer to start scanning frominIndex
- The index of the start of our copymaxIndex
- The length of our input buffer- Returns:
- The number of bytes for which our candidate copy is a repeat of
-
bitsToEncode
private static int bitsToEncode(int value)
Calculates the minimum number of bits required to encode a value. This can then in turn be used to calculate the number of septets or octets (as appropriate) to use to encode a length parameter.- Parameters:
value
- The value to calculate the minimum number of bits required to encode- Returns:
- The minimum number of bits required to encode the supplied value
-
encodeLiteral
static void encodeLiteral(ByteBuf in, ByteBuf out, int length)
Writes a literal to the supplied output buffer by directly copying from the input buffer. The literal is taken from the current readerIndex up to the supplied length.- Parameters:
in
- The input buffer to copy fromout
- The output buffer to copy tolength
- The length of the literal to copy
-
encodeCopyWithOffset
private static void encodeCopyWithOffset(ByteBuf out, int offset, int length)
-
encodeCopy
private static void encodeCopy(ByteBuf out, int offset, int length)
Encodes a series of copies, each at most 64 bytes in length.- Parameters:
out
- The output buffer to write the copy pointer tooffset
- The offset at which the original instance lieslength
- The length of the original instance
-
readPreamble
private static int readPreamble(ByteBuf in)
Reads the length varint (a series of bytes, where the lower 7 bits are data and the upper bit is a flag to indicate more bytes to be read).- Parameters:
in
- The input buffer to read the preamble from- Returns:
- The calculated length based on the input buffer, or 0 if no preamble is able to be calculated
-
decodeLiteral
static int decodeLiteral(byte tag, ByteBuf in, ByteBuf out)
Reads a literal from the input buffer directly to the output buffer. A "literal" is an uncompressed segment of data stored directly in the byte stream.- Parameters:
tag
- The tag that identified this segment as a literal is also used to encode part of the length of the datain
- The input buffer to read the literal fromout
- The output buffer to write the literal to- Returns:
- The number of bytes appended to the output buffer, or -1 to indicate "try again later"
-
decodeCopyWith1ByteOffset
private static int decodeCopyWith1ByteOffset(byte tag, ByteBuf in, ByteBuf out, int writtenSoFar)
Reads a compressed reference offset and length from the supplied input buffer, seeks back to the appropriate place in the input buffer and writes the found data to the supplied output stream.- Parameters:
tag
- The tag used to identify this as a copy is also used to encode the length and part of the offsetin
- The input buffer to read fromout
- The output buffer to write to- Returns:
- The number of bytes appended to the output buffer, or -1 to indicate "try again later"
- Throws:
DecompressionException
- If the read offset is invalid
-
decodeCopyWith2ByteOffset
private static int decodeCopyWith2ByteOffset(byte tag, ByteBuf in, ByteBuf out, int writtenSoFar)
Reads a compressed reference offset and length from the supplied input buffer, seeks back to the appropriate place in the input buffer and writes the found data to the supplied output stream.- Parameters:
tag
- The tag used to identify this as a copy is also used to encode the length and part of the offsetin
- The input buffer to read fromout
- The output buffer to write to- Returns:
- The number of bytes appended to the output buffer, or -1 to indicate "try again later"
- Throws:
DecompressionException
- If the read offset is invalid
-
decodeCopyWith4ByteOffset
private static int decodeCopyWith4ByteOffset(byte tag, ByteBuf in, ByteBuf out, int writtenSoFar)
Reads a compressed reference offset and length from the supplied input buffer, seeks back to the appropriate place in the input buffer and writes the found data to the supplied output stream.- Parameters:
tag
- The tag used to identify this as a copy is also used to encode the length and part of the offsetin
- The input buffer to read fromout
- The output buffer to write to- Returns:
- The number of bytes appended to the output buffer, or -1 to indicate "try again later"
- Throws:
DecompressionException
- If the read offset is invalid
-
validateOffset
private static void validateOffset(int offset, int chunkSizeSoFar)
Validates that the offset extracted from a compressed reference is within the permissible bounds of an offset (0 < offset < Integer.MAX_VALUE), and does not exceed the length of the chunk currently read so far.- Parameters:
offset
- The offset extracted from the compressed referencechunkSizeSoFar
- The number of bytes read so far from this chunk- Throws:
DecompressionException
- if the offset is invalid
-
calculateChecksum
static int calculateChecksum(ByteBuf data)
Computes the CRC32C checksum of the supplied data and performs the "mask" operation on the computed checksum- Parameters:
data
- The input data to calculate the CRC32C checksum of
-
calculateChecksum
static int calculateChecksum(ByteBuf data, int offset, int length)
Computes the CRC32C checksum of the supplied data and performs the "mask" operation on the computed checksum- Parameters:
data
- The input data to calculate the CRC32C checksum of
-
validateChecksum
static void validateChecksum(int expectedChecksum, ByteBuf data)
Computes the CRC32C checksum of the supplied data, performs the "mask" operation on the computed checksum, and then compares the resulting masked checksum to the supplied checksum.- Parameters:
expectedChecksum
- The checksum decoded from the stream to compare againstdata
- The input data to calculate the CRC32C checksum of- Throws:
DecompressionException
- If the calculated and supplied checksums do not match
-
validateChecksum
static void validateChecksum(int expectedChecksum, ByteBuf data, int offset, int length)
Computes the CRC32C checksum of the supplied data, performs the "mask" operation on the computed checksum, and then compares the resulting masked checksum to the supplied checksum.- Parameters:
expectedChecksum
- The checksum decoded from the stream to compare againstdata
- The input data to calculate the CRC32C checksum of- Throws:
DecompressionException
- If the calculated and supplied checksums do not match
-
maskChecksum
static int maskChecksum(long checksum)
From the spec: "Checksums are not stored directly, but masked, as checksumming data and then its own checksum can be problematic. The masking is the same as used in Apache Hadoop: Rotate the checksum by 15 bits, then add the constant 0xa282ead8 (using wraparound as normal for unsigned integers)."- Parameters:
checksum
- The actual checksum of the data- Returns:
- The masked checksum
-
-