Class PackParser
- java.lang.Object
-
- org.eclipse.jgit.transport.PackParser
-
- Direct Known Subclasses:
DfsPackParser
,FsckPackParser
,ObjectDirectoryPackParser
public abstract class PackParser extends java.lang.Object
Parses a pack stream and imports it for anObjectInserter
.Applications can acquire an instance of a parser from ObjectInserter's
ObjectInserter.newPackParser(InputStream)
method.Implementations of
ObjectInserter
should subclass this type and provide their own logic for the variouson*()
event methods declared to be abstract.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description private static class
PackParser.DeltaChain
private static class
PackParser.DeltaVisit
private class
PackParser.InflaterStream
static class
PackParser.ObjectTypeAndSize
Type and size information about an object in the database buffer.static class
PackParser.Source
Location data is being obtained from.static class
PackParser.UnresolvedDelta
Information about an unresolved delta in this pack stream.
-
Field Summary
Fields Modifier and Type Field Description private boolean
allowThin
private ObjectIdOwnerMap<PackParser.DeltaChain>
baseById
private LongMap<PackParser.UnresolvedDelta>
baseByPos
private ObjectIdSubclassMap<ObjectId>
baseObjectIds
Objects referenced by their name from deltas, that aren't in this pack.(package private) int
bAvail
private long
bBase
Position in the input stream ofbuf[0]
.private int
bOffset
(package private) byte[]
buf
private static int
BUFFER_SIZE
Size of the internal stream buffer.private boolean
checkEofAfterPackFooter
private boolean
checkObjectCollisions
private BlockList<PackedObjectInfo>
collisionCheckObjs
Objects need to be double-checked for collision after indexing.private int
deltaCount
private PackedObjectInfo[]
entries
private int
entryCount
private boolean
expectDataAfterPackFooter
private long
expectedObjectCount
private byte[]
hdrBuf
private java.io.InputStream
in
private PackParser.InflaterStream
inflater
private java.lang.String
lockMessage
Message to protect the pack data from garbage collection.private long
maxObjectSizeLimit
Git object size limitprivate boolean
needBaseObjectIds
private ObjectIdSubclassMap<ObjectId>
newObjectIds
Every object contained within the incoming pack.private ObjectChecker
objCheck
private ObjectDatabase
objectDatabase
Object database used for loading existing objects.private SHA1
objectHasher
private java.security.MessageDigest
packDigest
private ObjectReader
readCurs
private ReceivedPackStatistics.Builder
stats
private byte[]
tempBuffer
private MutableObjectId
tempObjectId
-
Constructor Summary
Constructors Modifier Constructor Description protected
PackParser(ObjectDatabase odb, java.io.InputStream src)
Initialize a pack parser.
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description private void
addObjectAndTrack(PackedObjectInfo oe)
protected byte[]
buffer()
Get a temporary byte array for use by the caller.protected abstract boolean
checkCRC(int oldCRC)
Check the current CRC matches the expected value.private void
checkIfTooLarge(int typeCode, long size)
private void
checkObjectCollision()
private void
checkObjectCollision(AnyObjectId obj, int type, byte[] data, long sizeBeforeInflating)
private void
checkObjectCollision(PackedObjectInfo obj)
private void
endInput()
(package private) int
fill(PackParser.Source src, int need)
private PackParser.UnresolvedDelta
firstChildOf(PackedObjectInfo oe)
ObjectIdSubclassMap<ObjectId>
getBaseObjectIds()
Get set of objects the incoming pack assumed for delta purposesjava.lang.String
getLockMessage()
Get the message to record with the pack lock.ObjectIdSubclassMap<ObjectId>
getNewObjectIds()
Get the new objects that were sent by the userPackedObjectInfo
getObject(int nth)
Get the information about the requested object.int
getObjectCount()
Get the number of objects in the stream.long
getPackSize()
Get the size of the newly created pack.ReceivedPackStatistics
getReceivedPackStatistics()
Returns the statistics of the parsed pack.java.util.List<PackedObjectInfo>
getSortedObjectList(java.util.Comparator<PackedObjectInfo> cmp)
Get all of the objects, sorted by their name.private void
growEntries(int extraObjects)
private void
indexOneObject()
private java.io.InputStream
inflate(PackParser.Source src, long inflatedSize)
private byte[]
inflateAndReturn(PackParser.Source src, long inflatedSize)
private void
inflateAndSkip(PackParser.Source src, long inflatedSize)
boolean
isAllowThin()
Whether a thin pack (missing base objects) is permitted.boolean
isCheckEofAfterPackFooter()
Whether the EOF should be read from the input after the footer.protected boolean
isCheckObjectCollisions()
Whether received objects are verified to prevent collisions.boolean
isExpectDataAfterPackFooter()
Whether there is data expected after the pack footer.private boolean
needNewObjectIds()
protected PackedObjectInfo
newInfo(AnyObjectId id, PackParser.UnresolvedDelta delta, ObjectId deltaBase)
Construct a PackedObjectInfo instance for this parser.protected abstract boolean
onAppendBase(int typeCode, byte[] data, PackedObjectInfo info)
Provide the implementation with a base that was outside of the pack.protected abstract void
onBeginOfsDelta(long deltaStreamPosition, long baseStreamPosition, long inflatedSize)
Event notifying start of a delta referencing its base by offset.protected abstract void
onBeginRefDelta(long deltaStreamPosition, AnyObjectId baseId, long inflatedSize)
Event notifying start of a delta referencing its base by ObjectId.protected abstract void
onBeginWholeObject(long streamPosition, int type, long inflatedSize)
Event notifying the start of an object stored whole (not as a delta).protected PackParser.UnresolvedDelta
onEndDelta()
Event notifying the current object.protected abstract void
onEndThinPack()
Event indicating a thin pack has been completely processed.protected abstract void
onEndWholeObject(PackedObjectInfo info)
Event notifying the current object.protected abstract void
onInflatedObjectData(PackedObjectInfo obj, int typeCode, byte[] data)
Invoked for commits, trees, tags, and small blobs.protected abstract void
onObjectData(PackParser.Source src, byte[] raw, int pos, int len)
Store (and/or checksum) a portion of an object's data.protected abstract void
onObjectHeader(PackParser.Source src, byte[] raw, int pos, int len)
Store (and/or checksum) an object header.protected abstract void
onPackFooter(byte[] hash)
Provide the implementation with the original stream's pack footer.protected abstract void
onPackHeader(long objCnt)
Provide the implementation with the original stream's pack header.protected abstract void
onStoreStream(byte[] raw, int pos, int len)
Store bytes received from the raw stream.private PackParser.ObjectTypeAndSize
openDatabase(PackedObjectInfo obj, PackParser.ObjectTypeAndSize info)
private PackParser.ObjectTypeAndSize
openDatabase(PackParser.UnresolvedDelta delta, PackParser.ObjectTypeAndSize info)
PackLock
parse(ProgressMonitor progress)
Parse the pack stream.PackLock
parse(ProgressMonitor receiving, ProgressMonitor resolving)
Parse the pack stream.private void
processDeltas(ProgressMonitor resolving)
protected abstract int
readDatabase(byte[] dst, int pos, int cnt)
Read from the database's current position into the buffer.private int
readFrom(PackParser.Source src)
protected PackParser.ObjectTypeAndSize
readObjectHeader(PackParser.ObjectTypeAndSize info)
Read the header of the current object.private void
readPackFooter()
private void
readPackHeader()
private PackParser.UnresolvedDelta
removeBaseById(AnyObjectId id)
private void
resolveDeltas(ProgressMonitor progress)
private void
resolveDeltas(PackedObjectInfo oe, ProgressMonitor progress)
private void
resolveDeltas(PackParser.DeltaVisit visit, int type, PackParser.ObjectTypeAndSize info, ProgressMonitor progress)
private void
resolveDeltasWithExternalBases(ProgressMonitor progress)
private static PackParser.UnresolvedDelta
reverse(PackParser.UnresolvedDelta c)
protected abstract PackParser.ObjectTypeAndSize
seekDatabase(PackedObjectInfo obj, PackParser.ObjectTypeAndSize info)
Reposition the database to re-read a previously stored object.protected abstract PackParser.ObjectTypeAndSize
seekDatabase(PackParser.UnresolvedDelta delta, PackParser.ObjectTypeAndSize info)
Reposition the database to re-read a previously stored object.void
setAllowThin(boolean allow)
Configure this index pack instance to allow a thin pack.void
setCheckEofAfterPackFooter(boolean b)
Ensure EOF is read from the input stream after the footer.protected void
setCheckObjectCollisions(boolean check)
Enable checking for collisions with existing objects.void
setExpectDataAfterPackFooter(boolean e)
Set if there is additional data in InputStream after pack.protected void
setExpectedObjectCount(long expectedObjectCount)
Set the expected number of objects in the pack stream.void
setLockMessage(java.lang.String msg)
Set the lock message for the incoming pack data.void
setMaxObjectSizeLimit(long limit)
Set the maximum allowed Git object size.void
setNeedBaseObjectIds(boolean b)
Configure this index pack instance to keep track of the objects assumed for delta bases.void
setNeedNewObjectIds(boolean b)
Configure this index pack instance to keep track of new objects.void
setObjectChecker(ObjectChecker oc)
Configure the checker used to validate received objects.void
setObjectChecking(boolean on)
Configure the checker used to validate received objects.private long
streamPosition()
private void
sync()
(package private) void
use(int cnt)
protected void
verifySafeObject(AnyObjectId id, int type, byte[] data)
Verify the integrity of the object.private void
whole(long pos, int type, long sz)
-
-
-
Field Detail
-
BUFFER_SIZE
private static final int BUFFER_SIZE
Size of the internal stream buffer.- See Also:
- Constant Field Values
-
objectDatabase
private final ObjectDatabase objectDatabase
Object database used for loading existing objects.
-
inflater
private PackParser.InflaterStream inflater
-
tempBuffer
private byte[] tempBuffer
-
hdrBuf
private byte[] hdrBuf
-
objectHasher
private final SHA1 objectHasher
-
tempObjectId
private final MutableObjectId tempObjectId
-
in
private java.io.InputStream in
-
buf
byte[] buf
-
bBase
private long bBase
Position in the input stream ofbuf[0]
.
-
bOffset
private int bOffset
-
bAvail
int bAvail
-
objCheck
private ObjectChecker objCheck
-
allowThin
private boolean allowThin
-
checkObjectCollisions
private boolean checkObjectCollisions
-
needBaseObjectIds
private boolean needBaseObjectIds
-
checkEofAfterPackFooter
private boolean checkEofAfterPackFooter
-
expectDataAfterPackFooter
private boolean expectDataAfterPackFooter
-
expectedObjectCount
private long expectedObjectCount
-
entries
private PackedObjectInfo[] entries
-
newObjectIds
private ObjectIdSubclassMap<ObjectId> newObjectIds
Every object contained within the incoming pack.This is a subset of
entries
, as thin packs can add additional objects toentries
by copying already existing objects from the repository onto the end of the thin pack to make it self-contained.
-
deltaCount
private int deltaCount
-
entryCount
private int entryCount
-
baseById
private ObjectIdOwnerMap<PackParser.DeltaChain> baseById
-
baseObjectIds
private ObjectIdSubclassMap<ObjectId> baseObjectIds
Objects referenced by their name from deltas, that aren't in this pack.This is the set of objects that were copied onto the end of this pack to make it complete. These objects were not transmitted by the remote peer, but instead were assumed to already exist in the local repository.
-
baseByPos
private LongMap<PackParser.UnresolvedDelta> baseByPos
-
collisionCheckObjs
private BlockList<PackedObjectInfo> collisionCheckObjs
Objects need to be double-checked for collision after indexing.
-
packDigest
private java.security.MessageDigest packDigest
-
readCurs
private ObjectReader readCurs
-
lockMessage
private java.lang.String lockMessage
Message to protect the pack data from garbage collection.
-
maxObjectSizeLimit
private long maxObjectSizeLimit
Git object size limit
-
stats
private final ReceivedPackStatistics.Builder stats
-
-
Constructor Detail
-
PackParser
protected PackParser(ObjectDatabase odb, java.io.InputStream src)
Initialize a pack parser.- Parameters:
odb
- database the parser will write its objects into.src
- the stream the parser will read.
-
-
Method Detail
-
isAllowThin
public boolean isAllowThin()
Whether a thin pack (missing base objects) is permitted.- Returns:
true
if a thin pack (missing base objects) is permitted.
-
setAllowThin
public void setAllowThin(boolean allow)
Configure this index pack instance to allow a thin pack.Thin packs are sometimes used during network transfers to allow a delta to be sent without a base object. Such packs are not permitted on disk.
- Parameters:
allow
- true to enable a thin pack.
-
isCheckObjectCollisions
protected boolean isCheckObjectCollisions()
Whether received objects are verified to prevent collisions.- Returns:
- if true received objects are verified to prevent collisions.
- Since:
- 4.1
-
setCheckObjectCollisions
protected void setCheckObjectCollisions(boolean check)
Enable checking for collisions with existing objects.By default PackParser looks for each received object in the repository. If the object already exists, the existing object is compared byte-for-byte with the newly received copy to ensure they are identical. The receive is aborted with an exception if any byte differs. This check is necessary to prevent an evil attacker from supplying a replacement object into this repository in the event that a discovery enabling SHA-1 collisions is made.
This check may be very costly to perform, and some repositories may have other ways to segregate newly received object data. The check is enabled by default, but can be explicitly disabled if the implementation can provide the same guarantee, or is willing to accept the risks associated with bypassing the check.
- Parameters:
check
- true to enable collision checking (strongly encouraged).- Since:
- 4.1
-
setNeedNewObjectIds
public void setNeedNewObjectIds(boolean b)
Configure this index pack instance to keep track of new objects.By default an index pack doesn't save the new objects that were created when it was instantiated. Setting this flag to
true
allows the caller to usegetNewObjectIds()
to retrieve that list.- Parameters:
b
-true
to enable keeping track of new objects.
-
needNewObjectIds
private boolean needNewObjectIds()
-
setNeedBaseObjectIds
public void setNeedBaseObjectIds(boolean b)
Configure this index pack instance to keep track of the objects assumed for delta bases.By default an index pack doesn't save the objects that were used as delta bases. Setting this flag to
true
will allow the caller to usegetBaseObjectIds()
to retrieve that list.- Parameters:
b
-true
to enable keeping track of delta bases.
-
isCheckEofAfterPackFooter
public boolean isCheckEofAfterPackFooter()
Whether the EOF should be read from the input after the footer.- Returns:
- true if the EOF should be read from the input after the footer.
-
setCheckEofAfterPackFooter
public void setCheckEofAfterPackFooter(boolean b)
Ensure EOF is read from the input stream after the footer.- Parameters:
b
- true if the EOF should be read; false if it is not checked.
-
isExpectDataAfterPackFooter
public boolean isExpectDataAfterPackFooter()
Whether there is data expected after the pack footer.- Returns:
- true if there is data expected after the pack footer.
-
setExpectDataAfterPackFooter
public void setExpectDataAfterPackFooter(boolean e)
Set if there is additional data in InputStream after pack.- Parameters:
e
- true if there is additional data in InputStream after pack. This requires the InputStream to support the mark and reset functions.
-
getNewObjectIds
public ObjectIdSubclassMap<ObjectId> getNewObjectIds()
Get the new objects that were sent by the user- Returns:
- the new objects that were sent by the user
-
getBaseObjectIds
public ObjectIdSubclassMap<ObjectId> getBaseObjectIds()
Get set of objects the incoming pack assumed for delta purposes- Returns:
- set of objects the incoming pack assumed for delta purposes
-
setObjectChecker
public void setObjectChecker(ObjectChecker oc)
Configure the checker used to validate received objects.Usually object checking isn't necessary, as Git implementations only create valid objects in pack files. However, additional checking may be useful if processing data from an untrusted source.
- Parameters:
oc
- the checker instance; null to disable object checking.
-
setObjectChecking
public void setObjectChecking(boolean on)
Configure the checker used to validate received objects.Usually object checking isn't necessary, as Git implementations only create valid objects in pack files. However, additional checking may be useful if processing data from an untrusted source.
This is shorthand for:
setObjectChecker(on ? new ObjectChecker() : null);
- Parameters:
on
- true to enable the default checker; false to disable it.
-
getLockMessage
public java.lang.String getLockMessage()
Get the message to record with the pack lock.- Returns:
- the message to record with the pack lock.
-
setLockMessage
public void setLockMessage(java.lang.String msg)
Set the lock message for the incoming pack data.- Parameters:
msg
- if not null, the message to associate with the incoming data while it is locked to prevent garbage collection.
-
setMaxObjectSizeLimit
public void setMaxObjectSizeLimit(long limit)
Set the maximum allowed Git object size.If an object is larger than the given size the pack-parsing will throw an exception aborting the parsing.
- Parameters:
limit
- the Git object size limit. If zero then there is not limit.
-
getObjectCount
public int getObjectCount()
Get the number of objects in the stream.The object count is only available after
parse(ProgressMonitor)
has returned. The count may have been increased if the stream was a thin pack, and missing bases objects were appending onto it by the subclass.- Returns:
- number of objects parsed out of the stream.
-
getObject
public PackedObjectInfo getObject(int nth)
Get the information about the requested object.The object information is only available after
parse(ProgressMonitor)
has returned.- Parameters:
nth
- index of the object in the stream. Must be between 0 andgetObjectCount()
-1.- Returns:
- the object information.
-
getSortedObjectList
public java.util.List<PackedObjectInfo> getSortedObjectList(java.util.Comparator<PackedObjectInfo> cmp)
Get all of the objects, sorted by their name.The object information is only available after
parse(ProgressMonitor)
has returned.To maintain lower memory usage and good runtime performance, this method sorts the objects in-place and therefore impacts the ordering presented by
getObject(int)
.- Parameters:
cmp
- comparison function, if null objects are stored by ObjectId.- Returns:
- sorted list of objects in this pack stream.
-
getPackSize
public long getPackSize()
Get the size of the newly created pack.This will also include the pack index size if an index was created. This method should only be called after pack parsing is finished.
- Returns:
- the pack size (including the index size) or -1 if the size cannot be determined
- Since:
- 3.3
-
getReceivedPackStatistics
public ReceivedPackStatistics getReceivedPackStatistics()
Returns the statistics of the parsed pack.This should only be called after pack parsing is finished.
- Returns:
ReceivedPackStatistics
- Since:
- 4.6
-
parse
public final PackLock parse(ProgressMonitor progress) throws java.io.IOException
Parse the pack stream.- Parameters:
progress
- callback to provide progress feedback during parsing. If null,NullProgressMonitor
will be used.- Returns:
- the pack lock, if one was requested by setting
setLockMessage(String)
. - Throws:
java.io.IOException
- the stream is malformed, or contains corrupt objects.- Since:
- 6.0
-
parse
public PackLock parse(ProgressMonitor receiving, ProgressMonitor resolving) throws java.io.IOException
Parse the pack stream.- Parameters:
receiving
- receives progress feedback during the initial receiving objects phase. If null,NullProgressMonitor
will be used.resolving
- receives progress feedback during the resolving objects phase.- Returns:
- the pack lock, if one was requested by setting
setLockMessage(String)
. - Throws:
java.io.IOException
- the stream is malformed, or contains corrupt objects.- Since:
- 6.0
-
processDeltas
private void processDeltas(ProgressMonitor resolving) throws java.io.IOException
- Throws:
java.io.IOException
-
resolveDeltas
private void resolveDeltas(ProgressMonitor progress) throws java.io.IOException
- Throws:
java.io.IOException
-
resolveDeltas
private void resolveDeltas(PackedObjectInfo oe, ProgressMonitor progress) throws java.io.IOException
- Throws:
java.io.IOException
-
resolveDeltas
private void resolveDeltas(PackParser.DeltaVisit visit, int type, PackParser.ObjectTypeAndSize info, ProgressMonitor progress) throws java.io.IOException
- Throws:
java.io.IOException
-
checkIfTooLarge
private final void checkIfTooLarge(int typeCode, long size) throws java.io.IOException
- Throws:
java.io.IOException
-
readObjectHeader
protected PackParser.ObjectTypeAndSize readObjectHeader(PackParser.ObjectTypeAndSize info) throws java.io.IOException
Read the header of the current object.After the header has been parsed, this method automatically invokes
onObjectHeader(Source, byte[], int, int)
to allow the implementation to update its internal checksums for the bytes read.When this method returns the database will be positioned on the first byte of the deflated data stream.
- Parameters:
info
- the info object to populate.- Returns:
info
, after populating.- Throws:
java.io.IOException
- the size cannot be read.
-
removeBaseById
private PackParser.UnresolvedDelta removeBaseById(AnyObjectId id)
-
reverse
private static PackParser.UnresolvedDelta reverse(PackParser.UnresolvedDelta c)
-
firstChildOf
private PackParser.UnresolvedDelta firstChildOf(PackedObjectInfo oe)
-
resolveDeltasWithExternalBases
private void resolveDeltasWithExternalBases(ProgressMonitor progress) throws java.io.IOException
- Throws:
java.io.IOException
-
growEntries
private void growEntries(int extraObjects)
-
readPackHeader
private void readPackHeader() throws java.io.IOException
- Throws:
java.io.IOException
-
readPackFooter
private void readPackFooter() throws java.io.IOException
- Throws:
java.io.IOException
-
endInput
private void endInput()
-
indexOneObject
private void indexOneObject() throws java.io.IOException
- Throws:
java.io.IOException
-
whole
private void whole(long pos, int type, long sz) throws java.io.IOException
- Throws:
java.io.IOException
-
verifySafeObject
protected void verifySafeObject(AnyObjectId id, int type, byte[] data) throws CorruptObjectException
Verify the integrity of the object.- Parameters:
id
- identity of the object to be checked.type
- the type of the object.data
- raw content of the object.- Throws:
CorruptObjectException
- Since:
- 4.9
-
checkObjectCollision
private void checkObjectCollision() throws java.io.IOException
- Throws:
java.io.IOException
-
checkObjectCollision
private void checkObjectCollision(PackedObjectInfo obj) throws java.io.IOException
- Throws:
java.io.IOException
-
checkObjectCollision
private void checkObjectCollision(AnyObjectId obj, int type, byte[] data, long sizeBeforeInflating) throws java.io.IOException
- Throws:
java.io.IOException
-
streamPosition
private long streamPosition()
- Returns:
- current position of the input stream being parsed.
-
openDatabase
private PackParser.ObjectTypeAndSize openDatabase(PackedObjectInfo obj, PackParser.ObjectTypeAndSize info) throws java.io.IOException
- Throws:
java.io.IOException
-
openDatabase
private PackParser.ObjectTypeAndSize openDatabase(PackParser.UnresolvedDelta delta, PackParser.ObjectTypeAndSize info) throws java.io.IOException
- Throws:
java.io.IOException
-
readFrom
private int readFrom(PackParser.Source src) throws java.io.IOException
- Throws:
java.io.IOException
-
use
void use(int cnt)
-
fill
int fill(PackParser.Source src, int need) throws java.io.IOException
- Throws:
java.io.IOException
-
sync
private void sync() throws java.io.IOException
- Throws:
java.io.IOException
-
buffer
protected byte[] buffer()
Get a temporary byte array for use by the caller.- Returns:
- a temporary byte array for use by the caller.
-
newInfo
protected PackedObjectInfo newInfo(AnyObjectId id, PackParser.UnresolvedDelta delta, ObjectId deltaBase)
Construct a PackedObjectInfo instance for this parser.- Parameters:
id
- identity of the object to be tracked.delta
- if the object was previously an unresolved delta, this is the delta object that was tracking it. Otherwise null.deltaBase
- if the object was previously an unresolved delta, this is the ObjectId of the base of the delta. The base may be outside of the pack stream if the stream was a thin-pack.- Returns:
- info object containing this object's data.
-
setExpectedObjectCount
protected void setExpectedObjectCount(long expectedObjectCount)
Set the expected number of objects in the pack stream.The object count in the pack header is not always correct for some Dfs pack files. e.g. INSERT pack always assume 1 object in the header since the actual object count is unknown when the pack is written.
If external implementation wants to overwrite the expectedObjectCount, they should call this method during
onPackHeader(long)
.- Parameters:
expectedObjectCount
- a long.- Since:
- 4.9
-
onStoreStream
protected abstract void onStoreStream(byte[] raw, int pos, int len) throws java.io.IOException
Store bytes received from the raw stream.This method is invoked during
parse(ProgressMonitor)
as data is consumed from the incoming stream. Implementors may use this event to archive the raw incoming stream to the destination repository in large chunks, without paying attention to object boundaries.The only component of the pack not supplied to this method is the last 20 bytes of the pack that comprise the trailing SHA-1 checksum. Those are passed to
onPackFooter(byte[])
.- Parameters:
raw
- buffer to copy data out of.pos
- first offset within the buffer that is valid.len
- number of bytes in the buffer that are valid.- Throws:
java.io.IOException
- the stream cannot be archived.
-
onObjectHeader
protected abstract void onObjectHeader(PackParser.Source src, byte[] raw, int pos, int len) throws java.io.IOException
Store (and/or checksum) an object header.Invoked after any of the
onBegin()
events. The entire header is supplied in a single invocation, before any object data is supplied.- Parameters:
src
- where the data came fromraw
- buffer to read data from.pos
- first offset within buffer that is valid.len
- number of bytes in buffer that are valid.- Throws:
java.io.IOException
- the stream cannot be archived.
-
onObjectData
protected abstract void onObjectData(PackParser.Source src, byte[] raw, int pos, int len) throws java.io.IOException
Store (and/or checksum) a portion of an object's data.This method may be invoked multiple times per object, depending on the size of the object, the size of the parser's internal read buffer, and the alignment of the object relative to the read buffer.
Invoked after
onObjectHeader(Source, byte[], int, int)
.- Parameters:
src
- where the data came fromraw
- buffer to read data from.pos
- first offset within buffer that is valid.len
- number of bytes in buffer that are valid.- Throws:
java.io.IOException
- the stream cannot be archived.
-
onInflatedObjectData
protected abstract void onInflatedObjectData(PackedObjectInfo obj, int typeCode, byte[] data) throws java.io.IOException
Invoked for commits, trees, tags, and small blobs.- Parameters:
obj
- the object info, populated.typeCode
- the type of the object.data
- inflated data for the object.- Throws:
java.io.IOException
- the object cannot be archived.
-
onPackHeader
protected abstract void onPackHeader(long objCnt) throws java.io.IOException
Provide the implementation with the original stream's pack header.- Parameters:
objCnt
- number of objects expected in the stream.- Throws:
java.io.IOException
- the implementation refuses to work with this many objects.
-
onPackFooter
protected abstract void onPackFooter(byte[] hash) throws java.io.IOException
Provide the implementation with the original stream's pack footer.- Parameters:
hash
- the trailing 20 bytes of the pack, this is a SHA-1 checksum of all of the pack data.- Throws:
java.io.IOException
- the stream cannot be archived.
-
onAppendBase
protected abstract boolean onAppendBase(int typeCode, byte[] data, PackedObjectInfo info) throws java.io.IOException
Provide the implementation with a base that was outside of the pack.This event only occurs on a thin pack for base objects that were outside of the pack and came from the local repository. Usually an implementation uses this event to compress the base and append it onto the end of the pack, so the pack stays self-contained.
- Parameters:
typeCode
- type of the base object.data
- complete content of the base object.info
- packed object information for this base. Implementors must populate the CRC and offset members if returning true.- Returns:
- true if the
info
should be included in the object list returned bygetSortedObjectList(Comparator)
, false if it should not be included. - Throws:
java.io.IOException
- the base could not be included into the pack.
-
onEndThinPack
protected abstract void onEndThinPack() throws java.io.IOException
Event indicating a thin pack has been completely processed.This event is invoked only if a thin pack has delta references to objects external from the pack. The event is called after all of those deltas have been resolved.
- Throws:
java.io.IOException
- the pack cannot be archived.
-
seekDatabase
protected abstract PackParser.ObjectTypeAndSize seekDatabase(PackedObjectInfo obj, PackParser.ObjectTypeAndSize info) throws java.io.IOException
Reposition the database to re-read a previously stored object.If the database is computing CRC-32 checksums for object data, it should reset its internal CRC instance during this method call.
- Parameters:
obj
- the object position to begin reading from. This is fromnewInfo(AnyObjectId, UnresolvedDelta, ObjectId)
.info
- object to populate with type and size.- Returns:
- the
info
object. - Throws:
java.io.IOException
- the database cannot reposition to this location.
-
seekDatabase
protected abstract PackParser.ObjectTypeAndSize seekDatabase(PackParser.UnresolvedDelta delta, PackParser.ObjectTypeAndSize info) throws java.io.IOException
Reposition the database to re-read a previously stored object.If the database is computing CRC-32 checksums for object data, it should reset its internal CRC instance during this method call.
- Parameters:
delta
- the object position to begin reading from. This is an instance previously returned byonEndDelta()
.info
- object to populate with type and size.- Returns:
- the
info
object. - Throws:
java.io.IOException
- the database cannot reposition to this location.
-
readDatabase
protected abstract int readDatabase(byte[] dst, int pos, int cnt) throws java.io.IOException
Read from the database's current position into the buffer.- Parameters:
dst
- the buffer to copy read data into.pos
- position withindst
to start copying data into.cnt
- ideal target number of bytes to read. Actual read length may be shorter.- Returns:
- number of bytes stored.
- Throws:
java.io.IOException
- the database cannot be accessed.
-
checkCRC
protected abstract boolean checkCRC(int oldCRC)
Check the current CRC matches the expected value.This method is invoked when an object is read back in from the database and its data is used during delta resolution. The CRC is validated after the object has been fully read, allowing the parser to verify there was no silent data corruption.
Implementations are free to ignore this check by always returning true if they are performing other data integrity validations at a lower level.
- Parameters:
oldCRC
- the prior CRC that was recorded during the first scan of the object from the pack stream.- Returns:
- true if the CRC matches; false if it does not.
-
onBeginWholeObject
protected abstract void onBeginWholeObject(long streamPosition, int type, long inflatedSize) throws java.io.IOException
Event notifying the start of an object stored whole (not as a delta).- Parameters:
streamPosition
- position of this object in the incoming stream.type
- type of the object; one ofConstants.OBJ_COMMIT
,Constants.OBJ_TREE
,Constants.OBJ_BLOB
, orConstants.OBJ_TAG
.inflatedSize
- size of the object when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.- Throws:
java.io.IOException
- the object cannot be recorded.
-
onEndWholeObject
protected abstract void onEndWholeObject(PackedObjectInfo info) throws java.io.IOException
Event notifying the current object.- Parameters:
info
- object information.- Throws:
java.io.IOException
- the object cannot be recorded.
-
onBeginOfsDelta
protected abstract void onBeginOfsDelta(long deltaStreamPosition, long baseStreamPosition, long inflatedSize) throws java.io.IOException
Event notifying start of a delta referencing its base by offset.- Parameters:
deltaStreamPosition
- position of this object in the incoming stream.baseStreamPosition
- position of the base object in the incoming stream. The base must be before the delta, thereforebaseStreamPosition < deltaStreamPosition
. This is not the position returned by a prior end object event.inflatedSize
- size of the delta when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.- Throws:
java.io.IOException
- the object cannot be recorded.
-
onBeginRefDelta
protected abstract void onBeginRefDelta(long deltaStreamPosition, AnyObjectId baseId, long inflatedSize) throws java.io.IOException
Event notifying start of a delta referencing its base by ObjectId.- Parameters:
deltaStreamPosition
- position of this object in the incoming stream.baseId
- name of the base object. This object may be later in the stream, or might not appear at all in the stream (in the case of a thin-pack).inflatedSize
- size of the delta when fully inflated. The size stored within the pack may be larger or smaller, and is not yet known.- Throws:
java.io.IOException
- the object cannot be recorded.
-
onEndDelta
protected PackParser.UnresolvedDelta onEndDelta() throws java.io.IOException
Event notifying the current object.- Returns:
- object information that must be populated with at least the offset.
- Throws:
java.io.IOException
- the object cannot be recorded.
-
inflateAndSkip
private void inflateAndSkip(PackParser.Source src, long inflatedSize) throws java.io.IOException
- Throws:
java.io.IOException
-
inflateAndReturn
private byte[] inflateAndReturn(PackParser.Source src, long inflatedSize) throws java.io.IOException
- Throws:
java.io.IOException
-
inflate
private java.io.InputStream inflate(PackParser.Source src, long inflatedSize) throws java.io.IOException
- Throws:
java.io.IOException
-
addObjectAndTrack
private void addObjectAndTrack(PackedObjectInfo oe)
-
-