Class LogBuffer

  • Direct Known Subclasses:
    BlockLogBuffer

    abstract class LogBuffer
    extends LogObject
    Classes used as buffers in LogBufferManager must implement this interface.

    This abstract class implements methods common to all LogBuffer sub-classes.

    • Field Summary

      Fields 
      Modifier and Type Field Description
      (package private) int bsn
      buffer sequence number.
      (package private) java.nio.ByteBuffer buffer  
      (package private) int bytesUsed
      Number of used data bytes in the buffer.
      (package private) java.util.zip.Adler32 checksum  
      (package private) byte[] checksumBuffer
      Local buffer used to compute checksums.
      (package private) boolean doChecksum
      switch to enable computation of checksum.
      (package private) boolean forceNow
      set true by LogFileManager if the buffer must be forced due to file switch.
      (package private) int index
      buffer number used by owner (LogBufferManager) to workerID into an array of buffers.
      (package private) java.io.IOException ioexception
      IOException from last write
      (package private) int iostatus
      results of last write.
      (package private) LogFile lf
      LogFile associated with the current LogBuffer.
      (package private) java.lang.String name
      name of this buffer object.
      (package private) boolean rewind
      set true if this LogBuffer should issue a rewind on the FileChannel before writing ByteBuffer.
      (package private) long tod
      currentTimeMillis that buffer was initialized.
      protected int waitingThreads
      number of waiting threads.
      (package private) java.lang.Object waitingThreadsLock
      mutex for synchronizing access to waitingThreads
    • Constructor Summary

      Constructors 
      Constructor Description
      LogBuffer​(Configuration config)
      default constructor.
    • Method Summary

      All Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      (package private) int checksum()
      Computes a checksum over the the entire byte buffer backing this LogBuffer object.
      (package private) java.lang.String getName()
      May be used in traces or other purposes for debugging.
      (package private) abstract java.lang.String getStats()
      returns statistics for this LogBuffer object.
      (package private) int getWaitingThreads()
      returns the number of threads currently waiting for the buffer to be forced to disk.
      (package private) abstract LogBuffer init​(int bsn, LogFileManager lfm)
      initialize members for LogBuffer implementation class for reuse.
      (package private) abstract long put​(short type, byte[][] data, boolean sync)
      puts a data record into the buffer and returns a token for record.
      (package private) abstract LogBuffer read​(LogFile lf, long position)
      read a block of data from the LogFile object provided in the lf parameter starting at the position specified in the postiion parameter.
      (package private) int release()
      decrements count of waiting threads and returns updated value.
      (package private) abstract boolean shouldForce()
      returns true if the buffer should be forced to disk.
      (package private) void sync()
      park threads that are waiting for the ByteBuffer to be forced to disk.
      (package private) abstract void write()
      write ByteBuffer to the LogFile.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • buffer

        final java.nio.ByteBuffer buffer
      • index

        int index
        buffer number used by owner (LogBufferManager) to workerID into an array of buffers.

        Actual use of workerID is determined by the buffer manager implementation.

      • tod

        long tod
        currentTimeMillis that buffer was initialized.

        Implementations of LogBuffer should provide some means to persist tod to the file to allow recovery operations to determine when a buffer was written.

        Used during replay situations to validate block integrity. The TOD field from the block header is compared with the TOD field of the block footer to verify that an entire block was written.

      • waitingThreads

        protected int waitingThreads
        number of waiting threads.

        Always synchronized on (waitingThreadsLock). Design Note:
        Originally this field was volatile. It was changed at the suggestion of David Jencks to protected.

        We tried using (this) to protect access, but this caused some additional lock contention and performance fell off about 10%.

      • waitingThreadsLock

        final java.lang.Object waitingThreadsLock
        mutex for synchronizing access to waitingThreads
      • iostatus

        int iostatus
        results of last write.

        Value must be one of the constants defined in LogBuffer interface.

      • bsn

        int bsn
        buffer sequence number.

        LogBufferManager maintains a sequence number of buffers written. The sequence number is stored in the block header of each log block.

        Initialized to zero.

        Set to -1 by read() if bytes read is -1 (end of file)

      • rewind

        boolean rewind
        set true if this LogBuffer should issue a rewind on the FileChannel before writing ByteBuffer.
      • forceNow

        boolean forceNow
        set true by LogFileManager if the buffer must be forced due to file switch.

        BUG: 300505

      • ioexception

        java.io.IOException ioexception
        IOException from last write
      • name

        java.lang.String name
        name of this buffer object.
      • lf

        LogFile lf
        LogFile associated with the current LogBuffer.

        The LogBufferManager will have a pool of LogBuffers. If the containing Logger is managing a pool of files, then it is possible that some period of time, some buffers will be written to one file, while other buffers are written to another file. To allow writes and forces to separate files to be performed in parallel, each LogBuffer must keep track of its own LogFile.

        See Also:
        LogFileManager.getLogFileForWrite(LogBuffer)
      • doChecksum

        boolean doChecksum
        switch to enable computation of checksum.

        Since it takes some CPU to compute checksums over a buffer, it might be useful to disable the checksum, at least for performance measurements.

        When doChecksum is true then an implementation class should compute a checksum and store the value in the buffer during the write() method.

        Use of checksums is optional and depends on the actual implementation class.

      • bytesUsed

        int bytesUsed
        Number of used data bytes in the buffer.

        This is different than buffer capacity(). The bytes used is the number of bytes of usable data in a buffer. Bytes between the bytes used count and the buffer footer are undefined, possibly uninitialized or residue.

        set by operations that read data from files into the buffer such as read().

        checked by operations that retrieve logical records from the buffer get().

      • checksumBuffer

        byte[] checksumBuffer
        Local buffer used to compute checksums. The initial implementation of HOWL used the ByteBuffer.hashCode() method to compute a checksum. Since each vendor's implementation of the JVM is not guaranteed to generate the same value for hashCode, this approach is not portable, and could result in journal files that appear to be invalid if restarted using a JVM by a vendor that is different than the JVM that was used to write the journal initially. The problem is solved by implementing our own checksum method. The checksumBuffer member is used to obtain a local copy of the ByteBuffer contents for purposses of computing the checksum. Instead of obtaining each byte individually, the entire buffer is retrieved into checksumBuffer for computing the checksum.
      • checksum

        final java.util.zip.Adler32 checksum
    • Constructor Detail

      • LogBuffer

        LogBuffer​(Configuration config)
        default constructor.

        after creating a new instance of LogBuffer the caller must invoke config().

    • Method Detail

      • release

        final int release()
        decrements count of waiting threads and returns updated value.
        Returns:
        number of threads still waiting after the release.
      • getWaitingThreads

        final int getWaitingThreads()
        returns the number of threads currently waiting for the buffer to be forced to disk.
        Returns:
        current value of waitingThreads
      • sync

        final void sync()
                 throws java.io.IOException,
                        java.lang.InterruptedException
        park threads that are waiting for the ByteBuffer to be forced to disk.

        The count of waiting threads ( waitingThreads ) has been incremented in put() .

        Throws:
        java.io.IOException
        java.lang.InterruptedException
      • checksum

        int checksum()
        Computes a checksum over the the entire byte buffer backing this LogBuffer object.
        Returns:
        the computed checksum.
      • getName

        java.lang.String getName()
        May be used in traces or other purposes for debugging.
        Returns:
        name of LogBuffer object.
      • init

        abstract LogBuffer init​(int bsn,
                                LogFileManager lfm)
                         throws LogFileOverflowException
        initialize members for LogBuffer implementation class for reuse.

        LogBufferManager maintains a pool of LogBuffer objects. Each time a LogBuffer is allocated from the pool for use as the current collection buffer, the init() routine is called. After performing necessary initialization, the LogBuffer invokes the LogFileManager to obtain a LogFile for use when writing and forcing the buffer. If a file switch occurrs, the LogFileManager will store a file header record into this newly initialized buffer.

        Parameters:
        bsn - Logical Block Sequence Number of the buffer. LogBufferManager maintains a block sequence number to ensure correct order of writes to disk. Some implementations of LogBuffer may include the BSN as part of a record or block header.
        lfm - LogFileMaager to call to obtain a LogFile.
        Returns:
        this LogBuffer
        Throws:
        LogFileOverflowException
      • read

        abstract LogBuffer read​(LogFile lf,
                                long position)
                         throws java.io.IOException,
                                InvalidLogBufferException
        read a block of data from the LogFile object provided in the lf parameter starting at the position specified in the postiion parameter.

        Used by LogFileManager implementations to read blocks of data that are formatted by a specific LogBuffer implementation.

        The LogFileManager uses LogBufferManager.getLogBuffer() method to obtain a buffer that is used for reading log files.

        Parameters:
        lf - LogFile to read.
        position - within the LogFile to be read.
        Returns:
        this LogBuffer reference.
        Throws:
        java.io.IOException
        InvalidLogBufferException
      • shouldForce

        abstract boolean shouldForce()
        returns true if the buffer should be forced to disk.

        The criteria for determining if a buffer should be forced is implementation dependent. Typically, this method will return true if there are one or more threads waiting for the buffer to be forced and the amount of time the threads has been waiting has been longer than some implementation defined value.

        Returns:
        true if buffer should be forced immediately.
      • put

        abstract long put​(short type,
                          byte[][] data,
                          boolean sync)
                   throws LogRecordSizeException
        puts a data record into the buffer and returns a token for record.

        PRECONDITION: caller holds a bufferManager monitor.

        The caller must set the sync parameter true if the thread will ultimately call sync() after a successful put(). This strategy allows the waitingThreads counter to be incremented while the current thread holds the bufferManager monitor.

        Implementations should return a token that can be used later for replay, and for debugging purposes.

        The data record is passed as a byte[][] allowing callers to construct data records from individual bits of information. The arrays are concatenated into a single log record whose size is the sum of the individual array sizes. Each array is preceded by the size of the individual array. The entire record is preceded by the size of all arrays including the individual array size fields.

        The record format is as follows:

         +------+------------+----------------+---------+       +----------------+---------+
         | type | total size | data[0].length | data[0] | . . . | data[n].length | data[n] |
         +------+------------+----------------+---------+       +----------------+---------+
         

        During replay the entire record is returned as a single byte[]. The ReplayListener is responsible for decomposing the record into the original array of byte[].

        Parameters:
        type - short containing implementation defined record type information. The type is stored as the first field of the log record.
        data - byte[][] to be written to log.
        sync - true if thread will call sync following the put. Causes count of waitingThreads to be incremented.
        Returns:
        a long that contains the physical position of the record is returned. The value returned by put() is an encoding of the physical position. The format of the returned value is implementation specific, and should be treated as opaque by the caller. Returns 0 if there is no room for the record in the current buffer.
        Throws:
        LogRecordSizeException - if the sum of all data[] array sizes is larger than the maximum allowed record size for the configured buffer size.
      • write

        abstract void write()
                     throws java.io.IOException
        write ByteBuffer to the LogFile.

        updates the buffer header with the number of bytes used. Based on configuration, some implementations may compute a hash code or other integrety value and include the value in some implementation defined header information.

        The buffer is written using the LogFile.write() method to allow the LogFile to manage file position for circular journals.

        forcing and notification of waiting threads is the responsibility of the LogBufferManager that owns this LogBUffer.

        Throws:
        java.io.IOException - rethrows any IOExceptions thrown by FileChannel methods.

        QUESTION: should waiters be interupted if IO Error occurs?

        See Also:
        init(int, LogFileManager), LogFile.write(LogBuffer)
      • getStats

        abstract java.lang.String getStats()
        returns statistics for this LogBuffer object.

        information is returned in the form of an XML node that can be included as a nested element in a larger document containing stats for multiple LogBuffer objects, and any containing objects.

        Returns:
        statistics for this buffer as XML