Class LogFileManager
- java.lang.Object
-
- org.objectweb.howl.log.LogObject
-
- org.objectweb.howl.log.LogFileManager
-
class LogFileManager extends LogObject
Manage a set of log files. This class implements methods that can be called by LogBufferManager to obtain a LogFile for logger IO and to signal the LogFileManager when new buffers are being initialized. LogFileManager manages log files according to implementation specific policies. Some LogFileManagers may use a circular file policy while others may use a set of files. The most simple implementations will use a single file and allow it to grow as needed. QUESTION: do we need multiple implementations, or can we deal with different policies in this one class using configuration?
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description (package private) class
LogFileManager.EventManager
helper thread used invoke LogEventListener when log overflow (or other event) is about to occur.
-
Field Summary
Fields Modifier and Type Field Description (package private) long
activeMark
The log key for the oldest active entry in the log.(package private) boolean
automark
indicates whether log files will be marked automatically.(package private) byte[]
autoMarkOff
data written to log when autoMark is turned off(package private) byte[]
autoMarkOn
data written to log when autoMark is turned on(package private) LogBufferManager
bmgr
LogBufferManager used by methods that put() records to the log.(package private) byte[]
crlf
end of line for log records to make logs readable in text editors.(package private) long
currentKey
last key returned by put().(package private) LogFile
currentLogFile
(package private) int
event
event to be processed by the eventManagerThread.private LogEventListener
eventListener
The LogEventListener registered by the application that owns the logger.(package private) java.lang.Object
eventManagerLock
Monitor used by EventManager thread to wait for events that need to be notified.(package private) java.lang.Thread
eventManagerThread
Thread used by LogFileManager to send events to the registered LogEventListener.(package private) byte[][]
fileHeader
LogFile header record.(package private) java.nio.ByteBuffer
fileHeaderBB
ByteBuffer wrapper for fileHeader to facilitate conversion of numeric information to byte[] format.private java.lang.Object
fileManagerLock
lock controlling access to LogFile.(package private) LogFile[]
fileSet
set of LogFile objects associated with the physical log files.private boolean
initComplete
Indicates that LogFileManager initialization is complete.private long
initialKey
the first log key generated by this instance of the Logger.(package private) int
lfIndex
workerID to current entry in fileSet[](package private) static int
LOG_OVERFLOW_EVENT
Event types for event manager thread.private long
lowestSafeLogKey
The logKey to be used by LogEventListener.logOverflowNotification when moving records forward into the current log file.(package private) byte[][]
markRecord
MARK control Record.(package private) java.nio.ByteBuffer
markRecordBB
ByteBuffer wrapper for markRecord to facilitate conversion of numeric information to byte[] format.(package private) int
maxBlocksPerFile
maximum number of blocks to store in each LogFile.private int
overflowNotificationCount
Number of times log overflow event was notified.(package private) boolean
restartAutoMark
The automark value restored during log file initialization.
-
Constructor Summary
Constructors Constructor Description LogFileManager(Configuration config)
construct LogFileManager with Configuration supplied by caller.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description (package private) void
close()
Gracefully close the log files.(package private) void
closeBufferManager()
Write a CLOSE record and shut down the buffer manager if we have one.private void
detectLogOverflow(int bsn)
Detect pending Log Overflow and notify event listener.(package private) long
getHighMark()
Returns the highMark from the LogFile that is currently being written.(package private) LogFile
getLogFileForMark(long mark)
Returns the LogFile that contains the requested mark .(package private) LogFile
getLogFileForWrite(LogBuffer lb)
Called by LogBuffer.init() to obtain the LogFile that will be used to write a specific log block.(package private) java.lang.String
getStats()
Returns an XML node containing statistics for the LogFileManager.(package private) void
init(LogBufferManager bmgr)
validate LogFiles and set member variables.(package private) long
mark(long key)
calls mark(key, false)(package private) long
mark(long key, boolean force)
sets the LogFile's mark.(package private) void
open()
open pool of LogFile(s).(package private) int
read(LogBuffer lb, int bsn)
reads a block of data into LogBuffer lb .(package private) long
setAutoMark(boolean automark)
Sets the LogFile marking mode.(package private) void
setCurrentKey(long key)
updates currentKey member variable.private boolean
setLockOnFile(java.io.File name, boolean lock)
Create a JVM wide lock on a File.(package private) void
setLogEventListener(LogEventListener eventListener)
Registers a LogEventListener for log event notifications.(package private) void
setMarkData(java.nio.ByteBuffer data)
generates MARKKEY data record into supplied data parameter.(package private) void
validateFileHeader(LogBuffer lb)
Compares values in log file header record with current configuration.
-
-
-
Field Detail
-
maxBlocksPerFile
int maxBlocksPerFile
maximum number of blocks to store in each LogFile.controls when logging is switched to a new log file, and/or when a circular log is reset to seek address zero.
- See Also:
getLogFileForWrite(LogBuffer)
-
activeMark
long activeMark
The log key for the oldest active entry in the log.When automark is enabled (true) the activeMark is updated after every put() operation. When automark is disabled (as should be the case with JOTM) the activeMark is updated manually by a call to Logger.mark(), or during logOverflowNotification processing if a LogEventListener is registered.
- See Also:
Logger.mark(long)
-
automark
boolean automark
indicates whether log files will be marked automatically.When automark is false, the mark() method must be invoked by the log user. When automark is true, the Logger will automatically set the mark at the most recent record.
-
restartAutoMark
boolean restartAutoMark
The automark value restored during log file initialization.if a set of log files exist from a prior execution, the value of automark is set based on the log file header record processed during the init() processing. The value restored from the log file is saved in restartAutoMark so it can be reported by getStats().
-
currentKey
long currentKey
last key returned by put().updated by setCurrentKey(long)
-
initialKey
private long initialKey
the first log key generated by this instance of the Logger.Initialized by init() to the block immediately following the last block written in the prior execution.
-
autoMarkOn
final byte[] autoMarkOn
data written to log when autoMark is turned on
-
autoMarkOff
final byte[] autoMarkOff
data written to log when autoMark is turned off
-
fileManagerLock
private final java.lang.Object fileManagerLock
lock controlling access to LogFile.
-
fileSet
LogFile[] fileSet
set of LogFile objects associated with the physical log files.- See Also:
open()
-
lfIndex
int lfIndex
workerID to current entry in fileSet[]
-
currentLogFile
LogFile currentLogFile
-
fileHeader
byte[][] fileHeader
LogFile header record.protected by fileManagerLock
The first record of every LogFile is a FILE_HEADER record containing information that is used during recovery to reposition the log file and replay records starting from the active mark. byte[1] autoMark byte[1] long activeMark byte[8] global to all log files long lowMark byte[8] low mark for current file == high mark for previous file long prevSwitchTod byte[8] time of previous file switch int fileSet.length byte[4] number of files in fileSet int maxBlocksPerFile byte[4] byte[2] crlf byte[2]
-
fileHeaderBB
java.nio.ByteBuffer fileHeaderBB
ByteBuffer wrapper for fileHeader to facilitate conversion of numeric information to byte[] format.protected by fileManagerLock
-
markRecord
byte[][] markRecord
MARK control Record.A MARK control record containing the current state of the automark mode and the active mark is stored at the beginning of every block. This strategy allows the logger to read through a log file looking only at the BSN for each block to locate the last written block. Once the last written block is located, the current state of the MARK can be obtained by examining the MARK record at the beginning of the block.
To avokd garbage collection for a byte[] and ByteBuffer wrapper for each physical block, the class has a single markRecord member that is protected by fileManagerLock.
protected by fileManagerLock byte[1] autoMark byte[1] long activeMark byte[8] global to all log files int fileSet.length byte[4] number of files in fileSet int maxBlocksPerFile byte[4] byte[2] crlf byte[2]
- See Also:
setMarkData(ByteBuffer)
-
markRecordBB
java.nio.ByteBuffer markRecordBB
ByteBuffer wrapper for markRecord to facilitate conversion of numeric information to byte[] format.protected by fileManagerLock
-
crlf
byte[] crlf
end of line for log records to make logs readable in text editors.
-
bmgr
LogBufferManager bmgr
LogBufferManager used by methods that put() records to the log.For example, the mark() method uses bmgr to put() mark information to the log.
-
eventListener
private LogEventListener eventListener
The LogEventListener registered by the application that owns the logger. If eventListener is null then the application is not notified of log events.
-
eventManagerLock
final java.lang.Object eventManagerLock
Monitor used by EventManager thread to wait for events that need to be notified.
-
eventManagerThread
java.lang.Thread eventManagerThread
Thread used by LogFileManager to send events to the registered LogEventListener.
-
event
int event
event to be processed by the eventManagerThread.When a condition is encountered that requires the LogEventListener to be notified, the value is set to one of the event types.
-
LOG_OVERFLOW_EVENT
static final int LOG_OVERFLOW_EVENT
Event types for event manager thread.assign this value to the event member when invoking the event manager thread to handle a log overflow notification.
- See Also:
- Constant Field Values
-
lowestSafeLogKey
private long lowestSafeLogKey
The logKey to be used by LogEventListener.logOverflowNotification when moving records forward into the current log file.
-
overflowNotificationCount
private int overflowNotificationCount
Number of times log overflow event was notified.
-
initComplete
private boolean initComplete
Indicates that LogFileManager initialization is complete.Prior to being fully initialized, the LogFileManager should not attempt to put records to the journal because the buffer manager may not be initialized properly.
-
-
Constructor Detail
-
LogFileManager
LogFileManager(Configuration config)
construct LogFileManager with Configuration supplied by caller.- Parameters:
config
- Configuration object.
-
-
Method Detail
-
getLogFileForMark
LogFile getLogFileForMark(long mark)
Returns the LogFile that contains the requested mark .Called by LogBufferManager to locate a log file needed for a replay() request.
- Parameters:
mark
- A log key previously returned by LogBufferManager.put(). The log key is used to compute the desired file.- Returns:
- LogFile containing the requested mark .
Returns null if none of the files in fileSet[] contain the requested mark.
-
getLogFileForWrite
LogFile getLogFileForWrite(LogBuffer lb) throws LogFileOverflowException
Called by LogBuffer.init() to obtain the LogFile that will be used to write a specific log block.The buffer sequence number of the LogBuffer parameter ( lf.bsn ) represents an implementation specific value that is used to manage log file space. As buffers are written to disk the buffer sequence number is incremented. The LogFileManager is able to compute the seek address for a buffer as a function of lf.bsn and buffer size when using LogBuffer implementations with fixed buffer sizes.
In all cases, getLogFile records a header record into the buffer containing the current state of the automark mode and the current active mark.
- Parameters:
lb
- LogBuffer that is asking for the LogFile. LogFileManager implementations use lf.bsn to determine when to switch to a new file, or wrap a circular file back to seek address zero.- Returns:
- a LogFile to use for writing the LogBuffer
- Throws:
LogFileOverflowException
-
detectLogOverflow
private void detectLogOverflow(int bsn)
Detect pending Log Overflow and notify event listener.if current file is 50% full then we check to see if the next file contains the active mark. We notify the event listener to move records forward to prevent log overflow.
Called by: getLogFileForWrite(Logbuffer lb)
- Parameters:
bsn
- The block sequence number of the current LogBuffer.- See Also:
getLogFileForWrite(LogBuffer)
-
setMarkData
void setMarkData(java.nio.ByteBuffer data)
generates MARKKEY data record into supplied data parameter.- Parameters:
data
- ByteBuffer wrapping the target byte[]
-
mark
long mark(long key, boolean force) throws InvalidLogKeyException, java.io.IOException, java.lang.InterruptedException
sets the LogFile's mark.mark() provides a generalized method for callers to inform the Logger that log space can be released for reuse.
writes a MARKKEY control record to the log.
- Parameters:
key
- is an opaque log key returned by a previous call to put().force
- when set true causes the caller to be blocked until the new force record is forced to disk.- Returns:
- log key for the MARK record
- Throws:
InvalidLogKeyException
- if key parameter is out of range. key must be greater than current activeMark and less than the most recent key returned by put().java.io.IOException
java.lang.InterruptedException
-
mark
long mark(long key) throws InvalidLogKeyException, java.io.IOException, java.lang.InterruptedException
calls mark(key, false)- Parameters:
key
- a log key as described bymark(long,boolean)
- Returns:
- log key of the new mark record.
- Throws:
InvalidLogKeyException
java.io.IOException
java.lang.InterruptedException
- See Also:
mark(long, boolean)
-
read
int read(LogBuffer lb, int bsn) throws java.io.IOException, InvalidLogBufferException
reads a block of data into LogBuffer lb .Amount of data read is determined by lb.capacity().
sets lb.lf with the fileSet[] entry that contains the requested BSN.
- Parameters:
lb
- LogBuffer to read data into.bsn
- block sequence number of the block to be read. File position is computed as follows:- locate the entry within fileSet[] that contains the requested bsn.
- compute position as (requested bsn - first bsn in file) * block size;
- Returns:
- block serial number of block read.
returns -1 if the requested BSN does not exist in the current fileSet[].
- Throws:
java.io.IOException
InvalidLogBufferException
-
setAutoMark
long setAutoMark(boolean automark) throws InvalidLogKeyException, java.io.IOException, java.lang.InterruptedException, LogFileOverflowException
Sets the LogFile marking mode.writes an AUTOMARK control record to the log if the log is open.
- Parameters:
automark
- true to indicate automatic marking.- Returns:
- log key for the generated MARK control record.
- Throws:
InvalidLogKeyException
java.io.IOException
java.lang.InterruptedException
LogFileOverflowException
-
setCurrentKey
void setCurrentKey(long key)
updates currentKey member variable.Method must be synchronized to guarantee that only keys with larger values are assigned to currentKey.
If automark mode is enabled, then the activeMark is updated as well.
- Parameters:
key
- a log key returned by the buffer manager.
-
getHighMark
long getHighMark()
Returns the highMark from the LogFile that is currently being written.Log keys greater than this value are beyond the logical end of the journal.
- Returns:
- highMark from the current LogFile.
-
setLogEventListener
void setLogEventListener(LogEventListener eventListener)
Registers a LogEventListener for log event notifications.- Parameters:
eventListener
- object to be notified of logger events.
-
setLockOnFile
private boolean setLockOnFile(java.io.File name, boolean lock)
Create a JVM wide lock on a File.Feature 30922
The java.nio.channel.tryLock method is not guaranteed to respect locks that are set within the JVM by multiple threads. Consequently, it is possible for an application to create multiple instances of Logger resulting in corrupted log files as two separate instances of Logger write to the log.
To prevent multiple instances of Logger from allocating the same files within a JVM, we create a system property
- Parameters:
name
- File object to be locked- Returns:
- true if requested lock operation is successful.
-
open
void open() throws LogConfigurationException, java.io.IOException, java.io.FileNotFoundException, InvalidFileSetException
open pool of LogFile(s).- Throws:
java.io.FileNotFoundException
LogConfigurationException
java.io.IOException
InvalidFileSetException
-
init
void init(LogBufferManager bmgr) throws java.io.IOException, LogConfigurationException, InvalidLogBufferException, java.lang.InterruptedException
validate LogFiles and set member variables.activeMark set based on last block written during previous execution.
currentKey set to key of the last record written to the log.
currentLogFile set to the next available LogFile in fileSet[] with file position set to resume writing at the next block following the last block written.
- Throws:
java.io.IOException
LogConfigurationException
InvalidLogBufferException
java.lang.InterruptedException
-
validateFileHeader
void validateFileHeader(LogBuffer lb) throws LogConfigurationException, java.io.IOException, InvalidLogBufferException
Compares values in log file header record with current configuration.Throws LogConfigurationException if header does not match current configuration.
- Throws:
LogConfigurationException
java.io.IOException
InvalidLogBufferException
-
closeBufferManager
void closeBufferManager() throws java.lang.InterruptedException, java.io.IOException
Write a CLOSE record and shut down the buffer manager if we have one.This routine was originally inline in close(). It was refactored into a separate method to improve readability of close().
- Throws:
java.lang.InterruptedException
java.io.IOException
-
close
void close() throws java.io.IOException, java.lang.InterruptedException
Gracefully close the log files.- Throws:
java.io.IOException
- If FileChannel.close() encounters an error.java.lang.InterruptedException
- See Also:
AbstractInterruptibleChannel.close()
-
getStats
java.lang.String getStats()
Returns an XML node containing statistics for the LogFileManager.The nested
element contain entries for each LogFile object in the set of log files. - Returns:
- a String containing statistics as an XML node.
-
-