When running with journaling, MongoDB stores and applies write operations in memory and in the journal before the changes are in the data files. This document discusses the implementation and mechanics of Journaling in MongoDB systems, see Manage Journaling for information on configuring, tuning and managing journaling.
With journaling enabled, MongoDB creates a journal directory within the directory defined by dbpath, which is /data/db by default. The journal directory holds journal files, which contain write-ahead redo logs. The directory also holds a last-sequence-number file. A clean shutdown removes all the files in the journal directory.
Journal files are append-only files and have file names prefixed with j._. When a journal file holds 1 gigabyte of data, MongoDB creates a new journal file. Once MongoDB applies all the write operations in the journal files, it deletes these files. Unless you write many bytes of data per-second, the journal directory should contain only two or three journal files.
To speed the frequent sequential writes that occur to the current journal file, you can ensure that the journal directory is on a different system.
If you place the journal on a different filesystem from your data files you cannot use a filesystem snapshot to capture consistent backups of a dbpath directory.
Depending on your file system, you might experience a preallocation lag the first time you start a mongod instance with journaling enabled.
MongoDB may preallocate journal files if the mongod process determines that it is more efficient to preallocate journal files than create new journal files as needed. The amount of time required to pre-allocate lag might last several minutes, during which you will not be able to connect to the database. This is a one-time preallocation and does not occur with future invocations.
To avoid preallocation lag, see Avoid Preallocation Lag.
Journaling adds three storage views to MongoDB.
The shared view stores modified data for upload to the MongoDB data files. The shared view is the only view with direct access to the MongoDB data files. When running with journaling, mongod asks the operating system to map your existing on-disk data files to the shared view memory view. The operating system maps the files but does not load them. MongoDB later loads data files to shared view as needed.
The journal is an on-disk view that stores new write operations after MongoDB applies the operation to the private cache but before applying them to the data files. The journal provides durability. If the mongod instance were to crash without having applied the writes to the data files, the journal could replay the writes to the shared view for eventual upload to the data files.
MongoDB copies the write operations to the journal in batches called group commits. See journalCommitInterval for more information on the default commit interval. These “group commits” help minimize the performance impact of journaling.
Journaling stores raw operations that allow MongoDB to reconstruct the following:
As write operations occur, MongoDB writes the data to the private view in RAM and then copies the write operations in batches to the journal. The journal stores the operations on disk to ensure durability. MongoDB adds the operations as entries on the journal’s forward pointer. Each entry describes which bytes the write operation changed in the data files.
MongoDB next applies the journal’s write operations to the shared view. At this point, the shared view becomes inconsistent with the data files.
At default intervals of 60 seconds, MongoDB asks the operating system to flush the shared view to disk. This brings the data files up-to-date with the latest write operations.
When MongoDB flushes write operations to the data files, MongoDB removes the write operations from the journal’s behind pointer. The behind pointer is always far back from the advanced pointer.
As part of journaling, MongoDB routinely asks the operating system to remap the shared view to the private view, for consistency.
The interaction between the shared view and the on-disk data files is similar to how MongoDB works without journaling, which is that MongoDB asks the operating system to flush in-memory changes back to the data files every 60 seconds.