- Administration >
- Administration Concepts >
- Data Management >
- Capped Collections
Capped Collections¶
On this page
Capped collections are fixed-size collections that support high-throughput operations that insert, retrieve, and delete documents based on insertion order. Capped collections work in a way similar to circular buffers: once a collection fills its allocated space, it makes room for new documents by overwriting the oldest documents in the collection.
See createCollection()
or createCollection
for more information on creating capped collections.
Capped collections have the following behaviors:
- Capped collections guarantee preservation of the insertion order. As a result, queries do not need an index to return documents in insertion order. Without this indexing overhead, they can support higher insertion throughput.
- Capped collections guarantee that insertion order is identical to the order on disk (natural order) and do so by prohibiting updates that increase document size. Capped collections only allow updates that fit the original document size, which ensures a document does not change its location on disk.
- Capped collections automatically remove the oldest documents in the collection without requiring scripts or explicit remove operations.
For example, the oplog.rs collection that stores a log of the operations in a replica set uses a capped collection. Consider the following potential use cases for capped collections:
- Store log information generated by high-volume systems. Inserting documents in a capped collection without an index is close to the speed of writing log information directly to a file system. Furthermore, the built-in first-in-first-out property maintains the order of events, while managing storage use.
- Cache small amounts of data in a capped collections. Since caches are read rather than write heavy, you would either need to ensure that this collection always remains in the working set (i.e. in RAM) or accept some write penalty for the required index or indexes.
Recommendations and Restrictions¶
You can update documents in a collection after inserting them. However, these updates cannot cause the documents to grow. If the update operation causes the document to grow beyond their original size, the update operation will fail.
If you plan to update documents in a capped collection, create an index so that these update operations do not require a table scan.
You cannot delete documents from a capped collection. To remove all records from a capped collection, use the ‘emptycapped’ command. To remove the collection entirely, use the
drop()
method.You cannot shard a capped collection.
Capped collections created after 2.2 have an
_id
field and an index on the_id
field by default. Capped collections created before 2.2 do not have an index on the_id
field by default. If you are using capped collections with replication prior to 2.2, you should explicitly create an index on the_id
field.Warning
If you have a capped collection in a replica set outside of the
local
database, before 2.2, you should create a unique index on_id
. Ensure uniqueness using theunique: true
option to theensureIndex()
method or by using an ObjectId for the_id
field. Alternately, you can use theautoIndexId
option tocreate
when creating the capped collection, as in the Query a Capped Collection procedure.Use natural ordering to retrieve the most recently inserted elements from the collection efficiently. This is (somewhat) analogous to tail on a log file.
Procedures¶
Create a Capped Collection¶
You must create capped collections explicitly using the
createCollection()
method, which is a helper in the
mongo
shell for the create
command. When
creating a capped collection you must specify the maximum size of the
collection in bytes, which MongoDB will pre-allocate for the collection.
The size of the capped collection includes a small amount of space for
internal overhead.
Additionally, you may also specify a maximum number of documents for the
collection using the max
field as in the following document:
Important
The size
argument is always required, even when
you specify max
number of documents. MongoDB will remove older
documents if a collection reaches the maximum size limit before it
reaches the maximum document count.
See
createCollection()
and create
.
Query a Capped Collection¶
If you perform a find()
on a capped collection
with no ordering specified, MongoDB guarantees that the ordering of
results is the same as the insertion order.
To retrieve documents in reverse insertion order, issue
find()
along with the sort()
method with the $natural
parameter set to -1
, as shown
in the following example:
Check if a Collection is Capped¶
Use the isCapped()
method to determine if a
collection is capped, as follows:
Convert a Collection to Capped¶
You can convert a non-capped collection to a capped collection with
the convertToCapped
command:
The size
parameter specifies the size of the capped collection in
bytes.
Warning
This command obtains a global write lock and will block other operations until it has completed.
Changed in version 2.2: Before 2.2, capped collections did not have an index on _id
unless you specified autoIndexId
to the create
,
after 2.2 this became the default.
Automatically Remove Data After a Specified Period of Time¶
For additional flexibility when expiring data, consider MongoDB’s TTL indexes, as described in Expire Data from Collections by Setting TTL. These indexes allow you to expire and remove data from normal collections using a special type, based on the value of a date-typed field and a TTL value for the index.
TTL Collections are not compatible with capped collections.
Tailable Cursor¶
You can use a tailable cursor with capped collections. Similar to the
Unix tail -f
command, the tailable cursor “tails” the end of a
capped collection. As new documents are inserted into the capped
collection, you can use the tailable cursor to continue retrieving
documents.
See Create Tailable Cursor for information on creating a tailable cursor.