- MongoDB CRUD Operations >
- MongoDB CRUD Tutorials >
- Perform Two Phase Commits
Perform Two Phase Commits¶
Synopsis¶
This document provides a pattern for doing multi-document updates or “transactions” using a two-phase commit approach for writing data to multiple documents. Additionally, you can extend this process to provide a rollback like functionality.
Background¶
Operations on a single document are always atomic with MongoDB databases; however, operations that involve multiple documents, which are often referred to as “transactions,” are not atomic. Since documents can be fairly complex and contain multiple “nested” documents, single-document atomicity provides necessary support for many practical use cases.
Thus, without precautions, success or failure of the database operation cannot be “all or nothing,” and without support for multi-document transactions it’s possible for an operation to succeed for some operations and fail with others. When executing a transaction composed of several sequential operations the following issues arise:
- Atomicity: if one operation fails, the previous operation within the transaction must “rollback” to the previous state (i.e. the “nothing,” in “all or nothing.”)
- Isolation: operations that run concurrently with the transaction operation set must “see” a consistent view of the data throughout the transaction process.
- Consistency: if a major failure (i.e. network, hardware) interrupts the transaction, the database must be able to recover a consistent state.
Despite the power of single-document atomic operations, there are cases that require multi-document transactions. For these situations, you can use a two-phase commit, to provide support for these kinds of multi-document updates.
Because documents can represent both pending data and states, you can use a two-phase commit to ensure that data is consistent, and that in the case of an error, the state that preceded the transaction is recoverable.
Note
Because only single-document operations are atomic with MongoDB, two-phase commits can only offer transaction-like semantics. It’s possible for applications to return intermediate data at intermediate points during the two-phase commit or rollback.
Pattern¶
Overview¶
The most common example of transaction is to transfer funds from
account A to B in a reliable way, and this pattern uses this operation
as an example. In a relational database system, this operation would
encapsulate subtracting funds from the source (A
) account and
adding them to the destination (B
) within a single atomic
transaction. For MongoDB, you can use a two-phase commit in these
situations to achieve a compatible response.
All of the examples in this document use the mongo
shell to
interact with the database, and assume that you have two collections:
First, a collection named accounts
that will store data about
accounts with one account per document, and a collection named
transactions
which will store the transactions themselves.
Begin by creating two accounts named A
and B
, with the
following command:
To verify that these operations succeeded, use find()
:
mongo
will return two documents that
resemble the following:
Transaction Description¶
Set Transaction State to Initial¶
Create the transaction
collection by inserting the following
document. The transaction document holds the source
and
destination
, which refer to the name
fields of the
accounts
collection, as well as the value
field that
represents the amount of data change to the balance
field. Finally, the state
field reflects the current state of the
transaction.
To verify that these operations succeeded, use find()
:
This will return a document similar to the following:
Switch Transaction State to Pending¶
Before modifying either records in the accounts
collection, set
the transaction state to pending
from initial
.
Set the local variable t
in your shell session, to the transaction
document using findOne()
:
After assigning this variable t
, the shell will return the value
of t
, you will see the following output:
Use update()
to change the value of
state
to pending
:
The find()
operation will return the
contents of the transactions
collection, which should resemble the
following:
Apply Transaction to Both Accounts¶
Continue by applying the transaction to both accounts using the
update()
method.
In the query for the update()
, the condition
pendingTransactions: {$ne: t._id}
prevents the update from applying
the transaction t
to an account if the pendingTransactions
field for the account contains the _id
of the transaction t
:
The find()
operation will return the
contents of the accounts
collection, which should now resemble the
following:
Set Transaction State to Committed¶
Use the following update()
operation
to set the transaction’s state to committed
:
The find()
operation will return the
contents of the transactions
collection, which should now resemble
the following:
Recovering from Failure Scenarios¶
The most important part of the transaction procedure is not the prototypical example above, but rather the possibility for recovering from the various failure scenarios when transactions do not complete as intended. This section will provide an overview of possible failures and provide methods to recover from these kinds of events.
There are two classes of failures:
all failures that occur after the first step (i.e. setting the transaction set to initial) but before the third step (i.e. applying the transaction to both accounts.)
To recover, applications should get a list of transactions in the
pending
state and resume from the second step (i.e. switching the transaction state to pending.)all failures that occur after the third step (i.e. applying the transaction to both accounts) but before the fifth step (i.e. setting the transaction state to done.)
To recover, application should get a list of transactions in the
committed
state and resume from the fourth step (i.e. remove the pending transaction.)
Thus, the application will always be able to resume the transaction and eventually arrive at a consistent state. Run the following recovery operations every time the application starts to catch any unfinished transactions. You may also wish run the recovery operation at regular intervals to ensure that your data remains in a consistent state.
The time required to reach a consistent state depends, on how long the application needs to recover each transaction.
Rollback¶
In some cases you may need to “rollback” or undo a transaction when the application needs to “cancel” the transaction, or because it can never recover as in cases where one of the accounts doesn’t exist, or stops existing during the transaction.
There are two possible rollback operations:
- After you apply the transaction (i.e. the third step), you have fully committed the transaction and you should not roll back the transaction. Instead, create a new transaction and switch the values in the source and destination fields.
- After you create the transaction (i.e. the first step), but before you apply the transaction (i.e the third step), use the following process:
Set Transaction State to Canceling¶
Begin by setting the transaction’s state to canceling
using the
following update()
operation:
Multiple Applications¶
Transactions exist, in part, so that several applications can create and run operations concurrently without causing data inconsistency or conflicts. As a result, it is crucial that only one 1 application can handle a given transaction at any point in time.
Consider the following example, with a single transaction
(i.e. T1
) and two applications (i.e. A1
and A2
). If both
applications begin processing the transaction which is still in the
initial
state (i.e. step 1), then:
A1
can apply the entire whole transaction beforeA2
starts.A2
will then applyT1
for the second time, because the transaction does not appear as pending in theaccounts
documents.
To handle multiple applications, create a marker in the transaction
document itself to identify the application that is handling the
transaction. Use findAndModify()
method to modify the transaction:
When you modify and reassign the local shell variable t
, the
mongo
shell will return the t
object, which should
resemble the following:
Amend the transaction operations to ensure that only applications
that match the identifier in the value of the application
field
before applying the transaction.
If the application A1
fails during transaction execution, you can
use the recovery procedures, but
applications should ensure that they “owns” the transaction before
applying the transaction. For example to resume pending jobs, use a
query that resembles the following:
This will (or may) return a document from the transactions
document that resembles the following:
Using Two-Phase Commits in Production Applications¶
The example transaction above is intentionally simple. For example, it assumes that:
- it is always possible roll back operations on an account.
- account balances can hold negative values.
Production implementations would likely be more complex. Typically accounts need information about current balance, pending credits, pending debits. Then:
- when your application switches the transaction state to pending (i.e. step 2) it would also make sure that the account has sufficient funds for the transaction. During this update operation, the application would also modify the values of the credits and debits as well as adding the transaction as pending.
- when your application removes the pending transaction (i.e. step 4) the application would apply
the transaction on balance, modify the credits and debits as well as
removing the transaction from the
pending
field., all in one update.
Because all of the changes in the above two operations occur within a
single update()
operation, these
changes are all atomic.
Additionally, for most important transactions, ensure that:
- the database interface (i.e. client library or driver) has a reasonable write concern configured to ensure that operations return a response on the success or failure of a write operation.
- your
mongod
instance has journaling enabled to ensure that your data is always in a recoverable state, in the event of an uncleanmongod
shutdown.