MongoDB transactions are a powerful feature that lets you execute multiple operations across one or more documents atomically. That means either all the operations succeed, or none do – no partial writes. That is critical when consistency matters, like updating related data in multiple collections.
Unlike traditional relational databases, MongoDB initially focused on single-document atomicity, where each document update is atomic on its own. But as applications grew more complex, the need for multi-document transactions became pressing, and MongoDB added support for them starting in version 4.0.
At its core, a MongoDB transaction is a sequence of reads and writes that are grouped together. Once committed, the changes become visible all simultaneously. If something fails, the transaction aborts, and the database rolls back to the original state.
Transactions in MongoDB are session-scoped. You start a session, start a transaction on that session, do your reads and writes, then commit or abort. The driver keeps track of the transaction state so you don’t have to manually handle rollback logic.
Behind the scenes, MongoDB uses a combination of a write-ahead log and document-level locking to ensure consistency. This lets transactions scale well without blocking the entire database.
But before diving into code, remember that transactions come with overhead. They use more resources and can reduce throughput if overused. For many cases, careful schema design and single-document atomic operations suffice.
Still, when you do need transactions, understanding the lifecycle is key: start a session, start a transaction, execute operations, commit or abort, then end the session. The driver handles retries for transient errors, but you should be prepared to retry your transaction code too.
Here’s a very simple pseudo-flow of a transaction:
session = client.start_session()
session.start_transaction()
try:
collection1.update_one({"_id": 1}, {"$set": {"field": "value"}}, session=session)
collection2.insert_one({"related_id": 1, "data": "something"}, session=session)
session.commit_transaction()
except Exception as e:
session.abort_transaction()
raise e
finally:
session.end_session()
The key is passing the same session object to every operation inside the transaction, so MongoDB knows what to group.
Transactions also support read concerns and write concerns, which control the isolation and durability guarantees. The default read concern for transactions is “snapshot,” ensuring a consistent view of data throughout the transaction.
It’s important to note that MongoDB transactions are not unlimited. There’s a 60-second timeout by default before the transaction aborts automatically, and the total size of data written can be limited. These constraints encourage keeping transactions short and focused.
If you try to mix transactional and non-transactional operations in the same session, results can be unpredictable. Always encapsulate all related operations inside the transaction or outside it, but don’t mix.
Understanding these basics sets the stage for writing truly atomic, multi-document operations in your applications. Next, we’ll look at how to use Pymongo to write these atomic operations cleanly and idiomatically.
Writing atomic operations with Pymongo
When working with Pymongo, the first step is to start a client session, which acts as the transaction context. You then use start_transaction() on that session to begin your atomic block. All your CRUD operations inside that block must include the session parameter, ensuring they’re part of the same transaction.
Here’s a concise example where you transfer funds between two accounts atomically. Both the debit and credit updates must succeed or fail together:
from pymongo import MongoClient
from pymongo.errors import PyMongoError
client = MongoClient("mongodb://localhost:27017")
db = client.bank
accounts = db.accounts
def transfer_funds(sender_id, receiver_id, amount):
with client.start_session() as session:
try:
session.start_transaction()
sender = accounts.find_one({"_id": sender_id}, session=session)
if sender["balance"] < amount:
raise ValueError("Insufficient funds")
accounts.update_one(
{"_id": sender_id},
{"$inc": {"balance": -amount}},
session=session
)
accounts.update_one(
{"_id": receiver_id},
{"$inc": {"balance": amount}},
session=session
)
session.commit_transaction()
except (PyMongoError, ValueError) as e:
session.abort_transaction()
raise e
Notice how the session is passed explicitly to every operation inside the transaction block. If the debit update succeeds but the credit update fails, the whole transaction will abort, leaving balances unchanged.
For many use cases, a with block can simplify session handling, ensuring the session is cleaned up properly. Here’s a more idiomatic pattern that also handles transient errors with retries:
from pymongo.errors import (ConnectionFailure, OperationFailure, ConfigurationError)
import time
def run_transaction_with_retry(txn_func, session):
while True:
try:
txn_func(session)
break
except (ConnectionFailure, OperationFailure) as exc:
if exc.has_error_label("TransientTransactionError"):
time.sleep(1) # backoff before retrying
continue
else:
raise
def transfer_funds_retry(sender_id, receiver_id, amount):
def txn(session):
sender = accounts.find_one({"_id": sender_id}, session=session)
if sender["balance"] < amount:
raise ValueError("Insufficient funds")
accounts.update_one(
{"_id": sender_id},
{"$inc": {"balance": -amount}},
session=session
)
accounts.update_one(
{"_id": receiver_id},
{"$inc": {"balance": amount}},
session=session
)
with client.start_session() as session:
try:
session.start_transaction()
run_transaction_with_retry(txn, session)
session.commit_transaction()
except Exception:
session.abort_transaction()
raise
This pattern isolates the transactional logic in txn, which gets retried on transient errors like network blips or primary failovers. MongoDB marks those errors with a special label, so you know when retrying makes sense.
Another common pattern is batching multiple writes in a transaction to ensure consistency. For example, when creating an order and at once updating inventory counts:
def create_order(session, order_data, inventory_updates):
orders = db.orders
inventory = db.inventory
orders.insert_one(order_data, session=session)
for item_id, qty in inventory_updates.items():
inventory.update_one(
{"_id": item_id},
{"$inc": {"stock": -qty}},
session=session
)
Then you can wrap this in a transactional call:
with client.start_session() as session:
try:
session.start_transaction()
create_order(session, order_data, inventory_updates)
session.commit_transaction()
except Exception:
session.abort_transaction()
raise
Keep in mind that adding reads between writes inside the transaction can increase contention. If you’re reading and then writing based on that read, try to keep the transaction short to avoid conflicts with other operations.
Sometimes you want to specify explicit write and read concerns to control durability and isolation. You can do this when starting the transaction:
with client.start_session() as session:
session.start_transaction(
read_concern=ReadConcern("snapshot"),
write_concern=WriteConcern(w="majority")
)
# transactional operations here
session.commit_transaction()
Setting read_concern to snapshot ensures your reads see a consistent snapshot of data throughout the transaction, preventing phantom reads and non-repeatable reads. The write_concern set to majority guarantees that writes are acknowledged only after being replicated to a majority of nodes, improving durability.
But specifying these explicitly is rarely needed unless you have very specific consistency requirements. The defaults are usually sufficient.
In practice, you’ll often want to build utility functions around this pattern to keep your application logic clean and reduce boilerplate. Wrapping transactional retries, session management, and error handling into reusable components makes your code more robust and easier to maintain.
That said, Pymongo’s API is low-level, so you’re responsible for managing the session lifecycle carefully. Forgetting to pass the session to an operation, or mixing transactional and non-transactional operations, can cause subtle bugs or partial writes.
For example, this code silently breaks atomicity:
with client.start_session() as session:
session.start_transaction()
accounts.update_one({"_id": 1}, {"$set": {"balance": 100}}, session=session)
accounts.update_one({"_id": 2}, {"$set": {"balance": 200}}) # missing session!
session.commit_transaction()
The second update is outside the transaction, so if the commit fails, the first update may roll back but the second won’t. Always double-check that every operation inside the transaction uses the session parameter.
In the next section, we will explore common pitfalls when using transactions, including how to handle errors and avoid deadlocks or long-running transactions that can degrade performance or cause unexpected failures.
Dealing with common pitfalls and errors in transactions
One of the most frequent pitfalls when dealing with MongoDB transactions is exceeding the transaction lifetime. By default, a transaction times out after 60 seconds. If your operations are slow or blocked, the server aborts the transaction without warning, causing your commit to fail. To avoid this, keep transactions short and avoid any blocking calls inside the transaction scope.
Long-running transactions also hold locks on documents, increasing contention and reducing concurrency. If you see errors like LockTimeout or experience slow performance, review your transaction logic for unnecessary delays or heavy computations.
Another common error arises from transient network issues or failovers in a replica set. These lead to errors labeled TransientTransactionError. The recommended approach is to catch these errors and retry the entire transaction, as shown earlier. However, be careful not to retry non-idempotent operations blindly. Design your transaction function so that it can safely run multiple times without side effects.
Deadlocks can occur if two transactions try to update the same documents in opposite order, causing each to wait for the other. MongoDB detects deadlocks and aborts one of the transactions with an error labeled UnknownTransactionCommitResult or WriteConflict. Your retry logic should handle these errors by aborting and retrying the transaction from scratch.
Here’s a robust retry loop pattern that handles common transient errors, including deadlocks and network issues:
from pymongo.errors import (ConnectionFailure, OperationFailure, WriteConflictError)
import time
def with_transaction_retry(txn_func):
max_retries = 5
for attempt in range(max_retries):
with client.start_session() as session:
try:
session.start_transaction()
txn_func(session)
session.commit_transaction()
return
except (ConnectionFailure, WriteConflictError, OperationFailure) as e:
if e.has_error_label("TransientTransactionError") or e.has_error_label("UnknownTransactionCommitResult"):
session.abort_transaction()
time.sleep(2 ** attempt) # exponential backoff
continue
else:
session.abort_transaction()
raise
except Exception:
session.abort_transaction()
raise
raise RuntimeError("Transaction failed after max retries")
This pattern retries the entire transaction up to a maximum number of times, backing off exponentially between attempts. It checks for MongoDB’s error labels to decide which errors are safe to retry.
Another subtle issue is mixing causal consistency with transactions. By default, sessions are causally consistent, which can sometimes cause unexpected stale reads inside a transaction if you reuse a session across multiple transactions without resetting it. If you see inconsistent reads, consider creating a fresh session for each transaction or explicitly disabling causal consistency:
with client.start_session(causal_consistency=False) as session:
session.start_transaction()
# transaction operations
session.commit_transaction()
Keep in mind that causal consistency is a useful feature for many applications, but it can introduce complexity when combined with transactions and retries.
Another common mistake is trying to perform transactions on collections that do not support them. Transactions require replica sets or sharded clusters. Standalone MongoDB servers do not support multi-document transactions. Attempting to start a transaction on a standalone will result in errors like NotSupported. Always check your deployment architecture before relying on transactions.
In sharded clusters, transactions span multiple shards, which adds latency and complexity. Avoid large transactions that touch many shards or update large amounts of data. Instead, try to design your schema and queries to localize transactions to a single shard when possible.
Finally, watch out for error handling that swallows exceptions or fails to abort transactions. If you catch exceptions but forget to call abort_transaction(), the session will remain in a pending transaction state, potentially causing resource leaks or blocking subsequent operations on that session.
Here’s an example of incorrect error handling that leads to problems:
with client.start_session() as session:
session.start_transaction()
try:
accounts.update_one({"_id": 1}, {"$inc": {"balance": -100}}, session=session)
# something goes wrong here
except Exception:
print("Error occurred, but forgot to abort transaction")
session.commit_transaction() # this will fail or commit partial changes
Always pair your transaction logic with explicit abort calls in except blocks or use finally to ensure cleanup.
Handling transactions well means respecting their constraints and error semantics. By structuring your code to retry transient failures, keep transactions short, and carefully manage sessions, you prevent many common bugs and improve your application's robustness.
