# Databases & Storage Reliability — Part 8: MongoDB In Depth > **Series:** Databases & Storage Reliability (8 of 8) > **Part 1:** `01-replication-and-failover.md` — Replication & Failover > **Part 2:** `02-sharding-and-partitioning.md` — Sharding & Partitioning > **Part 3:** `03-backup-recovery-and-durability.md` — Backup, Recovery & Durability > **Part 4:** `04-sql-fundamentals.md` — SQL Language Fundamentals > **Part 5:** `05-nosql-database-types.md` — NoSQL Database Types > **Part 6:** `06-mysql-in-depth.md` — MySQL In Depth > **Part 7:** `07-dynamodb-in-depth.md` — DynamoDB In Depth > **Part 8:** This file — MongoDB In Depth > **Questions:** `questions.md` ## Table of Contents 1. [Why MongoDB Gets Its Own Dedicated Part](#why-mongodb-gets-its-own-dedicated-part) 2. [The Document Model, Concretely](#the-document-model-concretely) 3. [Schema Design: Embedding vs Referencing](#schema-design-embedding-vs-referencing) 4. [Replica Sets — MongoDB's Native Replication](#replica-sets--mongodbs-native-replication) 5. [Elections and Failover, Step by Step](#elections-and-failover-step-by-step) 6. [Write and Read Concerns](#write-and-read-concerns) 7. [Sharding a MongoDB Cluster](#sharding-a-mongodb-cluster) 8. [Choosing a Shard Key (Again — MongoDB-Specific Gotchas)](#choosing-a-shard-key-again--mongodb-specific-gotchas) 9. [Indexes in MongoDB](#indexes-in-mongodb) 10. [The Aggregation Pipeline](#the-aggregation-pipeline) 11. [Transactions in MongoDB](#transactions-in-mongodb) 12. [Essential Operational Commands](#essential-operational-commands) 13. [Backup and Recovery](#backup-and-recovery) 14. [MongoDB vs DynamoDB — A Direct Comparison](#mongodb-vs-dynamodb--a-direct-comparison) 15. [Common Mistakes](#common-mistakes) 16. [Worked Practice Problems](#worked-practice-problems) 17. [Summary and What's Next](#summary-and-whats-next) --- ## Why MongoDB Gets Its Own Dedicated Part Part 5 introduced MongoDB as the flagship example of a **document database**. It's the natural companion to Part 7's DynamoDB deep dive: both are widely deployed, both are document-oriented, but they diverge sharply in operating model — DynamoDB is fully managed with an opinionated, constrained design; MongoDB (self-hosted, or via the managed Atlas service) gives you a much richer query language and more schema flexibility, at the cost of more operational responsibility. Seeing both in equal depth makes that tradeoff concrete rather than abstract. --- ## The Document Model, Concretely A MongoDB **document** is essentially a JSON object (stored internally in a binary format called **BSON** — Binary JSON — which adds types like dates and precise numeric types that plain JSON lacks). Documents live in **collections**, which are the MongoDB analog of a SQL table. ```mermaid graph TD DB["Database: shop"] --> Coll["Collection: orders"] Coll --> Doc1["Document:
{ _id: 1, customer: 'alice',
items: [...], total: 49.99 }"] Coll --> Doc2["Document:
{ _id: 2, customer: 'bob',
total: 12.50, gift_note: 'enjoy!' }"] ``` **The same schema-flexibility point already made about document databases generally in Part 5 applies directly here: `Doc2` has a `gift_note` field that `Doc1` doesn't, and MongoDB doesn't complain** — there's no `ALTER TABLE` step (Part 4) required to add a new field to future documents. This is a genuine strength for rapidly evolving applications, and a genuine risk for large teams without discipline, as covered in Part 5's "schema-on-read" tradeoff discussion. ```bash # Insert a document via the mongo shell (mongosh) mongosh --eval ' db.orders.insertOne({ customer: "alice", items: [{ sku: "A100", qty: 2 }], total: 49.99, created_at: new Date() }) ' ``` --- ## Schema Design: Embedding vs Referencing The single most important MongoDB-specific design decision — directly extending Part 4's normalization discussion, but from the opposite direction. ```mermaid graph TD Embed["EMBEDDING: put related
data INSIDE the parent
document (e.g. an order's
line items live inside the
order document itself)"] --> EmbedNote["✅ ONE read gets everything
⚠️ Document can grow large;
duplicated data if the same
info appears in many places"] Ref["REFERENCING: store just
an ID, and look up the
related document
separately (like a SQL
foreign key, Part 4)"] --> RefNote["✅ No duplication, smaller
documents
⚠️ Requires a SECOND query
(or a \\$lookup join) to
assemble full data"] ``` **A genuinely important, frequently-tested interview framing, worth stating explicitly: "MongoDB's schema design philosophy deliberately inverts the normalization principle from Part 4 — instead of designing the schema first and asking how to query it, you design AROUND your actual query patterns, exactly the same 'access patterns first' philosophy covered for DynamoDB's single-table design in Part 7. The rule of thumb: embed data that's ALWAYS read together and doesn't grow unbounded (like an order's line items); reference data that's independently queried, shared across many parents, or could grow without limit (like a product catalog referenced by thousands of orders)."** --- ## Replica Sets — MongoDB's Native Replication Directly, precisely the primary-replica replication pattern from Part 1 of this series, as MongoDB's own built-in, native implementation. ```mermaid graph TD Primary["PRIMARY node
(accepts ALL writes)"] -->|"oplog
(operation log)"| Secondary1["SECONDARY node 1
(replicates writes)"] Primary -->|"oplog"| Secondary2["SECONDARY node 2
(replicates writes)"] Primary -->|"oplog"| Arbiter["ARBITER
(votes in elections,
holds NO data)"] ``` **Why the "oplog" name is worth knowing precisely: it's MongoDB's own version of the write-ahead log / binlog concept already covered for PostgreSQL (Part 1) and MySQL (Part 6) — a continuously-appended, ordered record of every write, which secondaries replay to stay in sync.** A minimum production replica set is 3 data-bearing nodes (or 2 data nodes + 1 arbiter) specifically to guarantee a majority can always be reached for elections — directly reusing the quorum/majority-voting concept from the Reliability & Architecture Patterns series. ```bash # Initialize a 3-node replica set mongosh --eval ' rs.initiate({ _id: "rs0", members: [ { _id: 0, host: "mongo1:27017" }, { _id: 1, host: "mongo2:27017" }, { _id: 2, host: "mongo3:27017" } ] }) ' # Check replica set status and see who is currently PRIMARY mongosh --eval 'rs.status()' ``` --- ## Elections and Failover, Step by Step The exact same automated-failover pattern already covered generically in Part 1, now made concrete for MongoDB's specific mechanism. ```mermaid sequenceDiagram participant Sec1 as Secondary 1 participant Sec2 as Secondary 2 participant Primary Note over Primary: Primary crashes -
stops sending heartbeats Sec1->>Sec1: Detects missed heartbeats
(after electionTimeoutMillis,
default 10s) Sec1->>Sec2: Requests votes -
"I want to be PRIMARY" Sec2->>Sec2: Checks candidate's oplog is
at least as up to date as
its own Sec2->>Sec1: Grants vote Note over Sec1: Wins majority of votes -
becomes new PRIMARY Sec1->>Sec1: Begins accepting writes ``` **Why the "at least as up to date" check matters, worth calling out explicitly: MongoDB will not elect a secondary that's behind on replication as the new primary, specifically to avoid the split-brain and data-loss scenarios already discussed generically in the Reliability & Architecture Patterns series** — a node lagging behind loses the election even if it responds first, protecting data integrity over pure speed of failover. --- ## Write and Read Concerns MongoDB's own concrete, tunable exposure of the same consistency-vs-durability-vs-latency tradeoffs already discussed abstractly via CAP/PACELC in the Reliability & Architecture Patterns series. ```bash # Write concern: how many replica set members must # acknowledge a write before it's considered successful db.orders.insertOne( { customer: "alice", total: 49.99 }, { writeConcern: { w: "majority", wtimeout: 5000 } } ) # Read concern: what guarantee a read gives about the # freshness/durability of the data it returns db.orders.find({ customer: "alice" }).readConcern("majority") ``` | Write Concern | Meaning | Tradeoff | |---|---|---| | `w: 1` (default) | Only the primary must acknowledge | Fastest, but a primary crash right after can lose the write | | `w: "majority"` | A majority of replica set members must acknowledge | Slower, but the write survives any single-node failure | | `w: 0` | Fire-and-forget, no acknowledgment | Fastest possible, but the application has zero confirmation the write even happened | **Why `w: "majority"` is the production-safe default worth defending in an interview, directly connecting to Part 1's durability discussion: it guarantees a committed write has been replicated to enough nodes to survive an election, so a client that received a success response will never see that write silently disappear after a failover — exactly the same durability guarantee semi-synchronous replication provides in Part 1's MySQL/PostgreSQL discussion, just exposed as a per-operation setting instead of a server-wide configuration.** --- ## Sharding a MongoDB Cluster The same horizontal-partitioning concept from Part 2, as MongoDB's specific architecture for it. ```mermaid graph TD App["Application"] --> Router["mongos
(query router)"] Router --> ConfigServers["Config Server
Replica Set
(stores cluster
metadata: which shard
has which data)"] Router --> Shard1["Shard 1
(itself a replica set)"] Router --> Shard2["Shard 2
(itself a replica set)"] Router --> Shard3["Shard 3
(itself a replica set)"] ``` **A genuinely important structural point worth stating precisely: each individual "shard" in a MongoDB sharded cluster is ITSELF a full replica set** — sharding and replication are stacked, not alternatives. This directly combines the two techniques from Parts 1 and 2 of this series: replication provides high availability WITHIN a shard, sharding provides horizontal scale ACROSS shards. `mongos` (the query router) and the config servers exist specifically to make this composition transparent to the application, which just talks to `mongos` as if it were a single database. --- ## Choosing a Shard Key (Again — MongoDB-Specific Gotchas) The exact same hot-shard danger from Part 2 and Part 7 (DynamoDB), with MongoDB's own specific terminology and mechanisms. ```mermaid graph TD Bad["Shard key = created_at
(a MONOTONICALLY
increasing timestamp)"] --> BadNote["🚨 ALL new writes target
the SAME shard (whichever
owns the 'latest' range) —
the exact same monotonic-
key hot-shard problem
already flagged in Part 2"] Good["Shard key = hashed
customer_id"] --> GoodNote["✅ Hashed sharding spreads
writes evenly, same
consistent-hashing principle
as Part 2 and Part 7"] ``` ```bash # Enable sharding on a database and collection, # using a HASHED shard key to avoid the monotonic-key trap sh.enableSharding("shop") sh.shardCollection("shop.orders", { customer_id: "hashed" }) ``` **Why this is worth flagging as a recurring, cross-database pattern rather than a MongoDB-only quirk, a strong interview line: "Whether it's MongoDB's shard key, DynamoDB's partition key (Part 7), or a hand-rolled sharding scheme (Part 2), the SAME underlying failure mode keeps showing up — a low-cardinality or monotonically increasing key concentrates load on one physical partition no matter how much total capacity the cluster has. MongoDB's `hashed` shard key type exists specifically as a built-in guard against the monotonic-key version of this mistake."** --- ## Indexes in MongoDB Directly extending the indexing concepts from Part 4 (SQL) and Part 6 (MySQL/InnoDB) — MongoDB indexes are, under the hood, the same B-tree structure. ```bash # Create a single-field index db.orders.createIndex({ customer: 1 }) # 1 = ascending # Create a compound index (order of fields matters, # exactly like the leftmost-prefix rule in Part 6) db.orders.createIndex({ customer: 1, created_at: -1 }) # Check whether a query is actually using an index db.orders.find({ customer: "alice" }).explain("executionStats") ``` **Why the leftmost-prefix callback to Part 6 matters here too, worth stating explicitly: a compound index on `{ customer: 1, created_at: -1 }` can efficiently serve a query filtering on `customer` alone, or on `customer` AND `created_at` together, but NOT a query filtering on `created_at` alone — the exact same leftmost-prefix rule already covered for MySQL's InnoDB indexes in Part 6, because both are, structurally, B-tree indexes.** A query missing this index entirely triggers a `COLLSCAN` (collection scan) in `explain()` output — the MongoDB name for exactly the same full-table-scan problem flagged in Parts 4 and 6. --- ## The Aggregation Pipeline MongoDB's answer to SQL's `GROUP BY`/`JOIN`/analytical queries (Part 4) — a sequence of data-transformation stages. ```mermaid graph LR Input["orders collection"] --> Match["$match:
filter documents
(like SQL WHERE)"] Match --> Group["$group:
aggregate by field
(like SQL GROUP BY)"] Group --> Sort["$sort:
order results
(like SQL ORDER BY)"] Sort --> Output["Final result set"] ``` ```bash # Aggregation pipeline: total spend per customer, # for orders over $10, sorted highest first db.orders.aggregate([ { $match: { total: { $gt: 10 } } }, { $group: { _id: "$customer", totalSpend: { $sum: "$total" } } }, { $sort: { totalSpend: -1 } } ]) ``` **A directly useful mental-model bridge for someone coming from SQL (Part 4), worth stating plainly: "the aggregation pipeline is conceptually the exact same idea as a SQL query built from `WHERE`, `GROUP BY`, and `ORDER BY` clauses (Part 4) — just expressed as an explicit, ordered sequence of transformation stages instead of a single declarative statement. `$match` is `WHERE`, `$group` is `GROUP BY`, `$sort` is `ORDER BY`, and `$lookup` is the closest thing to a `JOIN`."** --- ## Transactions in MongoDB Directly connects to Part 4's ACID transaction discussion — MongoDB added true multi-document ACID transactions in version 4.0, a genuinely significant evolution from its early reputation. ```bash # A multi-document transaction in the mongo shell session = db.getMongo().startSession() session.startTransaction({ readConcern: { level: "snapshot" }, writeConcern: { w: "majority" } }) try { session.getDatabase("shop").accounts.updateOne({ _id: "alice" }, { $inc: { balance: -50 } }) session.getDatabase("shop").accounts.updateOne({ _id: "bob" }, { $inc: { balance: 50 } }) session.commitTransaction() } catch (e) { session.abortTransaction() throw e } ``` **Why this is a genuinely important point to get right in an interview, worth stating precisely: "A SINGLE document write in MongoDB has always been atomic — a document either fully updates or doesn't, with no partial-write risk, since a document is the natural atomic unit. What version 4.0 added was atomicity ACROSS multiple documents/collections in one transaction — the exact same all-or-nothing guarantee from Part 4's ACID discussion, now available for the cases where good schema design (embedding, from earlier in this file) alone can't avoid needing it."** Because embedding-first design already keeps most related data in one document, multi-document transactions are used less often in MongoDB than in a typical relational schema — but they're a real, fully-supported safety net when genuinely needed. --- ## Essential Operational Commands ```bash # Check overall server/replica set status mongosh --eval 'rs.status()' mongosh --eval 'db.serverStatus()' # Check current database and collection sizes mongosh --eval 'db.stats()' mongosh --eval 'db.orders.stats()' # Check current operations in progress (useful for # diagnosing a slow or "stuck" node) mongosh --eval 'db.currentOp()' # Kill a long-running, problematic operation mongosh --eval 'db.killOp()' # View slow queries via the built-in profiler mongosh --eval 'db.setProfilingLevel(1, { slowms: 100 })' mongosh --eval 'db.system.profile.find().sort({ ts: -1 }).limit(5)' ``` --- ## Backup and Recovery Directly extending the backup/recovery discipline from Part 3. ```bash # Logical backup: dumps data as BSON files mongodump --uri="mongodb://localhost:27017" --out=/backup/2026-08-18 # Restore from a logical backup (to a NEW instance/database # first, per Part 3's "never restore over a live system" rule) mongorestore --uri="mongodb://localhost:27017" /backup/2026-08-18 # For sharded clusters, coordinate backups across shards # consistently using the balancer-stop pattern to avoid # capturing chunks mid-migration sh.stopBalancer() # ... perform per-shard backups ... sh.startBalancer() ``` **Why stopping the balancer before a sharded-cluster backup matters, worth explaining: the balancer continuously moves data chunks between shards to keep the cluster evenly loaded (directly related to the resharding/rebalancing discussion in Part 2) — backing up while a chunk migration is mid-flight risks capturing an inconsistent snapshot where a chunk appears in neither, or both, shard backups. Pausing it during the backup window is a simple, real operational safeguard.** --- ## MongoDB vs DynamoDB — A Direct Comparison Closing the loop between Parts 7 and 8, both concrete document-oriented databases with sharply different operating philosophies. | Dimension | DynamoDB (Part 7) | MongoDB (Part 8) | |---|---|---| | Operating model | Fully managed only (AWS) | Self-hosted, or managed (Atlas) — real choice | | Query flexibility | Deliberately constrained — access patterns must be modeled upfront | Rich query language, ad hoc queries and aggregation pipeline supported | | Schema philosophy | Single-table design for known access patterns | Embed-vs-reference, generally more per-collection flexibility | | Scaling | Automatic (on-demand) or provisioned, fully managed | Manual replica set + sharded cluster setup, or managed via Atlas | | Multi-region | Global Tables, last-writer-wins | Replica sets can span regions; more manual tuning of read/write concerns | | Transactions | Limited, item/transaction API | Full multi-document ACID transactions since v4.0 | | Best fit | Massive, predictable scale with well-known access patterns | Evolving schemas, richer query needs, teams wanting more control | **A strong, senior-level closing line tying Parts 5, 7, and 8 together: "Both are document-oriented NoSQL databases at heart, but they sit at different points on the flexibility-vs-operational-simplicity spectrum from Part 5's decision framework — DynamoDB trades query flexibility for operational simplicity and near-infinite managed scale; MongoDB trades some of that operational simplicity for a genuinely richer query language and more schema flexibility. Neither is universally 'better' — the right choice depends on how well-known the access patterns are and how much operational ownership the team wants."** --- ## Common Mistakes | Mistake | Why It's Wrong | Fix | |---|---|---| | Using a monotonically increasing field (like `created_at`) as a shard key | Every new write targets whichever shard owns the "latest" range — a hot shard, identical to the problem flagged for DynamoDB in Part 7 | Use a hashed shard key, or a high-cardinality field that doesn't increase monotonically | | Embedding unbounded arrays (e.g. "all of a user's activity log," ever) inside a single document | MongoDB documents have a 16MB size limit; unbounded embedding eventually breaks | Reference unbounded, ever-growing data instead of embedding it | | Running `w: 0` (fire-and-forget) writes in production for important data | Gives zero confirmation a write even reached the primary, let alone replicated | Use `w: "majority"` for any write whose durability actually matters | | Ignoring `explain()` output and assuming an index is being used | A missing or wrong-order compound index silently falls back to a full `COLLSCAN` | Regularly run `explain("executionStats")` on important queries, exactly as recommended for SQL `EXPLAIN` in Part 4 | | Treating a 2-node replica set (no arbiter, no third member) as production-safe | A single node failure leaves no majority available to elect a new primary — the cluster can't accept writes | Always run an odd number of voting members (3+ data nodes, or 2 + an arbiter) | | Backing up a sharded cluster without stopping the balancer | Risks capturing an inconsistent snapshot mid-chunk-migration | Stop the balancer for the backup window, then restart it | --- ## Worked Practice Problems **Problem 1:** A team embeds a customer's entire order history (going back years) as an array inside that customer's document, reasoning "we always want to see a customer's orders together." After 18 months in production, they start hitting document-size errors on their most active customers. What went wrong, and how would you redesign it? *Answer:* This is a classic unbounded-embedding mistake — MongoDB documents have a hard 16MB size limit, and "a customer's entire order history, forever" is an unbounded, ever-growing dataset, exactly the kind of relationship this tutorial flags as a poor fit for embedding. The fix: switch to referencing — store orders as their own collection, each with a `customer_id` field pointing back to the customer document (the same foreign-key pattern from Part 4), and use an index on `customer_id` to efficiently query "all orders for this customer" via a normal `find()` (or `$lookup` in an aggregation pipeline) instead of embedding. **Problem 2:** A 3-node replica set (1 primary, 2 secondaries) loses network connectivity between the primary and BOTH secondaries simultaneously, though the primary itself is still running and reachable by the application. What happens, and why is this the correct, safe behavior? *Answer:* The primary can no longer reach a majority of voting members (it can only "see" itself — 1 out of 3), so per MongoDB's majority-based safety design, it automatically steps DOWN from primary to secondary and stops accepting writes, even though it's technically still running and reachable by clients. This is deliberate, safe behavior: without a majority, the primary can't be sure it isn't in a network-partitioned minority (directly reusing the quorum/split-brain-prevention concept from the Reliability & Architecture Patterns series) — continuing to accept writes in that state risks creating conflicting, unreconcilable data if the "other side" of the partition also has a primary. The application will see write failures until connectivity is restored and a new primary is elected by a real majority. **Problem 3:** An application team reports that a query filtering on `{ status: "pending" }` is slow, despite the collection having an index on `{ customer_id: 1, created_at: -1 }`. What's the likely cause, and what would you check first? *Answer:* The existing compound index starts with `customer_id`, and per the leftmost-prefix rule (shared with SQL/InnoDB indexing in Parts 4 and 6), a query filtering ONLY on `status` — a field that isn't the leftmost field of any existing index — can't use that index at all, and falls back to a full collection scan (`COLLSCAN`). I'd first confirm this by running `db.orders.find({status: "pending"}).explain("executionStats")` and checking the `winningPlan` stage — if it shows `COLLSCAN` instead of `IXSCAN`, that confirms the diagnosis. The fix is a dedicated index on `{ status: 1 }` (or a compound index starting with `status`, if `status` is commonly queried alongside other fields). --- ## Summary and What's Next - MongoDB's **document model** stores JSON-like BSON documents in collections, with genuine per-document schema flexibility — the same schema-on-read tradeoff introduced generically in Part 5. - **Embedding vs. referencing** is MongoDB's central design decision, directly inverting Part 4's normalization-first approach in favor of designing around actual query/access patterns — the same philosophy as DynamoDB's single-table design in Part 7. - **Replica sets** are MongoDB's native implementation of Part 1's primary-replica replication, using the **oplog** (MongoDB's version of a binlog/WAL) and majority-vote elections to guarantee safe, quorum-based failover. - **Write and read concerns** are MongoDB's per-operation, tunable exposure of the durability-vs-latency tradeoffs discussed abstractly via CAP/PACELC — `w: "majority"` is the production-safe default. - **Sharded MongoDB clusters stack sharding on top of replication** — each shard is itself a full replica set — and shard key choice carries the exact same hot-partition risk already seen for DynamoDB (Part 7) and generic sharding (Part 2). - **Indexes** are B-trees with the same leftmost-prefix rule already covered for MySQL/InnoDB in Part 6; `explain()` is the direct MongoDB analog of SQL `EXPLAIN` from Part 4. - The **aggregation pipeline** is MongoDB's structured, stage-based equivalent of a SQL query built from `WHERE`/`GROUP BY`/`ORDER BY`. - **Multi-document ACID transactions**, added in v4.0, provide the same all-or-nothing guarantee as Part 4's ACID discussion, for the cases good schema design alone can't avoid. **This completes the Databases & Storage Reliability series.** Across all eight parts, the throughline has been consistent: replication and quorum-based failover (Part 1), sharding and hot-partition avoidance (Part 2), safe backup and recovery discipline (Part 3), the relational foundation (Part 4), the NoSQL landscape and decision framework (Part 5), and three concrete, deeply examined real-world databases — MySQL (Part 6), DynamoDB (Part 7), and MongoDB (Part 8) — each showing the same small set of core reliability principles applied through a different lens. See `questions.md` in this folder for the full interview question bank covering all eight parts.