MongoDB Time-Series Data

Ever tried to store millions of sensor readings, stock ticks, or server metrics in a regular MongoDB collection, only to watch your queries crawl and your storage balloon? The pain is real: standard collections aren't optimized for the relentless flow of time-stamped data, leading to slow range queries, massive index overhead, and painful data pruning. In this lesson, you'll learn how to store and query time-series data in MongoDB using a purpose-built collection type that slashes storage and accelerates time-based queries.

The problem this lesson solves

Traditional MongoDB collections treat each document as a standalone entity, which is great for general-purpose data. But time-series data—like IoT sensor temperature readings every second, stock prices every millisecond, or website metrics every minute—has unique characteristics:

• High write volume: You're constantly inserting new data points, often at a high rate.
• Time-centric queries: You need to fetch values between timestamps, aggregate over intervals, or find the latest measurements.
• Data aging: Old data is often less valuable and needs to be efficiently pruned or compressed.
• Large storage footprint: Storing one document per data point inflates storage and index size, which impacts performance and cost.

In a regular collection, a time-based range query like db.sensors.find({timestamp: {$gte: start, $lte: end}}) might scan many documents, and even with an index on timestamp, you'll still use significant disk and memory. Plus, managing data retention (e.g., delete data older than 30 days) becomes a maintenance nightmare.

The solution is MongoDB's time-series collections, introduced in MongoDB 5.0. They are designed to handle exactly this kind of data efficiently.

Core concept / mental model

Think of a time-series collection as a smart storage container that automatically groups related data points into buckets based on a time field and an optional metaField.

If you're familiar with time-series databases like InfluxDB, you'll catch on quickly. In MongoDB, you don't interact with buckets directly—MongoDB manages them transparently. You still query the collection as if it were a normal collection, and MongoDB transparently unwraps the buckets for you.

Define the key terms:

• timeField: A required field in each document that holds the timestamp (BSON Date type). MongoDB uses this field to bucket data.
• metaField: An optional field that acts as a metadata tag to group related series. For example, sensor_id or hostname. Combining timeField and metaField allows MongoDB to group data points for the same sensor into the same bucket, improving compression and query speed.

Why bucketing works: Instead of storing each data point as a separate document with all its overhead, MongoDB stores many measurements from the same metadata group in a single internal bucket document. This reduces storage footprint significantly (because of automatic compression) and makes time-range queries faster, as MongoDB can skip irrelevant buckets using the bucket's min/max timestamps.

For example, data in a measurements collection might look like:

{ "timestamp": ISODate("2023-01-01T00:00:00Z"), "device": "sensor-A", "temp": 21.5 },
{ "timestamp": ISODate("2023-01-01T00:00:01Z"), "device": "sensor-A", "temp": 21.6 },
{ "timestamp": ISODate("2023-01-01T00:00:00Z"), "device": "sensor-B", "temp": 22.0 }

Internally, MongoDB groups the sensor-A documents into one bucket, and sensor-B into another bucket.

How it works step by step

Now let's walk through the process of creating and using a time-series collection.

Step 1: Create a time-series collection

You create a time-series collection using the create command or a db.createCollection() helper. You must specify the timeseries option with at least the timeField. You can also set metaField and granularity.

db.createCollection(
  "sensor_data",
  {
    timeseries: {
      timeField: "timestamp",
      metaField: "metadata",
      granularity: "seconds"
    }
  }
)

• timeField: the name of the field storing the timestamp (must be a BSON Date).
• metaField: optional—a field name for metadata that groups related series.
• granularity: 'seconds', 'minutes', or 'hours'—tells MongoDB the expected time span between consecutive data points. This influences bucketing efficiency.

Step 2: Insert documents

You insert documents like you would in a normal collection. MongoDB automatically organizes them into buckets.

db.sensor_data.insertMany([
  {
    "metadata": { "device": "sensor-A" },
    "timestamp": new Date("2023-01-01T00:00:00Z"),
    "temperature": 21.5,
    "humidity": 55.2
  },
  {
    "metadata": { "device": "sensor-A" },
    "timestamp": new Date("2023-01-01T00:00:01Z"),
    "temperature": 21.6,
    "humidity": 55.3
  },
  {
    "metadata": { "device": "sensor-B" },
    "timestamp": new Date("2023-01-01T00:00:00Z"),
    "temperature": 22.0,
    "humidity": 60.1
  }
])

Step 3: Query time-series data

You query a time-series collection using standard MongoDB query methods. Time-based range queries are automatically optimized.

// Find all readings for sensor-A between two timestamps
db.sensor_data.find({
  "metadata.device": "sensor-A",
  "timestamp": {
    $gte: ISODate("2023-01-01T00:00:00Z"),
    $lte: ISODate("2023-01-01T00:00:10Z")
  }
})

Step 4: Aggregate data over time intervals

Use the aggregation pipeline to compute summaries (averages, min, max) over time buckets. $dateTrunc (or $dateToString) helps group by time intervals.

db.sensor_data.aggregate([
  {
    $match: {
      "metadata.device": "sensor-A",
      "timestamp": { $gte: ISODate("2023-01-01T00:00:00Z") }
    }
  },
  {
    $group: {
      _id: { $dateTrunc: { date: "$timestamp", unit: "minute" } },
      avgTemp: { $avg: "$temperature" },
      maxTemp: { $max: "$temperature" },
      count: { $sum: 1 }
    }
  },
  { $sort: { _id: 1 } }
])

Step 5: Manage data retention

Set a time-to-live (TTL) index on the time field to automatically delete old data. In a time-series collection, the TTL index is automatically created on the timeField when you enable expireAfterSeconds (available from MongoDB 5.2 onward).

db.runCommand({
  collMod: "sensor_data",
  timeseries: { expireAfterSeconds: 2592000 } // 30 days
})

Hands-on walkthrough

Let's put it all together in a complete example. Open your mongosh and run the following.

Create a time-series collection

// Connect to your database
db = db.getSiblingDB("iot_db")

// Create a time-series collection
db.createCollection(
  "environment_metrics",
  {
    timeseries: {
      timeField: "timestamp",
      metaField: "location",
      granularity: "seconds"
    }
  }
)

Insert sample data

// Insert 3 readings from two locations
db.environment_metrics.insertMany([
  { location: "room-1", timestamp: new Date(), temperature: 22.1, humidity: 45 },
  { location: "room-1", timestamp: new Date(Date.now() + 1000), temperature: 22.3, humidity: 45.2 },
  { location: "room-2", timestamp: new Date(), temperature: 21.5, humidity: 50 }
])

// Verify the inserted documents
db.environment_metrics.find().pretty()

Expected output (timestamps will differ):

{ "_id": ObjectId("..."), "location": { "room": "room-1" }, "timestamp": ISODate("..."), "temperature": 22.1, "humidity": 45 },
{ "_id": ObjectId("..."), "location": { "room": "room-1" }, "timestamp": ISODate("..."), "temperature": 22.3, "humidity": 45.2 },
...

Query data for the last 5 minutes

const fiveMinutesAgo = new Date(Date.now() - 5 * 60 * 1000)

db.environment_metrics.find({
  "location.room": "room-1",
  timestamp: { $gte: fiveMinutesAgo }
}).sort({ timestamp: 1 })

Compute average temperature per location per minute

db.environment_metrics.aggregate([
  {
    $match: {
      timestamp: { $gte: fiveMinutesAgo }
    }
  },
  {
    $group: {
      _id: {
        room: "$location.room",
        minute: { $dateTrunc: { date: "$timestamp", unit: "minute" } }
      },
      avgTemperature: { $avg: "$temperature" },
      readings: { $sum: 1 }
    }
  },
  { $sort: { "_id.minute": 1 } }
])

This aggregation returns average temperature per room per minute, which is exactly what a dashboard would need.

Compare options / when to choose what

You might wonder, why not just create an index on timestamp in a regular collection? Here's a quick comparison:

Use time-series collections when:

• You have a high-frequency data stream (e.g., IoT, financial tick data).
• You need efficient range queries over timestamps.
• You want automatic compression and data expiration.

Stick with regular collections when:

• Your data is not timestamp-centric.
• You need to frequently update/delete individual documents (time-series collections are append-only; updates and deletes are restricted, though you can delete entire buckets indirectly).
• Your schema changes frequently and you need flexible documents.

Troubleshooting & edge cases

Can't create a time-series collection on an existing collection

If you get an error like Cannot create a time-series collection from an existing collection, that's because time-series collections must be created from scratch. You can't convert a regular collection to a time-series collection. You'll need to create a new collection and migrate data.

Timestamps are not in BSON Date format

If you insert a timestamp as a string or a number, MongoDB will throw a WriteError. The timeField must store a BSON Date. Convert strings to dates using new Date("2023-01-01T00:00:00Z").

$dateTrunc not available

$dateTrunc was introduced in MongoDB 5.0. If you're on an older version, use $dateToString with a format that truncates to minute/hour, or use the deprecated $dateToString: { format: "%Y-%m-%dT%H:%M:00Z", date: "$timestamp" }.

TTL index not working

In MongoDB below 5.2, you cannot create a TTL index on a time-series collection's timeField manually; you must use the collMod command or create the collection with expireAfterSeconds from the start (MongoDB 5.2+). Verify your MongoDB version and use db.runCommand({ collMod: ... }) as shown.

Incorrect granularity setting can reduce performance

If you set granularity to "hours" but you're writing every second, buckets may hold too many data points, affecting query performance. Match granularity to your actual data frequency.

What you learned & what's next

You now understand how to store and query time-series data in MongoDB. You learned that time-series collections use a timeField to bucket data, how to create one, insert data, run time-based queries, and aggregate over intervals. You also saw how to set data retention with TTL and how to choose between time-series and regular collections.

You've achieved both learning objectives: explaining the core concept and completing a practical exercise. Now you're ready to move to the next lesson in the MongoDB path, where you'll explore more advanced querying patterns or aggregation frameworks to derive deeper insights from your data.