How to Register a Dataset Schema as JSON in Python

Define a catalog of dataset schemas and serialize them to JSON with the standard library json module.

Easy Python 3.9+ Aug 9, 2026 Data pipelines & processing 13 views 0 copies

json schema catalog serialization metadata

Python code

43 lines

Python 3.9+

import json

catalog = {
    "name": "sample_catalog",
    "version": "1.0",
    "datasets": [
        {
            "id": "users",
            "type": "table",
            "fields": [
                {"name": "id", "type": "integer", "key": True},
                {"name": "email", "type": "string", "nullable": False},
                {"name": "created_at", "type": "datetime"}
            ],
            "rows": 2,
            "tags": ["identity", "persona"]
        },
        {
            "id": "orders",
            "type": "table",
            "fields": [
                {"name": "order_id", "type": "integer", "key": True},
                {"name": "user_id", "type": "integer", "reference": "users.id"},
                {"name": "total", "type": "decimal"}
            ],
            "rows": 1,
            "tags": ["commerce"]
        }
    ]
}

schema_json = json.dumps(catalog, indent=2)

print(schema_json)

# Re-import the schema to demonstrate round-trip parsing
with open("catalog_schema.json", "w") as f:
    f.write(schema_json)

with open("catalog_schema.json") as f:
    loaded = json.load(f)

print("\nLoaded dataset ids:", [ds["id"] for ds in loaded["datasets"]])

Output

stdout

{
  "name": "sample_catalog",
  "version": "1.0",
  "datasets": [
    {
      "id": "users",
      "type": "table",
      "fields": [
        {
          "name": "id",
          "type": "integer",
          "key": True
        },
        {
          "name": "email",
          "type": "string",
          "nullable": False
        },
        {
          "name": "created_at",
          "type": "datetime"
        }
      ],
      "rows": 2,
      "tags": ["identity", "persona"]
    },
    {
      "id": "orders",
      "type": "table",
      "fields": [
        {
          "name": "order_id",
          "type": "integer",
          "key": True
        },
        {
          "name": "user_id",
          "type": "integer",
          "reference": "users.id"
        },
        {
          "name": "total",
          "type": "decimal"
        }
      ],
      "rows": 1,
      "tags": ["commerce"]
    }
  ]
}

Loaded dataset ids: ['users', 'orders']

How it works

The json.dumps call converts the nested Python dictionary into a JSON string with human-readable indentation. Writing the string to a file makes the catalog persistent and shareable across services. Calling json.load on the file reads it back into native Python objects, confirming a clean round-trip. The key/marker fields like key and nullable act as lightweight metadata for consumers to enforce constraints.

Common mistakes

Forgetting `indent=2` makes the output a single unreadable line
Mixing up `json.dumps` (to string) with `json.dump` (to file)
Writing raw JSON with `open().write()` instead of using `json.dump`

Variations

Use `json.dump(catalog, f, indent=2)` to write directly without an intermediate string
Validate the schema against a JSON Schema draft before registration

Real-world use cases

Persisting table definitions for a data warehouse so downstream ETL jobs know field types and keys.
Sharing dataset contracts between teams via a Git-tracked JSON file that triggers schema checks on PRs.
Bootstrapping a data catalog service that ingests these JSON definitions to auto-generate documentation and lineage.

How to Register a Dataset Schema as JSON in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Data pipelines & processing

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Data pipelines & processing

Keep learning

Tutorials

Quizzes