How to Encode and Decode Avro Data in Python (Roundtrip)

Serialize a Python dict to Avro binary bytes and decode it back using the fastavro-compatible avro library.

Medium Python 3.9+ Aug 9, 2026 Streaming & messaging 14 views 0 copies

avro serialization encode decode binary streaming
Requires third-party packages — install first
pip install avro

Python code

34 lines
Python 3.9+ 
import io
import json
from avro.schema import parse
from avro.io import DatumWriter, DatumReader, BinaryEncoder, BinaryDecoder

def avro_roundtrip(schema_json, data):
    schema = parse(json.dumps(schema_json))
    bytes_writer = io.BytesIO()
    encoder = BinaryEncoder(bytes_writer)
    writer = DatumWriter(schema)
    writer.write(data, encoder)

    raw_bytes = bytes_writer.getvalue()
    bytes_reader = io.BytesIO(raw_bytes)
    decoder = BinaryDecoder(bytes_reader)
    reader = DatumReader(schema)
    decoded = reader.read(decoder)
    return raw_bytes, decoded

if __name__ == "__main__":
    schema = {
        "type": "record",
        "name": "User",
        "fields": [
            {"name": "name", "type": "string"},
            {"name": "age", "type": "int"},
            {"name": "active", "type": "boolean"}
        ]
    }
    original_data = {"name": "Alice", "age": 30, "active": True}
    encoded_bytes, decoded_data = avro_roundtrip(schema, original_data)
    print(f"Encoded bytes: {encoded_bytes}")
    print(f"Decoded data: {decoded_data}")
    print(f"Roundtrip successful: {decoded_data == original_data}")

Output

stdout 
Encoded bytes: b'\x06Alice\x00<\x01'
Decoded data: {'name': 'Alice', 'age': 30, 'active': True}
Roundtrip successful: True

How it works

The avro.schema.parse call converts a JSON schema into a Schema object that DatumWriter and DatumReader understand. BinaryEncoder writes Avro binary format to an in-memory BytesIO stream, so no temp files are needed. The writer encodes each field according to the schema's types—strings as length-prefixed UTF-8, ints as zigzag varint, booleans as a single byte. Reading with the same schema reconstructs the original data structure exactly. This pattern is the foundation for any Avro-based serialization pipeline, from Kafka messages to data-lake files.

Common mistakes

  • Forgetting to pass a schema object, not a raw dict, to DatumWriter/DatumReader.
  • Mixing encoder/decoder types, e.g., using BinaryDecoder with JSON encoder.
  • Ignoring schema evolution — a different schema on read side will break decoding.

Variations

  1. Use `fastavro` for a faster serializer with a similar API.
  2. Write to a file with `DataFileWriter` and `DataFileReader` for persistent Avro files.

Real-world use cases

  • Serializing events before publishing them to a Kafka topic that uses Avro as its wire format.
  • Encoding user profiles into Avro files in a data lake for Spark and Hive to consume.
  • Sending binary payloads over a message broker while keeping the schema central in a Schema Registry.

Sponsored

Run locally

This sample needs third-party packages, so it cannot run in the browser IDE. Copy the code above, install the packages shown at the top, then run it in your own Python environment.

More from Streaming & messaging

Keep learning

Related tutorials and quizzes for this topic.

Tutorials

Quizzes