Convert Protobuf to JSON and Dict in Python
Provides static helper methods to convert between protobuf messages, JSON strings, and Python dictionaries using the google.protobuf library.
pip install protobuf
Python code
52 linesfrom google.protobuf.json_format import MessageToJson, Parse
import json
class DataConverter:
"""Helper class to convert between protobuf messages and common formats."""
@staticmethod
def to_json(message, indent=2):
"""Convert a protobuf message to JSON string."""
return MessageToJson(message, indent=indent)
@staticmethod
def from_json(json_str, message_type):
"""Parse JSON string back into a protobuf message."""
return Parse(json_str, message_type())
@staticmethod
def to_dict(message):
"""Convert a protobuf message to a Python dictionary (lowercase keys)."""
return json.loads(MessageToJson(message))
@staticmethod
def from_dict(data_dict, message_type):
"""Build a protobuf message from a Python dictionary."""
return Parse(json.dumps(data_dict), message_type())
if __name__ == "__main__":
# Example usage with a simple test message (using google.protobuf)
from google.protobuf import descriptor_pb2
# Create a FileDescriptorProto as sample message
msg = descriptor_pb2.FileDescriptorProto()
msg.name = "sample.proto"
msg.package = "example"
# Convert to JSON and back
json_output = DataConverter.to_json(msg)
print("JSON output:")
print(json_output)
# Parse back
restored = DataConverter.from_json(json_output, descriptor_pb2.FileDescriptorProto)
print("\nRestored name:", restored.name)
# Convert to dict and back
data_dict = DataConverter.to_dict(msg)
print("\nDict output:", data_dict)
restored_dict = DataConverter.from_dict(data_dict, descriptor_pb2.FileDescriptorProto)
print("Restored from dict:", restored_dict.name)
Output
JSON output:
{
"name": "sample.proto",
"package": "example"
}
Restored name: sample.proto
Dict output: {'name': 'sample.proto', 'package': 'example'}
Restored from dict: sample.proto
How it works
The MessageToJson function from google.protobuf.json_format serializes a protobuf message into a JSON string, preserving field names and indentation settings. The Parse function performs the reverse, converting valid JSON back into a protobuf message instance. Both functions rely on the message descriptor to map JSON fields to protobuf fields, ensuring type safety during round-trip conversions. Converting via JSON strings then loading with json.loads produces Python dictionaries, while from_dict serializes dictionaries to JSON before parsing.
Common mistakes
- Forgetting that field names in dicts are lowercase (proto3 default) when passing to from_dict
- Calling MessageToJson on a message with missing required fields, which raises an error
- Not passing an instantiated message type to Parse (must use message_type(), not the class itself)
- Assuming JSON key ordering matches proto field order
Variations
- Use MessageToDict and MessageFromDict directly for lossless conversion of scalar and enum types
- Set always_print_fields_with_no_presence=True in MessageToJson to include default-valued fields
Real-world use cases
- Serializing gRPC responses to JSON for REST facade endpoints that bridge protobuf services to web clients. Converting protobuf messages to Python dicts for logging or analytics pipelines that expect native Python data structures. Building protobuf messages from parsed JSON payloads in API gateway handlers or integratio
- Inside automation scripts and CLIs that need to convert protobuf to json and dict in python as one step of a larger job.
- In data-cleaning or ETL pipelines where you convert protobuf to json and dict in python before validating or storing records.
Sponsored
More from API design & gRPC
- Build a Bulk Array POST Mock Server in Python medium
- Build a Mock REST API with PUT and GET in Python medium
- Create a Data Helper in Python for gRPC-style APIs easy
- Format data in Python using dataclasses like gRPC messages easy
- Generate an OpenAPI Spec from Mock Routes in Python easy
- How to Add HATEOAS Links to a Python API Response easy
Keep learning
Related tutorials and quizzes for this topic.