Protocol Buffers
A language-neutral serialization format and interface definition language for defining service contracts and generating code.
Protocol Buffers (protobuf) is a binary serialization format and interface definition language developed by Google. You define your data structures and service methods in .proto files, then generate code for Go, Java, Python, and many other languages.
The main use case is gRPC, where protobuf defines the contract between services. You write a .proto file with your message types and RPC methods, run the code generator, and get strongly typed clients and servers. If you change the contract, the generated code changes too. Breaking changes show up as compile errors, not as mysterious runtime failures.
Protobuf's binary format is smaller and faster to parse than JSON. This matters for high-throughput service-to-service communication, but it makes messages harder to inspect manually. You can't just curl an endpoint and read the response. For public-facing APIs where human readability matters, OpenAPI with JSON is usually a better fit.
One of protobuf's strongest features is built-in support for schema evolution. Each field has a number that stays stable across versions. You can add new fields without breaking old consumers. They ignore what they don't recognize, and new consumers handle missing fields with defaults. This is the same idea behind Event Versioning strategies: make changes additive, never remove or rename.
Protobuf and OpenAPI serve a similar purpose. Both are API-first tools. OpenAPI targets HTTP/JSON APIs, while protobuf targets gRPC with binary serialization. gRPC's code generation is stricter. You can't accidentally return data that doesn't match the schema.
For event-driven systems, protobuf is a popular choice for serializing events. The schema evolution guarantees make it safer than JSON, where compatibility rules are informal and easy to break by accident.
References
- Robust gRPC communication on Google Cloud Run (but not only!) — Core post on gRPC in Go. Shows how to define services in .proto files, generate Go code with protoc, and use the generated clients and servers. Compares the strictness of Protobuf contracts with OpenAPI and highlights built-in support for field deprecation and backward compatibility.
- The Go libraries that never failed us: 22 libraries you need to know — Recommends protoc and the protoc Go Plugin for generating Go code from .proto files. Links to the Protocol Buffers Version 3 Language Specification and Well-Known Types.
- Common Anti-Patterns in Go Web Applications — Recommends generating gRPC models from Protobuf files as part of a code generation strategy. Generated code provides strong types and compile-time checks instead of passing interface{} to generic functions.
- 4 practical principles of high-quality database integration tests in Go — Notes that confidence in deployments increases when contracts between services are robust because of gRPC, Protobuf, or OpenAPI.
- Microservices test architecture. Can you sleep well without end-to-end tests? — Mentions that gRPC clients generated from Protobuf are straightforward to use in component tests, making it easy to call service methods without manual request construction.
- Introduction to DDD Lite: When microservices in Go are not enough — Demonstrates defining a gRPC service using .proto files with Protocol Buffers, generating Go code from the definition, and using generated clients for inter-service communication.
- SQLite Pub/Sub, Quickstart, and more — Watermill 1.5 Released — Announces deprecation of cqrs.ProtobufMarshaler in favor of cqrs.ProtoMarshaler, switching from github.com/gogo/protobuf to google.golang.org/protobuf.
- Live website updates with Go, SSE, and htmx — Mentions Protocol Buffers as one of the supported payload formats for Watermill messages alongside JSON and plain strings.
- Synchronous vs Asynchronous Architecture — Extensive discussion of Protobuf for event schemas and serialization. Covers using Protobuf for message contracts in async systems, its versioning support for evolving event schemas, and marshaling to both JSON and binary formats.
- DDD: A Toolbox, Not a Religion — Mentions Protocol Buffers over gRPC as the modern equivalent of XML over RPC, in the context of how DDD communication patterns have been reinvented over time.
- Watermill: from a hobby project to 8k stars on GitHub — Discusses Protobuf marshaling as one of the serialization formats Watermill handles automatically. Also mentions the protobuf marshaler naming challenge when deprecating the old version.
- When you shouldn't use frameworks in Go — Shares a cautionary tale about a custom Protobuf implementation that was no longer maintained, causing issues across tens of microservices. Illustrates the risk of depending on custom framework extensions.
