Protobuf Tooling

What we’re building

The buf toolchain, configured but not yet fed: buf.yaml (the proto workspace and its one external dependency) and buf.gen.yaml (the four plugins that will turn .proto files into Go code). The actual .proto contracts come in Module 2 — this lesson is entirely about having the tooling ready and verified before there’s anything to generate.

Why

Hand-rolling protobuf codegen means installing protoc plus a protoc-gen-go binary, a protoc-gen-go-grpc binary, and so on, keeping every contributor’s versions in sync by hand. buf replaces all of that: buf.gen.yaml references remote plugins hosted on the Buf Schema Registry (BSR), so buf generate downloads and runs the exact pinned plugin version for everyone — no local protoc-gen-* binaries to install at all. buf.yaml also gives us buf lint, which catches proto API design problems (bad naming, missing options) before they reach a code review.

Pros & cons

Pros

• No local protoc or protoc-gen-* binaries — plugins run remotely via the BSR and are version-pinned in buf.lock.
• buf lint enforces a consistent, well-known style (STANDARD rules) across every service’s .proto files automatically.
• One buf generate invocation runs all four plugins and writes everything into gen/.
• deps: pulls in google/api/annotations.proto from buf.build/googleapis/googleapis — the annotation type the gateway plugin needs — without vendoring it by hand.

Cons

• buf generate needs network access to the BSR the first time it resolves remote plugins (they’re cached locally afterward).
• It’s another config file and another CLI to learn, layered on top of protobuf itself.
• Pinned plugin versions need occasional, deliberate bumping — they won’t silently track upstream protoc-gen-go releases.

Set it up

1. Install buf

brew install bufbuild/buf/buf

Verify:

buf --version

2. buf.yaml — the proto workspace

This defines where your .proto files live, what lint rules apply, and the one external module dependency the gateway plugin needs:

version: v2
modules:
  - path: proto
deps:
  - buf.build/googleapis/googleapis
lint:
  use:
    - STANDARD

Save this as buf.yaml at the repo root. modules points at proto/ (currently empty — Module 2 fills it in); deps declares google/api/annotations.proto (from buf.build/googleapis/googleapis) as available to import, which is what lets a .proto file attach HTTP routes for grpc-gateway; lint.use: STANDARD turns on buf’s standard naming and style rules for every file under proto/.

3. buf.gen.yaml — the four codegen plugins

version: v2
clean: true
plugins:
  - remote: buf.build/protocolbuffers/go
    out: gen
    opt: paths=source_relative
  - remote: buf.build/grpc/go
    out: gen
    opt: paths=source_relative
  - remote: buf.build/grpc-ecosystem/gateway
    out: gen
    opt: paths=source_relative
  - remote: buf.build/grpc-ecosystem/openapiv2
    out: gen
inputs:
  - directory: proto

Save this as buf.gen.yaml at the repo root. clean: true wipes gen/ before each run, so it never accumulates stale files from renamed or deleted messages. paths=source_relative makes generated file paths mirror your .proto package layout instead of being flattened by Go import path — the convention this course uses throughout.

4. What each plugin produces (once Module 2 adds .proto files)

• protocolbuffers/go — the Go structs for every message: field getters, Marshal/Unmarshal, and everything encoding/json-adjacent code expects from a protobuf type.
• grpc/go — the gRPC client and server stubs for every service/rpc: a CatalogServiceClient interface to call, and a CatalogServiceServer interface each service implements.
• grpc-ecosystem/gateway — the reverse-proxy code that translates incoming REST/JSON requests into gRPC calls, driven by google.api.http annotations in the .proto files. This is what the API Gateway (Module 5) runs.
• grpc-ecosystem/openapiv2 — an OpenAPI (Swagger) v2 document generated from the same annotations, so the REST surface the gateway exposes is documented automatically, with zero hand-written API docs to keep in sync.

All four are configured to write into the single gen/ directory this course’s canonical layout uses.

Verify

buf --version
buf lint

proto/ is still empty at this point (Module 2 adds the first .proto file), and buf refuses to process a module with zero .proto files at all — for any subcommand, not just lint:

Failure: Module "path: "proto"" had no .proto files

That failure is expected, not a sign anything is broken: it proves buf.yaml parsed and buf found the proto module exactly where modules: - path: proto says to look — there’s just nothing in it yet. We can’t resolve the googleapis dependency yet either (buf dep update hits the same “had no .proto files” failure while the module is empty), so buf.lock doesn’t exist until Module 2. Once Module 2 lands the first real .proto files, buf dep update will succeed and write buf.lock, and from then on buf lint actually checks the files and buf generate produces real output in gen/ using the four plugins configured above.

Recap

buf is installed, and two config files are in place: buf.yaml defines the proto/ workspace, the STANDARD lint ruleset, and a deps: dependency on buf.build/googleapis/googleapis for the HTTP annotation types the gateway needs; buf.gen.yaml wires up four remote plugins — protocolbuffers/go (Go message types), grpc/go (client/server stubs), grpc-ecosystem/gateway (the REST-to-gRPC reverse proxy), and grpc-ecosystem/openapiv2 (generated API docs) — all writing into gen/. buf lint (and every other buf subcommand) refuses to run against an empty proto/ today with a “had no .proto files” failure — expected, since buf.yaml resolved correctly and there’s simply nothing in the module yet; Module 2 is where the first real .proto files arrive, buf dep update can finally write buf.lock, and this tooling starts producing generated Go code. Next, Infra & Compose → stands up the PostgreSQL, Kafka, and RabbitMQ this project runs against locally.