05 - Schema Evolution

Your proto files will change. You'll add fields, deprecate old ones, and restructure messages. Protobuf is designed for this, but there are rules. Break them and you'll corrupt data or crash services.

The Golden Rule

Never change the field number of an existing field. Field numbers are the identity of each field on the wire. If you change a number, old clients will read the wrong data.

// Version 1
message Link {
  int64 id = 1;
  string url = 2;
  string short_code = 3;
}

If you change url from field 2 to field 5, any old client that encoded url as field 2 will now decode it as... nothing. And whatever was at field 5 before will be read as url. Silent data corruption.

Adding Fields

Adding new fields is always safe. Old clients ignore fields they don't know about. New clients see zero values for fields that old messages don't include.

// Version 2 — added clicks and status
message Link {
  int64 id = 1;
  string url = 2;
  string short_code = 3;
  int64 clicks = 4;          // new
  LinkStatus status = 5;     // new
}

An old client that only knows fields 1 through 3 will skip fields 4 and 5 when decoding. A new client reading an old message will see clicks = 0 and status = LINK_STATUS_UNSPECIFIED. No crashes, no corruption.

Removing Fields

You can stop using a field, but you should never reuse its number. If you remove field 3 and later add a new field 3 with a different type, old messages with the original field 3 will be misinterpreted.

Use reserved to prevent accidental reuse:

message Link {
  int64 id = 1;
  string url = 2;
  reserved 3;              // was short_code, don't reuse
  reserved "short_code";   // prevent reuse of the name too
  int64 clicks = 4;
  string slug = 6;         // new field, new number
}

reserved tells the compiler to reject any future use of that field number or name. It's documentation and enforcement in one.

Renaming Fields

Field names don't matter on the wire. Only numbers do. You can rename a field freely:

// Before
message Link {
  string short_code = 3;
}

// After — same field number, different name
message Link {
  string slug = 3;
}

Old and new binaries will interoperate fine because they both use field number 3. But be careful: if you use protojson for JSON serialization, the JSON field name changes. That could break HTTP clients.

Changing Types

Some type changes are compatible, most are not.

Safe changes:

• int32 to int64 (or vice versa, with truncation risk)
• uint32 to uint64
• string to bytes (if the bytes are valid UTF-8)

Unsafe changes:

• int32 to string (completely different wire format)
• string to int64 (will corrupt data)
• repeated to non-repeated (different encoding)

When in doubt, don't change the type. Add a new field with the new type and deprecate the old one.

Deprecating Fields

Mark fields as deprecated to signal they shouldn't be used:

message Link {
  int64 id = 1;
  string url = 2;
  string short_code = 3 [deprecated = true];
  string slug = 6;
}

This generates a // Deprecated comment in Go, and some linters will warn on usage. The field still works on the wire, it's just a hint to developers.

Evolving Enums

Adding new enum values is safe. Old clients will see the raw integer for values they don't recognize.

enum LinkStatus {
  LINK_STATUS_UNSPECIFIED = 0;
  LINK_STATUS_ACTIVE = 1;
  LINK_STATUS_DISABLED = 2;
  LINK_STATUS_EXPIRED = 3;
  LINK_STATUS_ARCHIVED = 4;  // new, safe to add
}

Never reuse enum numbers. Never change the meaning of existing values. If ACTIVE = 1 meant "publicly visible" and you want it to mean "visible but unlisted," add a new value instead.

Practical Workflow

In gRPC, the server is the service that implements the RPC methods (our link shortener). The client is any program that calls those methods (a CLI tool, another microservice, a web backend). Both sides deserialize protobuf messages, so both are affected by schema changes.

1. Add new fields with new numbers. Never reuse old numbers.
2. Reserve removed field numbers and names.
3. Deploy the new server first (it can handle old and new messages).
4. Then deploy new clients.
5. Once all clients are updated, you can stop populating deprecated fields.

This order matters. If you deploy clients first, they'll send fields the server doesn't understand. The server will ignore them (which is fine), but the response won't include the new fields the client expects.

Key Takeaways

• Never change or reuse field numbers
• Adding fields is always safe, old clients ignore unknown fields
• Use reserved to prevent accidental reuse of removed field numbers
• Field names can change freely (they're not sent over the wire)
• Mark deprecated fields with [deprecated = true]
• Deploy servers before clients when evolving schemas
• When in doubt, add a new field instead of changing an existing one