Protocol Buffers Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data.

• How to install Protobuf on Ubuntu

see also

• Reverse Engineering Protobuf Definitions from Compiled Binaries
• Replacing Protobuf with Rust to go 5 times faster - The problem they have software written in Rust, and they need to use the libpg_query library, that is written in C. Because they can’t use the C library directly, they had to use a Rust-to-C binding library, that uses Protobuf for portability reasons. Problem is that it is slow.
  • So what they did is that they wrote their own non-portable but much more optimized Rust-to-C bindings, with the help of a LLM.
  • Calling the C function is not the problem here. It is dealing with the big data structure this function returns in a Rust-friendly manner.
    • This is something Protobuf does very well, at the cost of performance.

Alernatives

Cap’n Proto

Cap’n Proto is the result of years of experience working on Protobufs, listening to user feedback, and thinking about how things could be done better.

flatbuffers

Binary message format faster than protobuf

FlatBuffers is an efficient cross platform serialization library for C++, C#, C, Go, Java, JavaScript, TypeScript, PHP, and Python. It was originally created at Google for game development and other performance-critical applications.

Overview (source FlatBuffers)

• Cap’n’Proto promises to reduce Protocol Buffers much like FlatBuffers does, though with a more complicated binary encoding and less flexibility (no optional fields to allow deprecating fields or serializing with missing fields for which defaults exist). It currently also isn’t fully cross-platform portable (lack of VS support).

• msgpack: has very minimal forwards/backwards compatibility support when used with the typed C++ interface. Also lacks VS2010 support.

• Thrift: very similar to Protocol Buffers, but appears to be less efficient, and have more dependencies.

• YAML: a superset of JSON and otherwise very similar. Used by e.g. Unity.

• C# comes with built-in serialization functionality, as used by Unity also. Being tied to the language, and having no automatic versioning support limits its applicability.

• Project Anarchy (the free mobile engine by Havok) comes with a serialization system, that however does no automatic versioning (have to code around new fields manually), is very much tied to the rest of the engine, and works without a schema to generate code (tied to your C++ class definition).