Quick gRPC

Welcome to Quick gRPC

Ready to jump ship WSGI and join the microservices party? Let’s gRPC and roll!

https://ashutalks.us-cdp2.choreoapps.dev/quickgrpc/quickgrpc.slide#1

But REST api's are great... Why bother?

• Great for frontend integrations
• Easy to document with openapi spec
• clearly defined methods for various actions
• proxy level authz/ authn
• lots of mature frameworks in all langs
• custom serialization for obscure use cases

Are they though? 🫤

• Serialization/Validation? Optional. Good luck!
• Send a tangled JSON and hope for the best.
• Mismatched content types? Human error bingo!
• Bidirectional? Only if you're into websockets.
• Manage client SDKs? Or gamble on that OpenAPI spec.

So what exactly is gRPC

• Protocol: Think of HTTP/2 as the speedy highway for your data, zipping it around efficiently.
• Serialization: Protocol Buffers (protobuf) are like a universal translator, helping different services speak the same language.
• Language Support: Supports multiple languages, so you can have a polyglot party and everyone gets along!
• Streaming: Handles requests and responses like a pro, even when they’re long-winded or chatty.

Protobuf

• Two parts: messages and services.
• Messages are just structs/classes defining the schema, if you squint.
• Services are bundles of RPCs—input a message, get one back. Think Django viewsets, but less painful.
• At runtime, protobuf skips the string drama and goes straight to binary.
• Auto-generates server and client code, like serializers or Pydantic models, but smarter.
• Paves the way for standardized SDKs that won’t break on a whim.

Let's build one

We'll build a simple crud service

Let's define the protobud in a .proto file

We start a proto file by specifying the syntax version, like so

syntax = "proto3";

Here's a link to the protobuf definition

protobuf.dev

Let's first start with the services

service EmployeeService {
  rpc Create(EmployeeInp) returns (Employee);
  rpc List(Empty) returns (Employees);
  rpc Retrieve(EmployeeID) returns (Employee);
  rpc Delete(EmployeeID) returns (Empty);
}

The service handles creating, listing, retrieving, updating, and deleting employee records.

Handles everything from adding a new hire to giving them the boot, with input as EmployeeInp and output as Employee—because even computers need paperwork.

Now let's define the messages

syntax = "proto3";

message Empty{}

message Employee {
  uint32 id = 1;
  string name = 2;
  uint32 dept_id = 3;
  string designation = 4;
}

Create and retieve/update will need seperate messages

syntax = "proto3";

message Empty{}

message Employee {
  uint32 id = 1;
  string name = 2;
  uint32 dept_id = 3;
  string designation = 4;
}

message EmployeeInp {
  string name = 1;
  uint32 dept_id = 2;
  string designation = 3;
}

message EmployeeID {
  uint32 id = 1;
}

Finally, the list message

syntax = "proto3";

message Empty{}

message Employee {
  uint32 id = 1;
  string name = 2;
  uint32 dept_id = 3;
  string designation = 4;
}

message EmployeeInp {
  string name = 1;
  uint32 dept_id = 2;
  string designation = 3;
}

message EmployeeID {
  uint32 id = 1;
}
 
message Employees {
  repeated Employee employees = 1;
}

What next?

This is how it works

When working with protoc, think of it as the translator who turns your .proto files into code that your project can actually speak to.

protoc --<language>_out=<output_directory> <file>.proto does the trick, generating all the gRPC wrapper code you’ll need.

Here's a link to the docs

grpc.io/docs/languages/python/

So what's quickgRPC ?

It's a simple tool that makes this process much nicer, where you just give it your proto and it spits out boilerplate server and client skeletal code

Another thing it does is put the protoc generated code in site packages so that you don't have to ship it along with your server code.

Note: This is a simple tool I wrote a few years ago. Suggestions/improvements welcome as PRs!

ashupednekar.github.io/quickgrpc/Getting%20Started/

Demo time 😄

So far...

We wrote out proto file, with rpc's and their corresponding mmessages

Used the create_grpc_service command to generate the boilerplate code we'll need

Looked at proto file validations passed through from protoc

Few considerations

The quickgrpc library currently makes a few assumptions

• proto file name same as the service name, with the latter being in Title case
• single service in proto file

These things can change, if there's enough interest... maybe a go/rust rewrite/language support

Let's write the rpc logic, starting with Create

Summary

We're using a python dictionary as our db

Adding empoyees to the db as they come in

Note the pb2 snippets, this is where we interact with code generated by protoc

Listing these employees

Summary

Since our 'DB' is just a python dict, we need to convert that to the pb2 compatible object, that's what this is

Note: none of these are best practices for business logic, just an illustration

Let's quickly right retrieve and delete

Now let's update the tests

Voila

What happens under the hood

More stuff gRPC can do

What we've covered so far is only unary, which is the most analogous to REST apis

gRPC also supports

• client side streaming: the client can keep sending data and the server will respond when it's ready... e.g. KYC
• server side, i.e. SSE: client sends one request and the server keeps yielding results, e.g. progress bars
• bi-directional

Summary

As you saw, there were a few bugs in our service

• the employee_inp was being passed an id in the unittest. We changed the service to use dict length plus one instead
• the list extend call was not being made on .employees which is the repeated message as per our protobuf.
• We can also do client, server side or two way byte streaming by chaning proto and making our tests/server functions async

note: gRPC is not a silver bullet, it's meant for service to service communication alongside pubsub, most browsers don't support it directly

Than:q

That was it folks… questions?

come hang with me over at…

👋