Microservices in Python | Python Microservices with gRPC
In this tutorial, we will learn about microservices, why microservices are important and how to implement them using Python programming language. We will also discuss the gRPC and use it with Python.
In the current scenario, the technologies are more dedicated to "microservices architecture" and leaving behind old "monolithic architecture". We believe that "smaller things are much easier to handle", so we have microservices that suitably can be handled. We need to interact with different microservices.
Here we will learn how to get up and running with Python microservices using gRPC, one of the most popular frameworks.
Let's briefly introduce Microservices and how to use them in our project.
What are Microservices?
Microservices are a technique for organizing complex software systems. It is similar to breaking a complex problem into a smaller one and implementing it. Same as microservices, microservices don't put all the code, not into one app, developer breaks the code into the microservices that are deployed independently and communicate with each other.
Implementing a microservices framework is essential. When we plan to develop a framework to provide critical applications, we must ensure its robust and developer-friendly behavior.
Why Microservices are Important?
Suppose a developer is working for the popular e-commerce website that sells multiple products online. The several hundred workers are working in the company. Each developer is assigned to write code for the back-end feature or some product such as adding item to cart, generating recommendations, handling payment transaction, or dealing with warehouse history.
Now imagine, how hard to put all the code in one giant application? How hard would that be to understand? How long would it take to test? It definitely quite challenging especially as the business tries to grow fast.
It would be far better to have code corresponding to modular product features be, well modular? A cart microservice manage cart. An inventory microservice manages inventory.
Let's understand the reason behind to separate Python code into the microservices.
Modularity allows making the changes with the least resistance. Suppose the CEO wants to add a new feature, the buy-two-books-get-one-free feature. And you are the part of the team that's asked to launch it as quickly as possible. If we put all the code into a single application, we face some critical problems while implementing this feature.
Here you suppose the solution to the team - you can add some code to the cart logic to check if there are more than two books in the cart.
The product manager also tells to track this campaign's impact on the sales of the books. It is also simple. You can simply subtract the cheapest book from the cart total. Along with the buy-two-books-get-one-free feature in the code, you update the line in the checkout column that updates a new column on the transaction database to indicate the sale was part of the promotion buy_two_get_one_free_promo = true. Done.
Again another requirement comes; it should be valid for one use per customer. If you use this feature, you need to hide this banner on the home page.
After some years, the transaction database has grown too large and requires replacing it with a new shared database. That might be a cause of data loss. All those references need to be changed.
That's why putting all the code into a single application can be too dangerous in the long run.
The best solution is to create the transaction microservices, and we can also scale it in the future. Another part of the code can be communicated with the transaction through an abstract API that hides the implementation details.
Flexibility is the most important which comes with microservices. The developer can write the code in multiple languages. We can also expand the microservices independently. We will create sample microservices in the upcoming section.
If we put all the code into a single application, we also need to deploy it at once. This could be a big threat. When we try to make a change in the code, it means it can down the entire site.
Microservices provide the ownership of their code. When a single codebase is shared with multiple employees where the employees come and go. There is no clear vision for the architecture of the code. Microservice helps give a clear vision for the code, and that code will remain clean and organized. It also clarifies who's responsible for adding features to code or making changes when something goes wrong.
Example of Microservices
We will define some microservices for online Books for the website. We will create an API and write the Python code that implements them as microservices.
To make things short and easy, we will define only two microservices -
The user will interact with the Marketplace microservices via their browser, and the Marketplace microservice will interact with the Recommendation microservices.
The Recommendation API should have a few features -
The response will be a list of books. Each book will have the following data.
We have taken some limited features of the API to make it simpler. Now we can define this API more formally using the protocol buffers.
We create a proto file that has our API. Protobuf buffers were developed at Google, and to get the idea about the protobuf, you can visit our how to implement protobuf in Python. It might seem a bit confusing at first, but let's break the file by line.
It is a brief idea about the microservices using the protobuf. Now we will learn about the popular microservice technique gRPC.
What is gRPC?
According to the official documentation -
"gRPC or Google Remote Procedure Call is a modern open-source high-performance RPC framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking and authentication."
RPC stands for remote procedure calls which are quite same as the normal function in Python. They are messages that server sends to the remote system to get the task (or routine) completed.
Google's RPC provides the facility of a smooth and efficient communication between the services. It can be utilized in the different ways -
The gRPC provides many advantages over the other microservices. Let's understand some advantages.
Protocol buffers are same as the XML which are provides an efficient and automated mechanism for serializing structured data. They allow to define the structure of the data to be transmitted. Google claims they are better than the XML. Its features are -
Protobuf are defined in .proto files. It is easy to define them.
Types of gRPC implementation
There are four types of gRPC implementation. Let's understand each implementation -
1. Unary RPCs - It is a simplest gRPC which executes as the normal function call. It sends single request declared in the .proto file and get the single response from the server.
2. Server streaming RPCs - It returns the streams of message sequence to read corresponding to client message request. The client reads from that stream of messages until there are no messages.
3. Client Streaming RPCs - The client writes a message sequence using a write stream and sends the same to the server. The server reads all the message and wait for server to read all the messages and return a response.
4. Bidirectional Streaming RCPs - Both gRPC client and the gRPC server use a read-write stream to send message sequence. The gRPC client and gRPC server can write and read in any order they like.
Implementation of gRPC in Python
However, gRPC supports many programming languages such as Golang, C++, and Java. In this tutorial, we will implement using Python.
Follow the below steps to proceed further.
We will use the Unary gRPC, where we need to define three files.
Let's create the Unary service.
The above code contains the collection of services. Here we have implemented a single service GetServerResponse(). This service takes an input of type Message and returns a MessageResponse. Once we have done the .proto file, we need to generate the stubs using the below command.
When we press enter, it will generate the two files.
In the above file, the GetServerResponse() method takes the 'Message' from the client and returns a 'MessageResponse' as defined in the proto file. We have called the server() function in the main() to make sure that the server listens to the request. Now, we run the above server file.
Implementing the Client
Below is the implementation of the client.
In the above code, the __init__func has the stub variable using the and we ` self.stub = pb2_grpc.UnaryStub(self.channel)' and we have a get_url() function which calls to server using the above initialized stub.
Now we have completed the Unary gRPC service. Run the service the check the output.
message: "Hello Server Are you listening?" message: "Hello I am up and running. Received 'Hello Server you there?' message from you" received: true
We can also define the bidirectional services as below.
In the above code, we have declared a Birectional named service. It contains a collection of services. Currently, we have implemented only one service GetServerResponse(). This service accepts an input type Message and returns a Message. We follow the same process to generate the stubs. To generate the stub, we need to run the below command.
The above command will generate the two files bidirectional_pb2.py and bidirectional_pb2_grpc.py. These files will be used to generate gRPC server and client. You can implement this service by taking the above example as reference and you can also refer its official documentation.
Advantages and Disadvantages of Microservices
We have discussed all the important aspect of the microservices, what microservices are, also discussed the importance of using microservices in software architecture of our project. Below are the short descriptions of pro and cons of microservices.
Microservices are the most appropriate way to manage a complex system. They provide a natural way to organize code. Once we get familiar with the gRPC and how to implement microservices in Python effectively, we can become one step ahead in the development field.
In this tutorial, we have discussed microservices, how to implement Python microservices effectively with gRPC, the types of microservices, and the advantages and disadvantages of microservices. We have also discussed gRPC, an emerging RPC framework that makes communication between microservices smooth and effective.