# gRPC Communication Between Go and Python

By [Mozes721](https://paragraph.com/@mozes721) · 2024-08-22

---

gRPC is a powerful, high-performance Remote Procedure Call (RPC) framework that, despite being less commonly used than REST, offers significant advantages in certain scenarios.

In addition it’s language agnostic and can run in any environment, making it an ideal choice for server-to-server communication.

I will not delve into in whole explenation of it but [here](https://grpc.io/) is a general link of gRPC. I’ll provide a hands on turtorial

### Go gRPC client

Lets image our Go is client but is a server asfor frontend app React, Svelte etc.

    func getFirstArg() (string, error) {
        if len(os.Args) < 2 {
            return "", fmt.Errorf("expected 1 argument, but got none")
        }
        return os.Args[1], nil
    }
    
    func main() {
        filePath, err := getFirstArg()
        if err != nil {
            log.Fatalf("Failed to get file path from arguments: %v", err)
        }
    
        fileData, err := ioutil.ReadFile(filePath)
        if err != nil {
            log.Fatalf("Failed to read file: %v", err)
        }
    
     ...
    }
    

![](https://storage.googleapis.com/papyrus_images/73f2ebc61735c2792aed479c636b272a16bc0020cad6bee0a6acf7ba364ac8c8.png)

As an example React frontend uploads a file, Go process it but we need answers from excel we will use GPT API. While it can be done with Go, Python on the otherhand has more packages that can ease our lives like _langchan\_openai_, _pandas_ for excel and so forth.

* * *

Lets start with instalation of gRPC preferably in your virtualenv _.venv_

    $ go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
    $ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
    
    $ export PATH="$PATH:$(go env GOPATH)/bin"
    

Next up you should install protocol buffer in your OS can follow it [here](https://grpc.io/docs/protoc-installation/).

Let’s create a proto dir where you will store your protocol buffer file I will name it as _excel.proto_ and paste this:

    syntax = "proto3";
    
    option go_package = "client-gRPC/proto";
    
    service ExcelService {
        rpc UploadFile(FileRequest) returns (FileResponse);
    }
    
    message FileRequest {
        string file_name = 1;
        bytes file_content = 2;
    }
    
    message FileResponse {
        bytes file_content = 1;
    }
    

This gRPC service, `ExcelService`, allows clients to upload a file by sending its name and content. The server responds with the same file content.

For Go its essential to pass in go\_package in Python the line is not needed.

> **vscode-proto3** is a good extension to download if you use VSCode.

After all of this you can generate your proto files I preffer it in same level as prot dir, for that run this command:

    protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative proto/excel.proto
    

If succesfull two files should be generated, optionally if there would be a lot of adjustments add a Makefile and define it as proto + upper command.

    import (
        ....
    
        "google.golang.org/grpc"
        pb "client-gRPC/proto"
        "github.com/xuri/excelize/v2"
    )
    
    func main() {
        ....
    
        conn, err := grpc.Dial("localhost:50051", grpc.WithInsecure())
        if err != nil {
            log.Fatalf("Failed to connect to gRPC server: %v", err)
        }
        defer conn.Close()
    
        client := pb.NewExcelServiceClient(conn)
    
        req := &pb.FileRequest{
            FileName:    filePath,
            FileContent: fileData,
        }
    
        res, err := client.UploadFile(context.Background(), req)
        if err != nil {
            log.Fatalf("Failed to upload file: %v", err)
        }
    
        outputFile := "output.xlsx"
        err = saveBytesAsExcel(outputFile, res.FileContent)
        if err != nil {
            log.Fatalf("Failed to save bytes as Excel file: %v", err)
        }
    
        fmt.Printf("Excel file saved as: %s\n", outputFile)
    }
    
    func saveBytesAsExcel(filePath string, fileContent []byte) error {
        f, err := excelize.OpenReader(bytes.NewReader(fileContent))
        if err != nil {
            return fmt.Errorf("failed to open Excel file: %v", err)
        }
    
        if err := f.SaveAs(filePath); err != nil {
            return fmt.Errorf("failed to save Excel file: %v", err)
        }
        return nil
    }
    

We make a connection to listen to 50051 that will be our Python server, &pb.FileRequest was generated prior by using proto command and now we are importing the methods. If you run you will recive 👇 due to Python server not established yet.

    Failed to upload file: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:50051: connect: connection refused"
    

### Python gRPC server

As python will act as a server the approach will be slightly different but in essense same proto file appart from package field is not reqired. Lets start by creating a base _main.py_ without the gRPC just to give a glance of how GPT will populate the questions in excel.

    import os
    import openai
    import pandas as pd
    from dotenv import load_dotenv
    
    def get_answer_from_gpt(apikey: str, question: str):
        openai.api_key = apikey
        response = openai.ChatCompletion.create(
            model="gpt-4",
            messages=[
                {"role": "system", "content": "You are a helpful assistant."},
                {"role": "user", "content": question}
            ]
        )
        return response['choices'][0]['message']['content'].strip()
    
    def answer_questions_df(df: pd.DataFrame, apikey: str):
        answers = []
    
        for question in df.iloc[:, 0]: 
            answer = get_answer_from_gpt(apikey, question)
            answers.append(answer)
        return answers
    
    if __name__ == "__main__":
        load_dotenv()
    
        openai_api_key = os.getenv("OPENAI_API_KEY", "OpenAI API key hasn't been set.")
    
    
        df = pd.read_excel('Book1.xlsx')
        
        df['Answer'] = answer_questions_df(df, openai_api_key) 
    

Its a simple script that will answer questions that Go will send us but the LOC is less due to dedicated _openai_ library that makes it easier.

* * *

We start by as well adding proto dir with same file as above the option section can be removed as disccused. Install gRPC in your virtualenv preferably and follow [here](https://grpc.io/docs/languages/python/quickstart/) the instalation for proto generation I ran:

    python3 -m grpc_tools.protoc --proto_path=proto --python_out=proto --grpc_python_out=proto proto/excel.proto
    

To be in same lvl as my proto directory _remember to add \_init_\_.py!

Ones the files have been generated lets continue on.

    import io
    import grpc
    from proto import excel_pb2_grpc as excel_grpc
    from proto import excel_pb2
    
    
    
    class ExcelService(excel_grpc.ExcelServiceServicer):
        def UploadFile(self, request, context):
            try:
                # Convert bytes to a file-like object
                file_like_object = io.BytesIO(request.file_content)
    
                # Load the workbook from the file-like object
                workbook = openpyxl.load_workbook(file_like_object)
    
                # Access the first sheet (or use appropriate logic to get the sheet you need)
                sheet = workbook.active
    
                # Convert the sheet to a DataFrame
                data = sheet.values
                columns = next(data)  # Get the header row
                df = pd.DataFrame(data, columns=columns)
    
                print("Loaded DataFrame:")
                print(df.head())
    
                # Ensure that the DataFrame is not empty and has questions
                if df.empty or df.shape[1] < 1:
                    print("DataFrame is empty or does not have the expected columns.")
                    return excel_pb2.FileResponse(file_content=b'')
    
                # Get answers and add them to the DataFrame
                answers = answer_questions_df(df, openai_api_key)
                df['Answer'] = answers
    
                # Write the updated DataFrame back to a BytesIO object
                output = io.BytesIO()
                with pd.ExcelWriter(output, engine='openpyxl') as writer:
                    df.to_excel(writer, index=False, sheet_name='Sheet1')
    
                # Reset the buffer's position to the beginning
                output.seek(0)
    
                # Return the modified file content
                response = excel_pb2.FileResponse(file_content=output.read())
                return response
            except Exception as e:
                print(f"Error processing file: {e}")
                return excel_pb2.FileResponse(file_content=b'')
        
    def serve():
        server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
        excel_grpc.add_ExcelServiceServicer_to_server(ExcelService(), server)
        server.add_insecure_port('[::]:50051')
        server.start()
        print("Server running on port 50051.")
        server.wait_for_termination()
    
    
    if __name__ == "__main__":
        load_dotenv()
    
        openai_api_key = os.getenv("OPENAI_API_KEY", "OpenAI API key hasn't been set.")
    
        serve()
    

We define server and add ExcelService class what containes the methods generated by proto file. Because we recive file by bytes have to use io byte reader and commence further processing of the file and population the second column.

    response = excel_pb2.FileResponse(file_content=output.read())
    

At the end we are returning ☝️ for our Go client to recive.

> To be able to find proto files in python however you should define an export path**_export PYTHONPATH=$PYTHONPATH:mnt/c/own\_dev/gRPC/server/proto_**

#### Running Client and Server

If all is good you can run

    #First comes server
    
    python3 -m main
    
    #Then client
    
    go run client.go Book1.xlsx
    

And you should get the updated .xlsx file in Go client side.

#### Conclusion

In this article we explored the fundamentals of setting up gRPC communication between Python server and Go client. By leveraging gRPC, we established a seamless way to send an Excel file from a Go application to a Python server, process the file using OpenAI’s GPT API, and return the modified file back to the Go client.

---

*Originally published on [Mozes721](https://paragraph.com/@mozes721/grpc-communication-between-go-and-python)*