gRPC is a powerful, high-performance Remote Procedure Call (RPC) framework that, despite being less commonly used than REST, offers significant advantages in certain scenarios.
In addition it's language agnostic and can run in any environment, making it an ideal choice for server-to-server communication.
I will not delve into in whole explenation of it but here is a general link of gRPC. I'll provide a hands on turtorial
Lets image our Go is client but is a server asfor frontend app React, Svelte etc.
func getFirstArg() (string, error) { if len(os.Args)As an example React frontend uploads a file, Go process it but we need answers from excel we will use GPT API. While it can be done with Go, Python on the otherhand has more packages that can ease our lives like langchan_openai, pandas for excel and so forth.
Lets start with instalation of gRPC preferably in your virtualenv .venv
$ go install google.golang.org/protobuf/cmd/protoc-gen-go@latest $ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest $ export PATH="$PATH:$(go env GOPATH)/bin"Next up you should install protocol buffer in your OS can follow it here.
Let's create a proto dir where you will store your protocol buffer file I will name it as excel.proto and paste this:syntax = "proto3"; option go_package = "client-gRPC/proto"; service ExcelService { rpc UploadFile(FileRequest) returns (FileResponse); } message FileRequest { string file_name = 1; bytes file_content = 2; } message FileResponse { bytes file_content = 1; }This gRPC service, ExcelService, allows clients to upload a file by sending its name and content. The server responds with the same file content.
For Go its essential to pass in go_package in Python the line is not needed.
vscode-proto3 is a good extension to download if you use VSCode.
After all of this you can generate your proto files I preffer it in same level as prot dir, for that run this command:
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative proto/excel.proto
If succesfull two files should be generated, optionally if there would be a lot of adjustments add a Makefile and define it as proto upper command.
import ( .... "google.golang.org/grpc" pb "client-gRPC/proto" "github.com/xuri/excelize/v2" ) func main() { .... conn, err := grpc.Dial("localhost:50051", grpc.WithInsecure()) if err != nil { log.Fatalf("Failed to connect to gRPC server: %v", err) } defer conn.Close() client := pb.NewExcelServiceClient(conn) req := &pb.FileRequest{ FileName: filePath, FileContent: fileData, } res, err := client.UploadFile(context.Background(), req) if err != nil { log.Fatalf("Failed to upload file: %v", err) } outputFile := "output.xlsx" err = saveBytesAsExcel(outputFile, res.FileContent) if err != nil { log.Fatalf("Failed to save bytes as Excel file: %v", err) } fmt.Printf("Excel file saved as: %s\n", outputFile) } func saveBytesAsExcel(filePath string, fileContent []byte) error { f, err := excelize.OpenReader(bytes.NewReader(fileContent)) if err != nil { return fmt.Errorf("failed to open Excel file: %v", err) } if err := f.SaveAs(filePath); err != nil { return fmt.Errorf("failed to save Excel file: %v", err) } return nil }We make a connection to listen to 50051 that will be our Python server, &pb.FileRequest was generated prior by using proto command and now we are importing the methods. If you run you will recive ? due to Python server not established yet.
Failed to upload file: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:50051: connect: connection refused"Python gRPC server
As python will act as a server the approach will be slightly different but in essense same proto file appart from package field is not reqired. Lets start by creating a base main.py without the gRPC just to give a glance of how GPT will populate the questions in excel.
import os import openai import pandas as pd from dotenv import load_dotenv def get_answer_from_gpt(apikey: str, question: str): openai.api_key = apikey response = openai.ChatCompletion.create( model="gpt-4", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": question} ] ) return response['choices'][0]['message']['content'].strip() def answer_questions_df(df: pd.DataFrame, apikey: str): answers = [] for question in df.iloc[:, 0]: answer = get_answer_from_gpt(apikey, question) answers.append(answer) return answers if __name__ == "__main__": load_dotenv() openai_api_key = os.getenv("OPENAI_API_KEY", "OpenAI API key hasn't been set.") df = pd.read_excel('Book1.xlsx') df['Answer'] = answer_questions_df(df, openai_api_keyIts a simple script that will answer questions that Go will send us but the LOC is less due to dedicated openai library that makes it easier.
We start by as well adding proto dir with same file as above the option section can be removed as disccused. Install gRPC in your virtualenv preferably and follow here the instalation for proto generation I ran"
python3 -m grpc_tools.protoc --proto_path=proto --python_out=proto --grpc_python_out=proto proto/excel.protoTo be in same lvl as my proto directory remember to add __init.py!
Ones the files have been generated lets continue on.
import io import grpc from proto import excel_pb2_grpc as excel_grpc from proto import excel_pb2 class ExcelService(excel_grpc.ExcelServiceServicer): def UploadFile(self, request, context): try: # Convert bytes to a file-like object file_like_object = io.BytesIO(request.file_content) # Load the workbook from the file-like object workbook = openpyxl.load_workbook(file_like_object) # Access the first sheet (or use appropriate logic to get the sheet you need) sheet = workbook.active # Convert the sheet to a DataFrame data = sheet.values columns = next(data) # Get the header row df = pd.DataFrame(data, columns=columns) print("Loaded DataFrame:") print(df.head()) # Ensure that the DataFrame is not empty and has questions if df.empty or df.shape[1]We define server and add ExcelService class what containes the methods generated by proto file. Because we recive file by bytes have to use io byte reader and commence further processing of the file and population the second column.
response = excel_pb2.FileResponse(file_content=output.read())At the end we are returning ☝️ for our Go client to recive.
To be able to find proto files in python however you should define an export path
export PYTHONPATH=$PYTHONPATH:mnt/c/own_dev/gRPC/server/proto
Running Client and Server
If all is good you can run #First comes server python3 -m main #Then client go run client.go Book1.xlsxAnd you should get the updated .xlsx file in Go client side.
Conclusion
In this article we explored the fundamentals of setting up gRPC communication between Python server and Go client. By leveraging gRPC, we established a seamless way to send an Excel file from a Go application to a Python server, process the file using OpenAI's GPT API, and return the modified file back to the Go client.
Disclaimer: All resources provided are partly from the Internet. If there is any infringement of your copyright or other rights and interests, please explain the detailed reasons and provide proof of copyright or rights and interests and then send it to the email: [email protected] We will handle it for you as soon as possible.
Copyright© 2022 湘ICP备2022001581号-3