Build RESTful API mircoservice with Go

Note you can find the codebase for the blogpost in this Github repo.

We can’t skip the conversation of microservices when we talk about service-oriented architecture. Growth companies face the challenge of scaling technical applications to meet business needs. The trend of microservices is also promoted along with some new technology that allow developers to build API based service faster.

Recently I have been researching Golang and how to use it to build microservices. This blogpost documents my findings.

Objective

I lay out certain criteria to evaluate whether Golang is a suitable tool for buidling microservices, I deem these criteria as important guide in building a microservices, which can be further applied in evaluate other language and frameworks.

Experiment

To start the experiment, I decided to build an instruction service using Gin as the Golang webserver. The instruction service is simple service that consumes and provides instructions based on RESTful API. Additionally, I choose Gin as the framework because the community behind the framework is active, and comparing to other framework such as Martini, the set up is light weight, the syntax is easy to understand.

Set up webserver

Inside main.go, we set up the router for api/v1/.

package main

import (
  "github.com/gin-gonic/gin"
  "fmt"
)

func SetupRouter() *gin.Engine {
  router := gin.Default()

  v1 := router.Group("api/v1") 
  {
    v1.GET("/instructions", GetInstructions)
  }

  return router
}

func GetInstructions(c *gin.Context) {
  c.JSON(200, gin.H{"ok": "Welcome to Chicago!"})
  // curl -i http://localhost:8080/api/v1/Instructions
}

func main() {
  router := SetupRouter()
  router.Run(":8080")
}

After you start the server using go main.go, This file should serve response of “ok”: “Welcome to Chicago!” when you hit http://localhost:8080/api/v1/Instructions.

In preparation of adding more logic, we can move business logic into app folder. We can create a app folder and add app.go:

package app

import (
  "strconv"
  "github.com/gin-gonic/gin"
)

func GetInstructions(c *gin.Context) {
  c.JSON(200, gin.H{"ok": "GET api/v1/instructions"})
}

func GetInstruction(c *gin.Context) {
  c.JSON(200, gin.H{"ok": "GET api/v1/instructions/1"})
}

func PostInstruction(c *gin.Context) {
  c.JSON(200, gin.H{"ok": "POST api/v1/instructions"})

}

func UpdateInstruction(c *gin.Context) {
  c.JSON(200, gin.H{"ok": "PUT api/v1/instructions/1"})

}

func DeleteInstruction(c *gin.Context) {
  c.JSON(200, gin.H{"ok": "DELETE api/v1/instructions/1"})
}

Then inside main.go, we can point the request handler to app:

package main

import (
  "github.com/gin-gonic/gin"
  "./app"
)

func SetupRouter() *gin.Engine {
  router := gin.Default()

  v1 := router.Group("api/v1") 
  {
    v1.GET("/instructions", app.GetInstructions)
    v1.GET("/instructions/:id", app.GetInstruction)
    v1.POST("/instructions", app.PostInstruction)
    v1.PUT("/instructions/:id", app.UpdateInstruction)
    v1.DELETE("/instructions/:id", app.DeleteInstruction)
  }

  return router
}

func main() {
  router := SetupRouter()
  router.Run(":8080")
}

Struct and database

We need to restore Instruction object into a database, to do that we need to use Struct type, which can capture a collection of fields with strict typing. We have only three fields in the database table: an integer Id as the primary key, EventStatus as a string, and EventName as a string.


type Instruction struct {
 Id int64
 EventStatus string
 EventName string
}

Next, we need to connect the webservice with database, we choose mysql as the database and mysql go-sql-driver as the databaes driver. To do that, we create a db.go inside app folder because it belongs to the same package app.

package app

import (
  "database/sql"
  _ "github.com/go-sql-driver/mysql"
  "gopkg.in/gorp.v1"
  "log"
)

func initDb() *gorp.DbMap {
  db, err := sql.Open("mysql", "root:password@/instructions")
  checkErr(err, "sql.Open failed")

  dbmap := &gorp.DbMap{Db: db, Dialect: gorp.MySQLDialect{"InnoDB", "UTF8"}}
  dbmap.AddTableWithName(Instruction{}, "Instruction").SetKeys(true, "Id")

  err = dbmap.CreateTablesIfNotExists()
  checkErr(err, "Create table failed")
  return dbmap
}

func checkErr(err error, msg string) {
  if err != nil {
    log.Fatalln(msg, err)
  }
}

Inside app.go, we can declare dabatase level typing back to our original struct, and we can initialize webdriver client by calling initDb()

type Instruction struct {
 Id int64 `db:"id" json:"id"`
 EventStatus string `db:"event_status" json:"event_status"`
 EventName string `db:"event_name" json:"event_name"`
}

var dbmap = initDb()

Now that we have router setup and database connection established, let’s backfill our RESTful APIs with real actions:

func GetInstruction(c *gin.Context) {
  id := c.Params.ByName("id")
  var instruction Instruction
  
  err := dbmap.SelectOne(&instruction, "SELECT * FROM instruction WHERE id=?", id)
  if err == nil {
    instruction_id, _ := strconv.ParseInt(id, 0, 64)
  
    content := &Instruction{
      Id: instruction_id,
      EventStatus: instruction.EventStatus,
      EventName: instruction.EventName,
    }
 
    c.JSON(200, content)
  } else {
    c.JSON(404, gin.H{"error": "instruction not found"})
  }
  // curl -i http://localhost:8080/api/v1/Instructions/1
}

func PostInstruction(c *gin.Context) {
  var instruction Instruction
  c.Bind(&instruction)

  if instruction.EventStatus != "" && instruction.EventName != "" {
    if insert, _ := dbmap.Exec(`INSERT INTO instruction (event_status, event_name) VALUES (?, ?)`, instruction.EventStatus, instruction.EventName); insert != nil {
      instruction_id, err := insert.LastInsertId()
      if err == nil {
        content := &Instruction{
          Id: instruction_id,
          EventStatus: instruction.EventStatus,
          EventName: instruction.EventName,
        }
        c.JSON(201, content)
      } else {
        checkErr(err, "Insert failed")
      }
    }
  } else {
    c.JSON(422, gin.H{"error": "fields are empty"})
  }
  // curl -i -X POST -H "Content-Type: application/json" -d "{ \"event_status\": \"83\", \"event_name\": \"100\" }" http://localhost:8080/api/v1/instructions
}

func UpdateInstruction(c *gin.Context) {
  // ... more code...
  // curl -i -X PUT -H "Content-Type: application/json" -d "{ \"event_status\": \"83\", \"event_name\": \"100\" }" http://localhost:8080/api/v1/instructions/1
}

func DeleteInstruction(c *gin.Context) {
  // ... more code...
  // curl -i -X DELETE http://localhost:8080/api/v1/instructions/1
}

You can find complete code here

Testing

Golang provides a package testing to write tests. Two conventions must be followed to write tests, which is noted in testing API documentation:

Package testing provides support for automated testing of Go packages. It is intended to be used in concert with the “go test” command, which automates execution of any function of the form func TestXxx(*testing.T)

Gin also provides testing mode for writing API level tests along with Golang httptest, we then can start writing integration tests under main_test.og following testing convention. For example, we can test that our POST endpoint functions as expected:

package main

import (
  "bytes"
  "github.com/gin-gonic/gin"
  "net/http"
  "net/http/httptest"
  "testing"
)

func TestPostInstruction(t *testing.T) {
  gin.SetMode(gin.TestMode)
  testRouter := SetupRouter()

  body := bytes.NewBuffer([]byte("{\"event_status\": \"83\", \"event_name\": \"100\"}"))

  req, err := http.NewRequest("POST", "/api/v1/instructions", body)
  req.Header.Set("Content-Type", "application/json")
  if err != nil {
    t.Errorf("Post hearteat failed with error %d.", err)
  }

  resp := httptest.NewRecorder()
  testRouter.ServeHTTP(resp, req)

  if resp.Code != 201 {
    t.Errorf("/api/v1/instructions failed with error code %d.", resp.Code)
  }
}

Comparing to other testing framework such as Rspec in Rails, testing in go seems less elgant, however, the code seems less magical comparing to other dynmaic laungauge like Ruby.

You can find the complete testing code here.

Deployment

Docker image for golang is well supported and there are many established Dockerfile for Gin framework, one thing we need to watch out is the dependency based on package we introduced. Unfortunately, it does not seem like there is a centralized place inside my Golang app to manage package dependency. Below is my Dockerfile:

FROM        golang:1.8
MAINTAINER  Marshall Shen <marshall@test.com>

ENV     PORT  8080
   
# Setting up working directory
WORKDIR     /go/src/gin-container
ADD         . /go/src/gin-container

RUN     go get github.com/tools/godep
RUN     go get github.com/gin-gonic/gin
RUN     go get gopkg.in/gorp.v1
RUN     go get github.com/go-sql-driver/mysql
RUN     go install github.com/tools/godep
RUN     go install github.com/gin-gonic/gin

# Restore godep dependencies
#RUN godep restore

EXPOSE 8080
ENTRYPOINT  ["/usr/local/bin/go"]
CMD     ["run", "main.go"]

Evaluation

Based on the research, Golang and Gin seems like a great fit for building a microservice in handling a very specific task, such as serving RESTful API. The framework is API-ready, provides tools for testing and is easily deployable with Docker.

Conceptually, I am confident that this framework is great at handling tasks such asweb scrapping, ETL pipeline, or media transcoding. However, I am not confident that the framework is designed for building applications that encompass complicated business logic, such as a client-facing web application. There are several aspects of framework I’d like to investigate more in the future:

rss facebook twitter github youtube mail spotify instagram linkedin google pinterest medium vimeo