Welcome back, folks! Today, we will cover the end-to-end tests in an intriguing blog post. If you've never written these kinds of tests or if you strive to improve them, keep reading as I'll walk you through this exciting journey. By the end of the article, you'll know how to empower the usage of the testcontainers-go package to let your test suite shine.

The Premise ?

Before moving ahead, let's set the boundaries for this blog post since we will cover several concepts, tools, and techniques.

The Survival List ?️

Since we'll touch on several topics throughout the rest of the blog post, I feel it's a good idea to put them together here.

The tools I present throughout this blog post are a mix of tools I know well and some I used for the first time. Try not to use these tools without thinking, but evaluate them based on your scenario.

We're going to rely on:

• The Go programming language
• Docker
• The testcontainers-go package with the compose module
• The ginkgo testing framework and the gomega assertion package

To avoid bloating the reading, I won't cover every aspect and facet of the topics presented here. I will put the relevant documentation URLs where needed.

The Scenario ?

Let's assume we need to write end-to-end tests on a project we don't own. In my case, I want to write end-to-end tests on a project written with the Java programming language. Since I didn't know how to code in Java, my testing option was only end-to-end tests. The service I had to test was a set of REST APIs. The solution has been obvious: exercise the endpoints by issuing HTTP requests.

It allows testing the exposed features like a black box. We only have to deal with the public surface: what we send to the server and what we get back from it. Nothing more, nothing less.

We care about the api/accounts endpoint that lists the bank accounts in our database (a MySQL instance). We're going to issue these two requests:

Now, you should have a clearer idea of our goal. So, let's jump into the test code.

Let's Have Fun ?

In this section, I present all the relevant code we need to write for the dreaded end-to-end tests.

The docker-compose.yml file

Since we don't bother with the source code, the starting point is the docker-compose.yml file. The relevant code is:

services:
  mysqldb:
    image: "mysql:8.0"
    container_name: mysqldb
    restart: always
    ports:
      - 3307:3306
    networks:
      - springapimysql-net
    environment:
      MYSQL_DATABASE: transfers_db
      MYSQL_USER: bulk_user
      MYSQL_PASSWORD: root
      MYSQL_ROOT_PASSWORD: root
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s

  api_service:
    build: .
    container_name: api_service
    restart: always
    ports:
      - 8080:8080
    networks:
      - springapimysql-net
    environment:
      - spring.datasource.url=jdbc:mysql://mysqldb:3306/transfers_db
      - spring.datasource.username=bulk_user
      - spring.datasource.password=root
    depends_on:
      mysqldb:
        condition: service_healthy
    volumes:
      - .m2:/root/.m2

networks:
  springapimysql-net:

The file content is pretty straightforward. We can summarize the things defined in the following list:

• The mysqldb service doesn't deserve any further explanations
• The api_service service is the system we're testing
• The springapimysql-net network hosts the two services defined above

For further Docker Compose reference, you may have a look here. Now, let's see the end-to-end test code.

The ginkgo Testing Framework

The ginkgo testing framework helps us in building the test suite. It's entirely written in Go. Furthermore, it provides a CLI utility to set up and run the tests. Since we will use it later, let's download it from here. You can download it in two ways:

1. By using the go install command (if you've installed Go on your system)
2. By downloading the compiled binary (useful if you don't have Go installed on your system)

To check whether you have a working utility on your machine, you can run the command ginkgo version (at the time of writing, I have the version 2.20.2).

Please note that the ginkgo command is not mandatory to run the tests. You can still run the tests without this utility by sticking to the go test command.

However, I strongly suggest downloading it since we will use it to generate some boilerplate code.

Lay the Foundation with Ginkgo

Located in the root directory, let's create a folder called end2end to host our tests. Within that folder, initialize a Go module by issuing the command go mod init path/to/your/module.

Now, it's time to run the command ginkgo bootstrap. It should generate a new file called end2end_suite_test.go. This file triggers the test suite we'll define in a bit.

This approach is similar to the one with the testify/suite package. It enforces the code modularity and robustness since the definition and running phases are separated.

Now, let's add the tests to our suite. To generate the file where our tests will live, run another ginkgo command: ginkgo generate accounts. This time, the file accounts_test.go pops out. For now, let's leave it as is and switch to the terminal. We fix the missing packages by running the Go command go mod tidy to download the missing dependencies locally on our machine.

The end2end_suite_test.go file

Let's start with the entry point of the test suite. The content of the file looks neat:

//go:build integration

package end2end

import (
 "testing"

 . "github.com/onsi/ginkgo/v2"
 . "github.com/onsi/gomega"
)

func TestEnd2End(t *testing.T) {
 RegisterFailHandler(Fail)
 RunSpecs(t, "End2End Suite")
}

The only unusual thing might be the dot-import within the import section. You can read more about it in the documentation here.

Whoop! A wild testcontainers appears ?

At some points, we need some magic to get to the next testing level. It happened to be testcontainers-go. For the sake of this demo, we use the compose module (for further reference, please refer to here).

This tool can run the compose file we saw earlier and execute the end-to-end tests against the running containers.

This is an extract of the testcontainers-go capabilities. If you want to learn more, please refer to the doc or reach out. I'll be happy to walk you through its stunning features.

This package allows running the end-to-end suite with a single command. It's a more consistent and atomic way to run these tests. It allows me to:

1. Start the containers before the suite
2. Run the tests relying on these containers
3. Teardown of containers after the suite and cleanup of the resources used

Having the code written this way can help you avoid the hassle of dealing with docker cli commands and makefiles.

The accounts_test.go file

Now, let's look at the code where our tests live.

//go:build integration

package end2end

import (
 "context"
 "net/http"
 "os"

 . "github.com/onsi/ginkgo/v2"
 . "github.com/onsi/gomega"
 tc "github.com/testcontainers/testcontainers-go/modules/compose"
 "github.com/testcontainers/testcontainers-go/wait"
)

var _ = Describe("accounts", Ordered, func() {
 BeforeAll(func() {
 os.Setenv("TESTCONTAINERS_RYUK_DISABLED", "true")
 composeReq, err := tc.NewDockerComposeWith(tc.WithStackFiles("../docker-compose.yml"))
  Expect(err).Should(BeNil())
  DeferCleanup(func() {
   Expect(composeReq.Down(context.Background(), tc.RemoveOrphans(true), tc.RemoveImagesLocal)).Should(BeNil())
  })
 ctx, cancel := context.WithCancel(context.Background())
  DeferCleanup(cancel)
 composeErr := composeReq.
   WaitForService("api_service", wait.ForListeningPort("8080/tcp")).
   Up(ctx, tc.Wait(true))
  Expect(composeErr).Should(BeNil())
 })

 Describe("retrieving accounts", func() {
  Context("HTTP request is valid", func() {
   It("return accounts", func() {
 client := http.Client{}
 r, err := http.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/accounts?iban=IT10474608000005006107XXXXX", nil)
 res, err := client.Do(r)
    Expect(err).Should(BeNil())
    Expect(res).To(HaveHTTPStatus(http.StatusOK))
   })
  })

  Context("HTTP request is NOT valid", func() {
   It("err with invalid IBAN", func() {
 client := http.Client{}
 r, err := http.NewRequest(http.MethodGet, "http://127.0.0.1:8080/api/accounts?iban=abcd", nil)
    Expect(err).Should(BeNil())
 res, err := client.Do(r)
    Expect(err).Should(BeNil())
    Expect(res).To(HaveHTTPStatus(http.StatusBadRequest))
   })
  })
 })
})

At first glimpse, it might seem hard to digest. To keep things easier, let's break it down into smaller parts.

The Describe container node

The Describe container node is nothing but a wrapper to hold the relevant code for our suite. Everything must live within it. It's part of the scaffolded code: var _ = Describe("accounts", Ordered, func() {}. Within the {}, you should put all of the relevant code. To enforce the usage of setup nodes (like BeforeAll), we must define the Describe container as Ordered.

Do not worry if you forgot to add it since the Go compiler will complain.

Let's move on.

The BeforeAll setup node

This node allows us to extract the common setup logic. This code portion executes once and before the tests within the suite. Let's recap what's doing:

• Set the environment variable TESTCONTAINERS_RYUK_DISABLED to true. You can learn about the configuration here. If you're curious about Ryuk, you may want to look at this
• Create a *tc.DockerCompose variable based on the docker-compose.yml file we provided
• Defer the function invocation to terminate containers and cleanup of the resources
• Start the compose stack and wait for the container called api_service to be up and ready to listen on the 8080/tcp port

I simplified the test code since I don't want to make this blog post even longer ?.

Finally, the tests! ?

The test functions live within a Describe Container Node. You can find out how ginkgo handles the test specifications by referring to here. The Describe node allows you to group and organize tests based on their scope. You can nest this node inside other Describe ones.

The more you nest the Describe node, the more you narrow the test scope.

Then, we have the Context Container Node that qualifies the parent Describe. It qualifies the circumstances under which the tests are valid. Finally, we have the It section, the Spec Subject. It's the actual test we're performing and is the leaf level of the hierarchy tree. The test code is self-explanatory, so I'll jump to the section where we run the tests.

3, 2, 1... ?

Congrats ? We managed to get here. Now, we only miss the test-running operations. In the blink of an eye, we'll get our test execution report printed onto the terminal.

Let's switch to the terminal and run the command ginkgo --tags=integration -v. After a while, you'll see the output printed on the terminal.

Closing Notes ?

I know there are a lot of things condensed into this blog post. My goal has been to provide insights and approaches on how to write a good testing suite. You may want to adapt the presented tools, packages, and techniques to other kinds of tests or use cases.

Before leaving, I'd like to underline another beauty of the compose module of the testcontainers-go package.

If you stick to the configuration I provided, you're sure to use the latest Docker images, and you can avoid hours of troubleshooting due to outdated image usage. It's analogous to the command docker compose build --no-cache && docker compose up. You'll thank me ?

Thanks for the attention, folks! If you've got any questions, doubts, feedback, or comments, I'm available to listen and speak together. If you want me to cover some specific concepts, please reach me. Until the next time, take care and see you ?