The Unit Test Strategy In Vald

  • Software structural quality

What is the unit test?

According to Wikipedia, unit testing is a software testing method to test individual units (most likely functions/methods) of your source code to ensure it meets the requirements and behaves as intended. Unit tests are typically automated, which means they can be executed automatically before deploying your application.

func add(i, j int) int {
return i + j
}
func TestAdd(t *testing.T) {
got := add(1,2)
if got != 3 {
t.Errorf("add(1,2) = %d; want 3", got)
}

}

What is Vald?

Vald is an open-source project working on a highly scalable distributed ANN dense vector search engine written in Go.

Why do we need unit tests in Vald?

In Vald, we are working on many functions and the implementation is complex, it may have bugs hidden inside the implementation. To provide a better user experience, we try our best to keep Vald bug-free, and we’re working on bug fixes in high priority.

Unit test strategy

In this chapter, we will talk about the strategy of how do we implement unit tests in Vald. With these strategies, we believe that we will improve the unit test quality to help us to find more potential bugs and defects in our implementation to provide better quality software to our users.

Know what you are testing

Before thinking and designing the test case, we need to know and understand what we are going to test. We need to know the purpose of the target function or the class before we work on it, it helps us to think about what is required for the testing.

Test behaviors and results, not implementation

We should focus on testing the behaviors and results when we start thinking about the unit test case.

Test one thing at a time

Each test should have a clear, concise, and single objective. For each test case, it should only test for a single objective or use case, we should not mix multiple things into one unit test to avoid complexity. Also, it helps us to specify what is failed so we can easily fix the bug when it occurred.

func TestGetUserData(t *testing.T) {
t.Run("success to get user data", func(t *testing.T) {
// test implementation
})
t.Run("failed to connect to database (unauthorized)", func(t *testing.T) {
// test implementation
})
t.Run("failed to connect to database (host not found)", func(t *testing.T) {
// test implementation
})
t.Run("failed to get non-exists user data", func(t *testing.T) {
// test implementation
})
// other test cases
}

Make tests readable and understandable

Even unit test code is not production code, we should still maintain the test code as better as we should.

  • Test understandability
func TestInsertUserData(t *testing.T) {
t.Run("test insert user data", func(t *testing.T) { // define a understandable test case name
// define production-like data as test data
u := User {
Name: "Peter Parker",
Age: 17,
}
result := InsertUserData(u)
// check result
})
}

Make tests deterministic

A test should pass all the time or fail all the time until fixed. In some cases, it may be hard to maintain the test result stable in the same test. For example, if the test depends on runtime variables (e.g. time, random number, etc).

func genRand() int {
return rand.Intn(100)
}

Make tests independent and self-sufficient

Each test should be independent of another test. For each test case, the setup, execution, and verification steps should not depend on running other tests before it. For example, to test a function with an environment variable, we should pay attention to handle the environment variable correctly (e.g. clear the environment variable after the test case is finished), to avoid it affecting other test cases.

func GetEnvs(en string) []string {
e := os.Getenv(en)
if e == "" {
return nil
}
return strings.Split(e, ",")
}
func TestGetEnvs(t *testing.T) {
if err := os.Setenv("env_name", "value"); err != nil {
t.Error(err)
}
// implement the test case
}

Repeat yourself when necessary

Generally, we intended to keep the test as simple as we can. Sometimes duplicated test code might help us to understand the test case easier, if it makes the implementation simpler and easier to read, it is okay to violate the ‘do not repeat yourself’ principle.

Measure code coverage but focus on test coverage

The main difference of code coverage and test coverage is that code coverage only measure how well the production code is covered by the test, while test coverage measure how well the user requirement is covered.

Code coverage & Test coverage

Code coverage

The code coverage is an indicator to determine how many implementation codes are covered by the test code. It is examined by determining how much code is exercised when we run those test cases. It helps us to determine how reliable is the test implementation is. Higher coverage means the software is tested well by the unit test.

func calcSum(val ...int32) (sum int32) {
if len(val) == 0 {
return
} else {
for _, v := range val {
sum += v
}
return sum
}
}
func TestCalcSum(t *testing.T) {
sum := calcSum(1, 2)
if sum != 3 {
t.Errorf("calcSum return incorrect result")
}
}
$ go test . -cover
ok github.com/vdaas/vald 0.201s coverage: 80.0% of statements
func calcSum(val ...int32) (sum int32) {
if len(val) == 0 { // this statement is executed to check the argument length
return // this is not covered by the test
} else { // not executable code
for _, v := range val { // this part is covered until the return statement below
sum += v
} // not executable code
return sum
}
}

Test coverage

Test coverage is a test type to ensure the functional quality of the software. The goal of test coverage is that we want to cover as many use cases as we can, even it is not required by users. Unlike code coverage, test coverage focuses on thinking about the testing data and covers the part which is not defined in the requirement. It helps to find more hidden bugs or defects in the software.

func doSomething(i int) {
// do something
}
  • math.MinInt32
func doSomethingWithSlice(ss []string) {
// do something
}
  • []string{“”}
  • []string{}
  • []string{“string”}
  • []string{“string1”, “string2”}
func isAdult(a int) bool {
if a >= 20 {
return true
}
return false
}
  • 20
  • 21

How do we implement unit tests in Vald?

Vald is implemented by Go. In Go, we use the standard testing framework to implement the unit test. To reduce the complexity of the implementation of the unit test, we found that table-driven test is suitable in Vald. We use a tool called gotests to generate the test template in table-driven format, with some Vald-specific modification on the template, to make it suitable for our use case.

Conclusion

Thanks for taking the time to read this blog post. In this post, we introduced Vald and few unit test strategies applied in Vald.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store