Benchmarking in GoLang
Benchmarking in GoLang is the practice of evaluating a function or program’s performance to determine its speed and efficiency as well as to pinpoint areas that might need improvement. It facilitates the identification of bottlenecks and the evaluation of the performance effects of various methods or implementations.
How to Write Benchmarks in Go
Through the go test command and its testing package, Go has built-in support for benchmarking.
The following are the main customs and procedures:
File Naming: Benchmark functions, like regular tests, must be stored in files that end in _test.go.
Function Naming: The first word in the name of a benchmark function must be Benchmark. BenchmarkMyFunction is an example. Go test won’t automatically run functions that don’t adhere to this pattern (for example, benchmarkMyFunction whose signature starts with a lowercase “b”).
Function Signature: Functions that are benchmarks take a single parameter of type *testing.B and return func (b *testing.B) with no value.
Looping with b.N: for i := 0; i < b.N; i++, the code being benchmarked is usually executed in a loop inside a benchmark function. To make sure the benchmark runs long enough (at least 1 second by default) to gather reliable results, the Go testing framework dynamically modifies the value of b.N. When a benchmark runs too quickly, it is rerun with b.N. increasing (e.g., 1, 2, 5, 10, 20, 50, and so on).
Preventing Compiler Optimizations: The function’s result is typically stored in a variable and then assigned to a package-level global variable (result = r) to prevent the compiler from optimising away the function being benchmarked (since its result might appear unnecessary). The outcome is now “observable” and the compiler is unable to omit the calculation.
Example: Benchmarking Fibonacci Algorithms
Let’s look at an example where we compare various strategies for computing Fibonacci numbers. Because it requires a lot of mathematical computations, this situation is appropriate for benchmarking.
Define the Algorithms (benchmarkMe.go) Initially, we will define three Fibonacci functions in the benchmarkMe.go file:
package main
import (
"fmt"
)
// fibo1 uses recursion to calculate Fibonacci numbers (simple and somewhat slow)
func fibo1(n int) int {
if n == 0 {
return 0
} else if n == 1 {
return 1
} else {
return fibo1(n-1) + fibo1(n-2)
}
}
// fibo2 is almost identical to fibo1, with a slight change in the conditional structure
func fibo2(n int) int {
if n == 0 || n == 1 {
return n
}
return fibo2(n-1) + fibo2(n-2)
}
// fibo3 uses a map and a for loop for efficiency
func fibo3(n int) int {
fn := make(map[int]int)
for i := 0; i <= n; i++ {
var f int
if i <= 2 {
f = 1
} else {
f = fn[i-1] + fn[i-2]
}
fn[i] = f
}
return fn[n]
}
func main() {
fmt.Println(fibo1(40))
fmt.Println(fibo2(40))
fmt.Println(fibo3(40))
}Output
102334155
102334155
102334155Though their levels of efficiency differ, all three implementations yield the same numerical output.
Write Benchmark Functions (benchmarkMe_test.go), a Benchmark Function. To store the benchmark methods, we will next create a file called benchmarkMe_test.go. Additionally, this file will contain helper functions (benchmarkfibo1, benchmarkfibo2, benchmarkfibo3) that carry out the looping itself and stop compiler optimisations.
package main
import (
"testing"
)
var result int // Global variable to prevent compiler optimizations [10]
// Helper benchmark function for fibo1
func benchmarkfibo1(b *testing.B, n int) {
var r int
for i := 0; i < b.N; i++ {
r = fibo1(n)
}
result = r
}
// Helper benchmark function for fibo2
func benchmarkfibo2(b *testing.B, n int) {
var r int
for i := 0; i < b.N; i++ {
r = fibo2(n)
}
result = r
}
// Helper benchmark function for fibo3
func benchmarkfibo3(b *testing.B, n int) {
var r int
for i := 0; i < b.N; i++ {
r = fibo3(n)
}
result = r
}
// Actual benchmark functions for n=30
func Benchmark30fibo1(b *testing.B) {
benchmarkfibo1(b, 30)
}
func Benchmark30fibo2(b *testing.B) {
benchmarkfibo2(b, 30)
}
func Benchmark30fibo3(b *testing.B) {
benchmarkfibo3(b, 30)
}
// Actual benchmark functions for n=50
func Benchmark50fibo1(b *testing.B) {
benchmarkfibo1(b, 50)
}
func Benchmark50fibo2(b *testing.B) {
benchmarkfibo2(b, 50)
}
func Benchmark50fibo3(b *testing.B) {
benchmarkfibo3(b, 50)
}Output
goos: <your_os>
goarch: <your_architecture>
Benchmark30fibo1-8 300 4494213 ns/op
Benchmark30fibo2-8 300 4463607 ns/op
Benchmark30fibo3-8 500000 2829 ns/op
Benchmark50fibo1-8 1 67272089954 ns/op
Benchmark50fibo2-8 1 67300080137 ns/op
Benchmark50fibo3-8 300000 4138 ns/op
PASS
ok command-line-arguments 145.827sHow to Run Benchmarks
In order to run every benchmark function in your package, open the terminal and navigate to the directory that contains your.go and _test.go files. Then, use the go test command with the -bench flag set to a regular expression that matches all legitimate benchmark functions (for example,.):
go test -bench=. benchmarkMe.go benchmarkMe_test.go
The -benchmem flag is used to add memory allocation statistics to the output:
go test -benchmem -bench=. benchmarkMe.go benchmarkMe_test.go
Interpreting Benchmark Results
Comprehensive performance metrics are provided via the go test -bench command’s output:
goos: darwin
goarch: amd64
Benchmark30fibo1-8 300 4494213 ns/op
Benchmark30fibo2-8 300 4463607 ns/op
Benchmark30fibo3-8 500000 2829 ns/op
Benchmark50fibo1-8 1 67272089954 ns/op
Benchmark50fibo2-8 1 67300080137 ns/op
Benchmark50fibo3-8 300000 4138 ns/op
PASS
ok command-line-arguments 145.827sAdditionally, with -benchmem:
Benchmark30fibo1-8 300 4413791 ns/op 0 B/op 0 allocs/op
Benchmark30fibo2-8 300 4430097 ns/op 0 B/op 0 allocs/op
Benchmark30fibo3-8 500000 2774 ns/op 2236 B/op 6 allocs/op
Benchmark50fibo1-8 1 71534648696 ns/op 0 B/op 0 allocs/op
Benchmark50fibo2-8 1 72551120174 ns/op 0 B/op 0 allocs/op
Benchmark50fibo3-8 300000 4612 ns/op 2481 B/op 10 allocs/op
PASS
ok command-line-arguments 150.500sThe output means the following:
goos: darwin, goarch: amd64: The architecture and operating system used to run the benchmark are indicated.
-8 (after benchmark name): Indicates how many goroutines were used during execution; this usually corresponds to the value of the environment variable GOMAXPROCS.
Second column (e.g., 300, 500000, 1): How many times the function was used. Within the default time limit, faster functions are run more frequently to get precise readings.
Third column (ns/op): The average time, expressed in nanoseconds, for each operation. A lower number denotes superior performance.
Fourth column (B/op) (with –benchmem): The mean quantity of bytes allotted for each operation.
Fifth column (allocs/op) (with –benchmem): How much memory is allocated on average for each operation.
Particularly for higher n values like 50, it is evident from this output that fibo3 (using a map and a for loop) is far faster than fibo1 and fibo2 (recursive techniques). The recursive functions (fibo1, fibo2), in contrast to fibo3, do not employ extra data structures like maps, hence they display 0 B/op and 0 allocs/op.
Best Practices and Common Pitfalls
Optimize Bug-Free Code: Check your code for errors before trying to optimise it. Optimising code that has errors is useless and may cause further problems.
Avoid Premature Optimization: Write correct, understandable code first. Only when profiling pinpoints a particular performance bottleneck should you optimise.
Dedicated Environment: A busy computer or one used for other crucial tasks should never be used to benchmark your Go code because this can affect the accuracy of the measurements.
Correct Loop Usage in Benchmarks: Make sure to utilise b.N. appropriately while designing benchmark functions in order to regulate the benchmark loop. In the b.N loop, the proper method is to call the function with a fixed input (e.g., _ = fibo3(10)). When b.N is used as the input to the function being benchmarked (for example, _ = fibo1(i) or _ = fibo2(b.N)), the benchmark may not finish as the runtime increases with b.N. or may produce inaccurate results.
Profiling Tools: Frequently, benchmarking enhances profiling. Deeper insights into CPU and memory utilization can be obtained with tools like go tool pprof, which can assist in identifying the precise areas of your code that are using the most resources.
You can also read Testing In Go Made Easy: From Unit to Integration Tests