Profiling in Go
The method of dynamic program analysis known as profiling in Go measures a variety of program execution-related data. Its primary goal is to increase your program’s performance and assist you better understand how it behaves. Bugs in your code may potentially be discovered by profiling.
Types of Profiling
Two main types of profiling are supported by Go:
CPU profiling:Tracks CPU consumption over time to determine which processes use the most CPU cycles.
Memory profiling:Measures memory allocations in order to identify inefficient memory use or leaks.
Generally speaking, profiling an application for both CPU and memory at the same time is not advised because these two profiling techniques are incompatible.
How to Profile in Go
The main way that Go offers profiling features is through its runtime/pprof standard library package. The package net/http/pprof is used for web applications.
The standard stages for creating a profile are:
Import runtime/pprof: To generate profiling data, your software needs to import runtime/pprof, either directly or indirectly.
Create an output file: The file containing the profiling data should be created using os.Create().
Start and Stop Profiling:
- For CPU profiling, call
pprof.StartCPUProfile(file)to begin andpprof.StopCPUProfile()to end. It is good practice todefer pprof.StopCPUProfile()immediately after starting. - For memory profiling, call
pprof.WriteHeapProfile(file).
Run your program: Run your Go program to produce the profile information.
The profiler may not have enough time to gather the required samples if your application runs too quickly, which would leave you without any meaningful information. That’s why time.In order to slow down functions, examples occasionally include calls to sleep().
Code Example
To demonstrate both CPU and memory profiling, let’s look at profileMe.go, a program that benchmarks several Fibonacci algorithms and prime number checks.
Define the functions (profileMe.go)
This file has multiple routines to illustrate various performance attributes:
package main
import (
"fmt"
"math"
"os"
"runtime"
"runtime/pprof" // Required for profiling
"time"
)
// fibo1 uses recursion to calculate Fibonacci numbers (simple and somewhat slow)
func fibo1(n int) int64 {
if n == 0 || n == 1 {
return int64(n)
}
time.Sleep(time.Millisecond) // Added to slow it down for profiling example
return int64(fibo1(n-1)) + int64(fibo1(n-2))
}
// fibo2 uses a map and a for loop for efficiency
func fibo2(n int) int {
fn := make(map[int]int)
for i := 0; i <= n; i++ {
var f int
if i <= 2 {
f = 1
} else {
f = fn[i-1] + fn[i-2]
}
fn[i] = f
}
time.Sleep(50 * time.Millisecond) // Added to slow it down for profiling example
return fn[n]
}
// N1 finds if a number is prime, optimized by checking up to n/2
func N1(n int) bool {
k := math.Floor(float64(n/2 + 1))
for i := 2; i < int(k); i++ {
if (n % i) == 0 {
return false
}
}
return true
}
// N2 finds if a number is prime, checking up to n
func N2(n int) bool {
for i := 2; i < n; i++ {
if (n % i) == 0 {
return false
}
}
return true
}
func main() {
// --- CPU Profiling ---
cpuFile, err := os.Create("/tmp/cpuProfile.out") // Create file for CPU profile data
if err != nil {
fmt.Println(err)
return
}
pprof.StartCPUProfile(cpuFile) // Start CPU profiling
defer pprof.StopCPUProfile() // Ensure CPU profiling is stopped when main exits
total := 0
for i := 2; i < 100000; i++ {
if N1(i) { // Call optimized prime check
total = total + 1
}
}
fmt.Println("Total primes (N1):", total)
total = 0
for i := 2; i < 100000; i++ {
if N2(i) { // Call less optimized prime check
total = total + 1
}
}
fmt.Println("Total primes (N2):", total)
for i := 1; i < 30; i++ { // Smaller range for fibo1 to avoid excessively long runtime
fmt.Print(fibo1(i), " ") // Call recursive Fibonacci
}
fmt.Println()
for i := 1; i < 90; i++ { // Call iterative Fibonacci
fmt.Print(fibo2(i), " ")
}
fmt.Println()
runtime.GC() // Force garbage collection before memory profiling if needed
// --- Memory Profiling ---
memoryFile, err := os.Create("/tmp/memoryProfile.out") // Create file for memory profile data
if err != nil {
fmt.Println(err)
return
}
defer memoryFile.Close() // Ensure memory file is closed
for i := 0; i < 10; i++ {
s := make([]byte, 50000000) // Allocate large slices to trigger memory allocations
if s == nil {
fmt.Println("Operation failed!")
}
time.Sleep(50 * time.Millisecond)
}
err = pprof.WriteHeapProfile(memoryFile) // Write heap profile data
if err != nil {
fmt.Println(err)
return
}
}Output
Total primes (N1): 9592
Total primes (N2): 9592
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711 28657 46368 75025 121393 196418 317811 514229
1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711 28657 46368 75025 121393 196418 317811 514229 832040 1346269 2178309 3524578 5702887 9227465 14930352 24157817 39088169 63245986 102334155 165580141 267914296 433494437 701408733 1134903170 1836311903 2971215073 4807526976 7778742049 12586269025 20365011074 32951280099 53316291173 86267571272 139583862445 225851433717 365435296162 591286729879 956722026041 1548008755920 2504730781961 4052739537881 6557470319842 10610209857723 17167680177565 27777890035288 44945570212853 72723460248141 117669030461000 190392490709141 308061521170141 498454011879282 806515533049423 1304969544928705 2111485077978128 3416454622906833 5527939700884961 8944394323791794 14472334024676755 23416728348468549 37889062373145304 61305790721613853 99194853094759157 160500643816373010 Run the program
Launch the software and run the profileMe.Open the file:
go run profileMe.goThis command will run the program, and in the process, it will create two profiling data files: /tmp/cpuProfile.out and /tmp/memoryProfile.out.
$ ls -l /tmp/*Profile*
-rw-r--r-- 1 user group 1965 Mar 8 16:53 cpuProfile.out
-rw-r--r-- 1 user group 484 Mar 8 16:53 memoryProfile.outInterpreting Profiling Data with
You can use the go tool pprof command to examine the profiling data once it has been collected.
Examine CPU Profile:
This will launch a pprof shell that is interactive. The available commands will be listed by typing help.
- top command: Displays the top items, or functions, that are using up the most CPU time.
flat: Spending time in the function itself.flat%: Percentage that this function accounts for of all samples.cum: Total time in the function and all of the functions it calls.cum%: cumulative proportion of all samples.
- A bottleneck is evident from the result, which shows that main.N2, the less optimised prime checker, takes up a sizable amount of execution time.
- list function_name command: A function’s detailed, annotated source code is provided via the list function_name command, which also indicates the locations of time spent on the function.
- The if (n % i) == 0 line in the for loop is the primary cause of main.N1’s execution time, as this output demonstrates.
- Web Interface: The go utility pprof has a web-based user interface in versions 1.10 and up. To start it, use:
- With the help of this command, a web browser will launch and show an interactive representation of the profiling data, frequently in the form of an annotated source code or call graph. For the online interface or for creating graphical output, such as PDFs, Graphviz is necessary.
Examine Memory Profile:
The memory profile can be examined similarly to the CPU profile:
This will launch the pprof shell, which lets you examine memory allocations using commands like top and list.
Additional Profiling Tools
go tool trace utility: Syscalls, heap size changes, GC events, goroutine creation and blocking, and other execution events are all captured by the trace files that are viewed by this tool. It also offers a visualisation web interface.
github.com/pkg/profile: Using a single defer profile, this external Go package makes profiling setup easier.Start().Use the Stop() function.
Best Practices for Profiling
Optimize Bug-Free Code: Check your code for errors before trying to optimise it. Optimising code that contains errors is ineffective.
Avoid Premature Optimization: Write correct, understandable code first. Optimise only when a particular performance bottleneck is found through profiling.
Dedicated Environment: A busy computer or one used for other critical duties should never be used to benchmark or profile your Go code because this can affect the accuracy of the measurements.
Profile One Type at a Time: Avoid profiling memory and CPU at the same time, as previously stated.
Profiling Complements Benchmarking: While benchmarking aids in performance measurement, profiling aids in identifying performance problems.
You may quickly pinpoint areas for improvement and have a thorough understanding of how well your Go programs are performing by utilising these techniques and resources.
You can also read What Is Benchmarking In GoLang? Understanding Performance