【Go 开发】pprof 排查问题流程：排查程序 CPU 占用高的问题
原理部分
- 引入 net/http/pprof 进行代码中的分析
import _ "net/http/pprof"
import "net/http"
func main() {
// 启动 pprof 服务器
go func() {
http.ListenAndServe("localhost:6060", nil)
}()
// 其他业务代码
}
这样会在 localhost:6060 启动一个 HTTP 服务器，可以在浏览器打开 http://localhost:6060/debug/pprof/ 查看各类 pprof 信息。
- 生成和查看 pprof 文件
在终端使用 go tool pprof 命令生成和查看 pprof 文件。例如：
go tool pprof http://localhost:6060/debug/pprof/heap # 查看内存使用情况
go tool pprof http://localhost:6060/debug/pprof/profile # 查看 CPU 使用情况
你还可以将这些信息保存为文件以供离线分析：
curl -o heap.prof http://localhost:6060/debug/pprof/heap
curl -o cpu.prof http://localhost:6060/debug/pprof/profile?seconds=30
go tool pprof heap.prof
go tool pprof cpu.prof
-
常见的 pprof 分析命令
进入 pprof 交互式命令行界面后，可以使用以下命令进行分析：
• top：显示 CPU 或内存消耗最多的函数，默认按消耗排序。
• list <function_name>：查看指定函数的源码以及性能消耗信息。
• web：以图形界面展示性能分析结果（需要安装 Graphviz）。
• svg：生成 SVG 格式的分析图。
• png：生成 PNG 格式的分析图。 -
分析 CPU 和内存的使用情况
在排查问题时，可以重点关注以下两方面：
• CPU Profiler：用于分析 CPU 使用情况，帮助找到耗时较长的代码段。
• Heap Profiler：用于分析内存使用情况，帮助找到内存泄漏和过高的内存使用问题。 -
常见性能瓶颈
- CPU 使用过高：检查是否有热点函数（CPU 消耗较高的函数），并尝试优化相关代码。
- 内存泄漏：关注 heap 的 top 输出中是否有内存增长，或者某些数据结构未释放。
- 阻塞操作：查看 goroutine 的 profile，检查是否存在锁竞争或通道阻塞问题。
实例演示
内存
go tool pprof http://localhost:6060/debug/pprof/heap
Fetching profile over HTTP from http://localhost:6060/debug/pprof/heap
Saved profile in /home/hope/pprof/pprof.td2_data_center_new.alloc_objects.alloc_space.inuse_objects.inuse_space.001.pb.gz
File: td2_data_center_new
Build ID: 25244974efa19a16891d82f43c0aca01f0ccd04b
Type: inuse_space
Time: Nov 13, 2024 at 3:53pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 510.56MB, 94.54% of 540.07MB total
Dropped 34 nodes (cum <= 2.70MB)
Showing top 10 nodes out of 54
flat flat% sum% cum cum%
244.62MB 45.29% 45.29% 244.62MB 45.29% td2_data_center/app.LoadPushConf
185.02MB 34.26% 79.55% 185.52MB 34.35% gitlab.yeahka.com/tdcode2_go/pkg/river.(*TableFilter).slice2Map
22.90MB 4.24% 83.79% 22.90MB 4.24% github.com/confluentinc/confluent-kafka-go/kafka.NewProducer
20.52MB 3.80% 87.59% 20.52MB 3.80% github.com/go-mysql-org/go-mysql/packet.(*Conn).ReadPacketReuseMem
12MB 2.22% 89.81% 12MB 2.22% bufio.NewWriterSize
9MB 1.67% 91.48% 19.50MB 3.61% github.com/go-mysql-org/go-mysql/replication.(*RowsEvent).decodeValue
4.50MB 0.83% 92.31% 4.50MB 0.83% time.Time.Format
4MB 0.74% 93.06% 4MB 0.74% encoding/json.Marshal
4MB 0.74% 93.80% 4MB 0.74% github.com/syndtr/goleveldb/leveldb/memdb.New
4MB 0.74% 94.54% 155.42MB 28.78% gitlab.yeahka.com/tdcode2_go/pkg/river.(*TableFilter).parseUpdate
(pprof) list app.LoadPushConf
Total: 540.07MB
ROUTINE ======================== td2_data_center/app.LoadPushConf in /home/hope/go/src/td2_data_center/app/config.go
244.62MB 244.62MB (flat, cum) 45.29% of Total
. . 205: for _, pushItem := range MyPushInfo.PushConfig {
. . 206: // 每个商户都对应一个chan
. . 207: PushChanMapTmp := make(map[int]chan *river.RowData)
. . 208: for i := 0; i < MyCfg.PushCoMax; i++ {
. . 209: //pushChan := make(chan *river.RowData, 8000)
244.62MB 244.62MB 210: pushChan := make(chan *river.RowData, 10000)
. . 211: PushChanMapTmp[i] = pushChan
. . 212: }
. . 213:
. . 214: // pushChan 作为推送chan的标识，订单和流水这类有时序性的需要放在一起
. . 215: _, ok := PushChanMapList[pushItem.TableFilter.PushChan]
(pprof) svg
Generating report in profile002.svg
CPU
hope@apptest42 td2_data_center]$ go tool pprof http://localhost:6060/debug/pprof/heap
Fetching profile over HTTP from http://localhost:6060/debug/pprof/heap
Saved profile in /home/hope/pprof/pprof.td2_data_center_new.alloc_objects.alloc_space.inuse_objects.inuse_space.003.pb.gz
File: td2_data_center_new
Build ID: 25244974efa19a16891d82f43c0aca01f0ccd04b
Type: inuse_space
Time: Nov 13, 2024 at 3:55pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 510.56MB, 94.54% of 540.07MB total
Dropped 34 nodes (cum <= 2.70MB)
Showing top 10 nodes out of 54
flat flat% sum% cum cum%
244.62MB 45.29% 45.29% 244.62MB 45.29% td2_data_center/app.LoadPushConf
185.02MB 34.26% 79.55% 185.52MB 34.35% gitlab.yeahka.com/tdcode2_go/pkg/river.(*TableFilter).slice2Map
22.90MB 4.24% 83.79% 22.90MB 4.24% github.com/confluentinc/confluent-kafka-go/kafka.NewProducer
20.52MB 3.80% 87.59% 20.52MB 3.80% github.com/go-mysql-org/go-mysql/packet.(*Conn).ReadPacketReuseMem
12MB 2.22% 89.81% 12MB 2.22% bufio.NewWriterSize
9MB 1.67% 91.48% 19.50MB 3.61% github.com/go-mysql-org/go-mysql/replication.(*RowsEvent).decodeValue
4.50MB 0.83% 92.31% 4.50MB 0.83% time.Time.Format
4MB 0.74% 93.06% 4MB 0.74% encoding/json.Marshal
4MB 0.74% 93.80% 4MB 0.74% github.com/syndtr/goleveldb/leveldb/memdb.New
4MB 0.74% 94.54% 155.42MB 28.78% gitlab.yeahka.com/tdcode2_go/pkg/river.(*TableFilter).parseUpdate
(pprof) list app.LoadPushConf
Total: 540.07MB
ROUTINE ======================== td2_data_center/app.LoadPushConf in /home/hope/go/src/td2_data_center/app/config.go
244.62MB 244.62MB (flat, cum) 45.29% of Total
. . 205: for _, pushItem := range MyPushInfo.PushConfig {
. . 206: // 每个商户都对应一个chan
. . 207: PushChanMapTmp := make(map[int]chan *river.RowData)
. . 208: for i := 0; i < MyCfg.PushCoMax; i++ {
. . 209: //pushChan := make(chan *river.RowData, 8000)
244.62MB 244.62MB 210: pushChan := make(chan *river.RowData, 10000)
. . 211: PushChanMapTmp[i] = pushChan
. . 212: }
. . 213:
. . 214: // pushChan 作为推送chan的标识，订单和流水这类有时序性的需要放在一起
. . 215: _, ok := PushChanMapList[pushItem.TableFilter.PushChan]
(pprof) web
exec: "sensible-browser": executable file not found in $PATH
(pprof) svg
Generating report in profile001.svg