panicparse v2.0.1

🐎 Do race conditions in style

2020-08-10 golang panicparse

TL;DR

2020-12-19 Update: I used to recommend to use github.com/maruel/panicparse/v2/cmd/pp path but it’s not required anymore. To upgrade run:

go get -u github.com/maruel/panicparse/cmd/pp

Never used it? See usage.

New features

panicparse is a popular tool to condense and distill Go goroutine snapshots. v1.4.0 was great (see blog post) but there were a few big feature requests that I knew would require significant refactoring to support:

Race detector output parsing
HTML output when using package stack directly
Go module support
Process a stream of crashes

I’ve taken the last week to finish the implementation and polish these features. I’m glad to announce that panicparse v2.0.1 is now available for general use!

Race detector

The race detector output is not generated by the Go standard library but instead by the LLVM thread sanitizer (TSan). This is why the output is so different. As such, I had to significantly improve the parsing state machine to be able to process this format.

Try it at home by compiling the racy version of the testing tool panic:

go get github.com/maruel/panicparse/v2/cmd/pp
go get -race github.com/maruel/panicparse/v2/cmd/panic

The output of panic race will look like this:

$ panic race
GOTRACEBACK=all
==================
WARNING: DATA RACE
Read at 0x00c0001bc030 by goroutine 8:
  main.panicDoRaceRead()
      /home/maruel/go/pkg/mod/github.com/maruel/panicparse/v2@v2.0.1/cmd/panic/main.go:144 +0x3a
  main.panicRace.func2()
      /home/maruel/go/pkg/mod/github.com/maruel/panicparse/v2@v2.0.1/cmd/panic/main.go:161 +0x38

Previous write at 0x00c0001bc030 by goroutine 7:
  main.panicDoRaceWrite()
      /home/maruel/go/pkg/mod/github.com/maruel/panicparse/v2@v2.0.1/cmd/panic/main.go:139 +0x44
  main.panicRace.func1()
      /home/maruel/go/pkg/mod/github.com/maruel/panicparse/v2@v2.0.1/cmd/panic/main.go:158 +0x38

Goroutine 8 (running) created at:
  main.panicRace()
      /home/maruel/go/pkg/mod/github.com/maruel/panicparse/v2@v2.0.1/cmd/panic/main.go:160 +0xa6
  main.main()
      /home/maruel/go/pkg/mod/github.com/maruel/panicparse/v2@v2.0.1/cmd/panic/main.go:72 +0x712

Goroutine 7 (running) created at:
  main.panicRace()
      /home/maruel/go/pkg/mod/github.com/maruel/panicparse/v2@v2.0.1/cmd/panic/main.go:157 +0x84
  main.main()
      /home/maruel/go/pkg/mod/github.com/maruel/panicparse/v2@v2.0.1/cmd/panic/main.go:72 +0x712
==================

This is a bit hard to read. Thankfully, with the new race detector parsing algorithm, the output becomes much more manageable with: panic race |& pp

panicparse parsing a detected race condition.

HTML output

Another frequently requested feature is to officially support HTML output. While panicparse had the -html flag for a while, I never exposed it in the stack package. This is now available for general use! Try it locally:

panic race |& pp -html a.html

go get github.com/maruel/serve-dir
serve-dir

then visit http://localhost:8010/a.html. It will look like this:

panicparse generated HTML from the race detector.

Either use the ToHTML() method directory or better, seamlessly integrate the HTML generator live on your application with webstack.SnapshotHandler.

Go module

Go module detection was tricky to implement. Since the primary use case is for local use, that’s what I optimized for with the initial release. The implementation supports the use of the replace directive in go.mod.

Each detected source file is matched to one of these categories:

Go module, where a corresponding go.mod was found locally; this is considered to be the “main project” being investigated
Under $GOPATH/src, the legacy location of Go source files
Under $GOPATH/pkg/mod, where Go module dependencies are stored; read: third parties
Under $GOROOT/src, the standard library

With this new detection logic, the previous color coding of “yellow, red, green” had to be updated. A new color coded legend was created, as you can see in the HTML screenshot above. The pp tool also use new colors and the legend is printed with pp -help.

Stream of crashes

The stack.ParseDump() function was replaced with stack.ScanSnapshot(). It now stops fetching the input after it is done processing a goroutines snapshot, so it is now possible to process consecutive snapshots.

→ This enables extracting the crashes from a server log and do analytics on it!

Miscellaneous

There’s a ton of other improvements. I’ve improved panic and panicweb, tweaked the parsing to be more solid and improved performance. It now supports arbitrarily long lines while being memory efficient in the common case. I increased test coverage.

#TFW you spend 2 hours to write >100 lines of new unit test code to test corner cases, and the code coverage increases only by 0.3%.
— Marc-Antoine Ruel (@marcaruel) August 5, 2020

I revived the Go source parsing algorithm to enhance the arguments presentation, and updated the unit tests accordingly to ensure it doesn’t break again, then made it the default. The project is now tested on 1.9.7, 1.12.7 and the current version, and tested on MacOS, Ubuntu and Windows. Previously I was manually copying the code on laptops on different OSes to test that I didn’t break the project in an OS specific way. Let’s just say I wasn’t successful at that. This means that panicparse is going to be much more stable going forward.

I’ve removed the vendored code. As such Go 1.9.7 is now the minimum required version.

Why v2.0.1?

Why v2.0.1 instead of v2.0.0? The Go package paths were incorrect in v2.0.0, so I had to fix them in the source but the tag v2.0.0 was already in Go proxy sumdb, so it was too late for me to delete it.

In conclusion

Since I announced panicparse at Gophercon 2015, I invested hundreds of hours to improve the tool for all Go developers.

I hope you’ll appreciate the tool and it’ll help you diagnose issues faster. If so, please star the repository on github so I can better gauge the enthusiasm and see if it’s worth continuing to invest time in the project.

Thanks!