panicparse v1.6.0

This release was the hardest one ever

Astute readers using panicparse will ask themselves: Marc-Antoine, haven’t you released v2.0.1 several months ago and now you are releasing v1.6.0?

Yep I did.

Content warning

But oh boy I didn’t know what was coming.

Long term support

I’m the kind of person who likes to support things for a long time. I don’t enjoy breaking my users. In short, I wanted the following to work:

go get golang.org/dl/go1.9.7
go1.9.7 download
go1.9.7 get github.com/maruel/panicparse/cmd/pp

go get golang.org/dl/go1.16beta1
go1.16beta1 download
go1.16beta1 get github.com/maruel/panicparse/cmd/pp

In practice, I only needed go1.11 to work, but I got up to 1.9.7 for free. I hit a few challenges:

• I knew that go1.16beta1 breaks directory names with non-ascii characters (issue #43052) and that both v1.x and v2.x would be broken by. It used to be kinda broken only when fetching as a go module, and now it’s hard broken.
• Earlier go versions do not understand the “v2” directory so using a subdirectory named v2 would not work, even if this is what is recommended in the documentation.
• I was extremely bummed that go get github.com/maruel/panicparse/cmd/pp did the wrong thing (fetch v1) silently from the perspective of the users and had a hard time figuring out a way to fix this.
• Go modules is a rare case where there’s too much official documentation 1, 2, 3 and official blog posts 1, 2, 3, 4, 5. I don’t want to read an essay on the engineering decisions behing the current design, I want to release a OSS side project that I work on the weekend on, and every single minute I spend on this is time I don’t spend with my family. I find it odd that the Go programming language specification is less words than all the reading required to get up to speed with Go modules.
• Things will only fails when it’s too late: when you have published a git tag. Couple this with the fact that once the Go proxy knows about your version, it’s there forever. I didn’t find a way to vet my package before publishing, since it implicitly depends on the git tag.
• It really only start to break for real when you publish v3. The error is silent; you publish your broken package and it doesn’t show up on pkg.go.dev and go get will fetch the previous version without any mention of the more recent broken one. The only way to know if to specifically request a broken version. It will print you an error message. Thankfully I had a OSS toy project that I knew nobody dependended on that was already at v3 that I could test on. The error looks like this when toying with versioned subdirectories (which is incorrect):

  $ go get -u github.com/maruel/msgbus/v3@v3.1.1
  go get github.com/maruel/msgbus/v3@v3.1.1: github.com/maruel/msgbus@v3.1.1: invalid version: module contains a go.mod file, so major version must be compatible: should be v0 or v1, not v3
  

In addition to the challenges mentioned above, I also found many annoyances but I’ll save them for another day.

My goal

My goal was simple:

• Do not break my current user.
• Effectively version the CLI tool (the pp executable) independently from the library stack so that getting the v1 CLI tool provides the v2 (race detector parsing) functionality.
• I wanted it to work when not in Go modules mode. This will eventually become irrelevant but is still for my use case.
• Not loose my sanity in the process.

How to get there

I worked it out with a combination of a few tricks.

1. I had to drop support for really old versions. That was fine.
2. I removed the previously vendored packages because it was incompatible with my GitHub Actions tests and it caused me headaches for testing.
3. The really magic part was to vendor stack@v2.1.0 back into vendor/ in the v1.x branch. The magic here is that the vendor/ directory is going to be used only when not in go modules mode, which is exactly what we want.
4. Thankfully, the executables where already mostly a shim, so copied executables source code and internal/ directly from v2.1.0.

Learnings

I think my main take away is that in go modules days, you should seriously consider using separate git repositories for your executables and the packages. That’s all. In abstract it makes sense, you make separate versions, and it’s generally surprising for users if they get bumped to a new major version silently. Where this breaks down is in the following format (which panicparse follows):

- LICENSE
- go.mod
- cmd/
  - tool_foo/
    - main.go
- foo/
  - foo.go

and you want to make a breaking change to the reusable package foo without having to bump the version of the CLI tool tool_foo.

The challenge is that often I’ll write a toy CLI tool, then will carve out a package that I think has potential for reuse, which leads to a period of uncertainty for the API. Is it too specific for the tool use case or is it too generic? Fleshing an API out requires time, but in the meantime as a tool maintainer you have a useful CLI tool that you don’t want to break in the process. That’s where I find the status quo hurts the most, because the git tag and the versions must match starting with v3 so versioning directories do not help, and having go.mod files in subdirectories breaks go test ./....

Anyhow, I’m glad I found a way and hope the learnings here will help another package maintainer write Go packages that predate Go modules.