Installing Go directly from source is easier than you might think.
The Go team manages releases based on tags (versions). You can check out a specific version and install it from source.
Since Go 1.15, Go uses a Go compiler, meaning it compiles itself. If you already have Go installed, you can use it to compile a newer version, as long as the version gap is within two major releases.
All tagged sources are available at go.googlesource.com/go. The GitHub repository mainly holds the master branch, primarily for contributors.
Check it out and happy coding!
Start here: Install Go from Source (go.dev/doc/install/source)