103 is a new experimental HTTP status code defined in RFC 8297. It’s an informational status that can be sent by a server before the main HTTP response. Used in conjunction with the Link HTTP header and the preload relation, 103 gives the client the opportunity to fetch resources (assets, images, related API documents…) related to the explicitly requested one, as early as possible, and while the server is preparing the main response. Early Hints look like that:
HTTP/1.1 103 Early Hints
Link: </style.css>; rel=preload; as=style
Link: </script.js>; rel=preload; as=script
HTTP/1.1 200 OK
Date: Fri, 26 May 2017 10:02:11 GMT
Content-Length: 1234
Content-Type: text/html; charset=utf-8
Link: </style.css>; rel=preload; as=style
Link: </script.js>; rel=preload; as=script
[… rest of the response body is omitted from the example …]Early Hints in the Wild
The 103 Early Hints status code could be a good alternative to HTTP/2 Server Push, which will be removed from Chrome and discouraged in the spec. It adds 1 RTT compared to Server Push, but – among other advantages – it allows better caching and is (theoretically) easier to implement. Chrome and Fastly are running an experiment to measure the potential benefit of this new status code. But for this experiment to be successful, we need compatible servers… and servers are waiting for compatible browsers before implementing this new status code. We’re in a typical “the chicken or the egg” situation.
Go and Early Hints
As you may know, I’m very interested in the topic of resource preloading applied to web APIs. I created the Vulcain protocol, which is an alternative to (some features of) GraphQL. It allows designing fast and idiomatic client-driven APIs, strictly following the REST architectural style. Vulcain (the protocol) supports Early Hints since day one, and has been designed with compatibility with this status code in mind. We will publish a new revision of Vulcain taking into account the twilight of Server Push soon, and I’ll present what this changes for the protocol in depth during AFUP Day 2021.
However, the Vulcain Gateway Server (the reference implementation), doesn’t support the new status code yet. So servers using this component cannot participate in the experiment.
As the Mercure.rocks hub, the Vulcain Gateway Server is now available as a module for the brilliant Caddy Web Server. This has been possible because both Caddy and the library implementing Vulcain are written in Go. Unfortunately, the standard library of Go doesn’t support the 103 status code yet. This prevents using Early Hints with Caddy and Vulcain.
To move forward, I submitted to the Go project patches implementing the RFC for HTTP/1.1 and for HTTP/2. They aren’t merged yet, but as Go 1.16 has been released yesterday, they may land soon in the development branch. In the meantime, it’s already possible to use this feature in your own Go programs, and it’s what we’ll see in the rest of this article!
The Go toolchain (especially the gc compiler) has an interesting characteristic: it creates statically-linked binaries by default. This means that once compiled with a development version of Go supporting the new feature, your standalone binaries can be deployed without requiring any change to your servers.
First be sure that the current stable version of Go is installed on your system. Because the Go toolchain itself is written in Go, we need Go to compile Go. It’s called the bootstrapping process (another instance of the “chicken or the egg” problem).
Then, clone my fork of Go and checkout the branch containing the required changes for HTTP/1.1:
git clone https://github.com/dunglas/go.git dunglas-go
cd dunglas-go
git checkout feat/http-103-status-codeThis branch contains the patch for HTTP/1.1 but not for HTTP/2. The implementation of HTTP/2 of Go is stored in a separated module: x/net/http2. Before creating our custom build of Go, we need to retrieve the patched version of x/net/http2 and to bundle it in the standard library.
Start by replacing the x/net package by my fork of it, and update the vendored dependencies:
export GOROOT=$(pwd)
cd src/
go mod edit -replace="golang.org/x/net=github.com/dunglas/[email protected]"
go mod vendor
unset GOROOTThen, install the bundle command, and use it to bundle the net/http module:
go get golang.org/x/tools/cmd/bundle
cd net/http/
$(go env GOPATH)/bin/bundle -o=h2_bundle.go -dst net/http -prefix=http2 -tags='!nethttpomithttp2' golang.org/x/net/http2The file h2_bundle.go has been replaced by a bundle containing our patch!
Finally, go back in the src/ directory and build our 103-enabled Go version:
cd ../../
./make.bashOur enhanced Go compiler is ready!
Sample Program
Implementing the example provided in the RFC is straightforward:
// main.go
package main
import (
"io"
"log"
"net/http"
)
func main() {
helloHandler := func(w http.ResponseWriter, req *http.Request) {
w.Header().Add("Link", "</style.css>; rel=preload; as=style")
w.Header().Add("Link", "</script.js>; rel=preload; as=script")
w.WriteHeader(103)
// do your heavy tasks such as DB or remote APIs calls here
w.WriteHeader(200)
io.WriteString(w, "<!doctype html>\n[... rest of the response body is omitted from the example ...]")
}
http.HandleFunc("/hello", helloHandler)
log.Fatal(http.ListenAndServeTLS(":443", "cert.pem", "key.pem", nil))
}To generate a locally-trusted certificate TLS certificate (necessary to use HTTP/2), I recommend the mkcert command:
mkcert -cert-file ./cert.pem -key-file ./key.pem localhostUse the custom build of Go we created to compile the program, and start it:
/path/to/dunglas-go/bin/go build main.go
./main mainAlternatively, execute go run main.go to compile and start the program with just one command. Use curl to check if it works properly.
With HTTP/1.1:
curl -v --http1.1 https://localhost/helloAnd with HTTP/2:
curl -v https://localhost/helloYou should see something like that:
[snip]
< HTTP/2 103
< link: </style.css>; rel=preload; as=style
< link: </script.js>; rel=preload; as=script
< HTTP/2 200
< link: </style.css>; rel=preload; as=style
< link: </script.js>; rel=preload; as=script
< content-type: text/html; charset=utf-8
< content-length: 79
< date: Sat, 13 Feb 2021 09:47:27 GMT
<
<!doctype html>
[snip]Conditionally Sending Early Hints
Calling http.ResponseWriter.WriteHeader() several times will cause an error with the vanilla Go runtime. To be able to compile the same program with the stable compiler and with the patched one, you can wrap the call to http.ResponseWriter.WriteHeader(103) in a condition:
package main
import (
"io"
"log"
"net/http"
"runtime"
"strings"
)
func main() {
helloHandler := func(w http.ResponseWriter, req *http.Request) {
w.Header().Add("Link", "</style.css>; rel=preload; as=style")
w.Header().Add("Link", "</script.js>; rel=preload; as=script")
if strings.HasPrefix(runtime.Version(), "devel") {
// skip if compiled with a stable version of Go
w.WriteHeader(103)
}
w.WriteHeader(200)
io.WriteString(w, "<!doctype html>\n[... rest of the response body is omitted from the example ...]")
}
http.HandleFunc("/hello", helloHandler)
log.Fatal(http.ListenAndServeTLS(":443", "cert.pem", "key.pem", nil))
}103 Early Hints and HTTP/3
HTTP/3 is the upcoming version of the king of web protocols. It should become an RFC soon. HTTP/3 is already enabled by default in Safari and is available under a flag in Firefox and Chrome.
An experimental implementation of HTTP/3 for Go (which is used by Caddy and Vulcain) is available, but it doesn’t support the 103 status code either. So I also opened a Pull Request to add support for this status code to quic-go! To use the 103 status code with HTTP/3, try this patch.
And voilà! You’re ready to create programs supporting this new status code, and you can participate in this experiment to make the web faster and greener! If you do so, let me know on Twitter! The Vulcain Gateway Server will soon be updated to support the 103 status code using the approach we’ve seen in this article. And if you encounter any bug while playing with these patches, please report them!
If you liked this article, and want to contribute to my research work about web APIs, HTTP and Go, consider sponsoring me!
