Limiting upload size

When it sounds counter-intuitive to limit the upload size, it is actually a good practice to do so. Limiting the upload size can prevent the server from being overwhelmed by a large file upload which can take a lot of resources, take longer to finish, increase the possibility of failures during the upload, and increase the risk of denial of service attack.

In Go, you can use http.MaxBytesReader to limit how many bytes the handler reads from the request body. The reader does not send a 413 response automatically. Once the limit is exceeded, its next read returns a *http.MaxBytesError, and the handler must translate that error into the appropriate response.

import (
    "errors"
    "io"
    "log"
    "net/http"
    "os"
)

const maxUploadSize int64 = 10 << 20 // 10 MiB

func copyUpload(w http.ResponseWriter, body io.ReadCloser, dst io.Writer) error {
    limitedBody := http.MaxBytesReader(w, body, maxUploadSize)
    _, err := io.Copy(dst, limitedBody)
    return err
}

func binaryUpload() http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        defer r.Body.Close()

        f, err := os.CreateTemp("", "upload-*")
        if err != nil {
            log.Printf("create temporary upload: %v", err)
            http.Error(w, "could not create upload", http.StatusInternalServerError)
            return
        }
        name := f.Name()
        defer os.Remove(name)
        defer f.Close()

        if err := copyUpload(w, r.Body, f); err != nil {
            var maxBytesError *http.MaxBytesError
            if errors.As(err, &maxBytesError) {
                http.Error(w, "upload exceeds 10 MiB limit", http.StatusRequestEntityTooLarge)
                return
            }

            log.Printf("store upload: %v", err)
            http.Error(w, "could not store upload", http.StatusInternalServerError)
            return
        }

        w.WriteHeader(http.StatusOK)
    }
}

The limit above is 10 MiB (10 * 1024 * 1024 bytes). copyUpload wraps the request body before copying it to the destination, so an upload at or below that limit reaches the end of the body normally. For an oversized body, io.Copy returns the *http.MaxBytesError produced by MaxBytesReader; only that error becomes 413 Request Entity Too Large. File creation and other read or write failures are server errors and receive 500 Internal Server Error instead. Keeping the copy operation in a small helper also makes each case straightforward to unit test with controlled readers and writers.

You can verify both boundaries against the server from the previous lesson. These commands generate request bodies without creating large fixture files:

# Exactly 10 MiB: 200
head -c $((10 * 1024 * 1024)) /dev/zero |
  curl --silent --output /dev/null --write-out '%{http_code}\n' \
    --request POST --data-binary @- \
    'http://localhost:8080/api/v1/binary'

# 10 MiB plus one byte: 413
head -c $((10 * 1024 * 1024 + 1)) /dev/zero |
  curl --silent --output /dev/null --write-out '%{http_code}\n' \
    --request POST --data-binary @- \
    'http://localhost:8080/api/v1/binary'

Now, you must be wondering: if we limit the request body, how can we handle a logical file that is larger than the limit?

One option is a resumable upload protocol. The client splits the logical file into bounded requests and the server tracks their offsets. Each individual request still needs a size limit, and the server must separately enforce the declared total file size. Chunking does not make an unlimited upload safe; it gives the client a way to send an allowed large file across several controlled requests. We will build that protocol later in the course.