Precondition check

Getting the offset is something that we need to do before we can start uploading the content. The offset is the position where the content should be written to the file. If we know what the offset is, we should be able to write the content to the file at the correct position even when the upload is paused or interrupted.

PATCH request

The API we will be discussing here is the PATCH request. The PATCH request is used to upload the content to the server. The same URL returned on the Location header when initiating the upload will be used to upload the content.

Request:

PATCH /api/v3/files/d0c5aa97-4ef7-48db-a3e1-6b07170bf3d5 HTTP/1.1
Host: tus.example.org
Content-Type: application/offset+octet-stream
Content-Length: 30
Upload-Offset: 70
Tus-Resumable: 1.0.0

[remaining 30 bytes]

Response:

HTTP/1.1 204 No Content
Tus-Resumable: 1.0.0
Upload-Offset: 100

As you have seen, the PATCH request expcects us to provide the Upload-Offset header. The Upload-Offset header indicates the current offset of the upload in bytes. When writing the file, server must continue from the offset specified by the client and update its metadata. Once the server receives the content, it should return the Upload-Offset header to indicate the new offset of the upload in bytes. Client should use this value to calculate the next offset when uploading the next chunk if the upload is not completed.

Requirements

We will breakdown all requirements for uploading the content. As defined in the Tus protocol for PATCH request, at least there are the following requirements:

  • All PATCH requests MUST use Content-Type: application/offset+octet-stream, otherwise the server SHOULD return a 415 Unsupported Media Type status.

  • If the server receives a PATCH request against a non-existent resource it SHOULD return a 404 Not Found status.

  • The Upload-Offset header’s value MUST be equal to the current offset of the resource. If the offsets do not match, the Server MUST respond with the 409 Conflict status without modifying the upload resource.

  • The Server SHOULD accept PATCH requests against any upload URL and apply the bytes contained in the message at the given offset specified by the Upload-Offset header.

  • The Server SHOULD always attempt to store as much of the received data as possible.

  • The Server MUST acknowledge successful PATCH requests with the 204 No Content status. It MUST include the Upload-Offset header containing the new offset. The new offset MUST be the sum of the offset before the PATCH request and the number of bytes received and processed or stored during the current PATCH request.

  • The Client SHOULD send all the remaining bytes of an upload in a single PATCH request, but MAY also use multiple small requests successively for scenarios where this is desirable. One example for these situations is when the Checksum extension is used.

  • Both Client and Server, SHOULD attempt to detect and handle network errors predictably. They MAY do so by checking for read/write socket errors, as well as setting read/write timeouts. A timeout SHOULD be handled by closing the underlying connection.

What a requirements! But don’t worry, we will implement all of these requirements step by step.

Implementation

In this lesson, let’s try to implement all initial checks first. We will leave the actual content writing to the file for the next lesson.

Defining the handler

apiV3Router.HandleFunc("/files/{file_id}", controller.ResumeUpload()).Methods(http.MethodPatch)

Pretty much similar with HEAD request, we will need to get the file_id from the URL path for the PATCH request.

func (c *Controller) ResumeUpload() http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        vars := mux.Vars(r)
        fileID := vars["file_id"]
        // ... the rest of the code
    }
}

Checking Content-Type

The first thing we need to do is to check the Content-Type header. As defined in the requirements, we need to return 415 Unsupported Media Type if the Content-Type is not application/offset+octet-stream.

All PATCH requests MUST use Content-Type: application/offset+octet-stream, otherwise the server SHOULD return a 415 Unsupported Media Type status.

func (c *Controller) ResumeUpload() http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        // ... the rest of the code
        contentType := r.Header.Get(ContentTypeHeader)
        if contentType != "application/offset+octet-stream" {            
            writeError(w, http.StatusUnsupportedMediaType, 
                errors.New("invalid Content-Type header: expected application/offset+octet-stream"))
            return
        }
        // ... the rest of the code
    }
}

Easy, right? But, any of you wondering why do we should use application/offset+octet-stream? Why not application/json?

The reason is that the application/offset+octet-stream is a media type that is used to indicate that the content is an octet stream (8-bit bytes), which represents a binary data stream that can support random access. Specifically, it is often used in protocols or APIs that involve transmitting data in chunks and may require specifying an offset, like in scenarios involving file uploads, resuming uploads, or downloading partial content. On the other hand, application/json is a media type that is used to indicate that the content is a JSON object, which is a text-based data interchange format that is commonly used in APIs to represent structured data. In resumable upload case, the data is not structured data at all, it is just stream of bytes.

Checking resource availability

The next requirement is to check if the resource exists. If the resource does not exist, we should return 404 Not Found. Easy!

If the server receives a PATCH request against a non-existent resource it SHOULD return a 404 Not Found status.

In this case, we will need to check if the file metadata exists in the storage. If it doesn’t exist, we should return 404 Not Found.

func (c *Controller) ResumeUpload() http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        // ... the rest of the code
        fm, ok, err := c.store.Find(fileID)
        if !ok {
            writeError(w, http.StatusNotFound, errors.New("file not found"))
            return
        }
        if err != nil {
            writeError(w, http.StatusInternalServerError, err)
            return
        }
        // ... the rest of the code
    }
}

To differentiate whether the resource exists, I simply return boolean to indicate whether the resource exists or not. If the resource exists, I will return the metadata of the file. If the resource does not exist, I will return false and zero value for the metadata. If there is an error when fetching the metadata, I will return the internal server error status code.

Checking the offset

The next check we need to do is to check the Upload-Offset header. As defined in the requirements:

The Upload-Offset header’s value MUST be equal to the current offset of the resource. If the offsets do not match, the Server MUST respond with the 409 Conflict status without modifying the upload resource.

We will be storing the current offset to fm.UploadedSize whenever we received data from the client, no matter whether the request is completed or interrupted. Hence, whenever a new patch request is received, we can compare the Upload-Offset header with the fm.UploadedSize. If they are not the same, we should return 409 Conflict. Additionaly, for sanity check, we should also check if the Upload-Offset header is a valid number. Otherwise, we should return 400 Bad Request.

func (c *Controller) ResumeUpload() http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        // ... the rest of the code
        uploadOffset := r.Header.Get(UploadOffsetHeader)
        offset, err := strconv.ParseUint(uploadOffset, 10, 64)
        if err != nil {
            writeError(w, http.StatusBadRequest, 
                errors.New("invalid Upload-Offset header: not a number"))
            return
        }

        if offset != fm.UploadedSize {
            writeError(w, http.StatusConflict, 
                errors.New("upload-Offset header does not match the current offset"))
            return
		}
        // ... the rest of the code
    }
}

Conclusion

In this lesson, we have implemented the initial checks for the PATCH request. We have checked the Content-Type, the resource availability, and the offset. In the next lesson, we will implement the actual content writing to the file. The fun part! Stay tuned!