Handling metadata

First, let’s handle how creation api reads the metadata. To recall, here is the sample metadata header we will receive:

Upload-Metadata: filename d29ybGRfZG9taW5hdGlvbl9wbGFuLnBkZg==,content-type YXBwbGljYXRpb24vcGRm,checksum c2hhMjU2OjEyMzQ1Njc4OTA=

Since we decided to store metadata on the header, in this lesson we will implement a function to parse the metadata from the header.

First, let’s define a method named SetCustomMetadata on the File struct.

func (f *File) ParseMetadata(m string) error {
    return nil
}

Upload metadata header

Now from tus.io specification, we have the following requirements for the metadata:

The Upload-Metadata request and response header MUST consist of one or more comma-separated key-value pairs. The key and value MUST be separated by a space. The key MUST NOT contain spaces and commas and MUST NOT be empty. The key SHOULD be ASCII encoded and the value MUST be Base64 encoded. All keys MUST be unique. The value MAY be empty. In these cases, the space, which would normally separate the key and the value, MAY be left out.

We will be storing the metadata in a map. We will split the metadata by comma, then split each key-value pair by space. We will then decode the value from base64 and store it in the map.

func (f *File) ParseMetadata(m string) error {
    md := make(map[string]string)
    kvs := strings.Split(m, ",")
    for _, kv := range kvs {
        if kv == "" {
            continue
        }
        parts := strings.Fields(kv)
        if len(parts) != 2 {
            return errors.New("invalid metadata")
        }
        decoded, err := base64.StdEncoding.DecodeString(parts[1])
        if err != nil {
            return err
        }
        md[parts[0]] = string(decoded)
    }
}

In the above code, instead of using strings.Split(kv, " "), we use strings.Fields(kv) to split the key-value pair. This is because strings.Fields will remove any leading and trailing white spaces before splitting the string. This is important because the value might have leading or trailing white spaces.

Once we have the metadata in the map, we can then assign the value to the File struct.

func (f *File) ParseMetadata(m string) error {
    // ... the previous code
    contentType, ok := md["content-type"]
    if !ok {
        return errors.New("missing content-type")
    }
    checksum, ok := md["checksum"]
    if !ok {
        return errors.New("missing checksum")
    }
    name, ok := md["filename"]
    if !ok {
        return errors.New("missing filename")
    }
    f.Name = name
    f.ContentType = contentType
    f.Checksum = checksum
    return nil
}

Let’s go back to the Controller and implement the CreateUpload handler to parse the metadata.

const (
    // ... the rest of the constants
	UploadMetadataHeader    = "Upload-Metadata"
)

func (c *Controller) CreateUpload() http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        // ... the rest of the code        
        fm := NewFile()
        
        uploadMetadata := r.Header.Get(UploadMetadataHeader)
        err = fm.SetCustomMetadata(uploadMetadata)		
        if err != nil {
            writeError(w, http.StatusBadRequest, err)
            return
        }        
        // ... the rest of the code
    }
}

This is where we create an instance of File which is used to store upload metadata. We initialize the object by using totalSize and uploadMetadata. uploadMetadata value is taken from the Upload-Metadata header of the request and it is then passed to the NewFile function. Later, we save the metadata to the storage.

Another notes from the tus.io specification:

Since metadata can contain arbitrary binary values, Servers SHOULD carefully validate metadata values or sanitize them before using them as header values to avoid header smuggling.

We will not cover this in this lesson, but it is important to note that we should validate the metadata before using it.

We will discuss specification about totalSize in the next lesson.