MIME types, security management, and network optimization


#1

Optimization is a bit of a thing I try to look into, and while I may have discovered that GZIP is fantastic at compressing the already well-working KTX format, it did mean having to open a bit of an unintended window.

To better explain, we have to overview MIME types. MIME types are more or less a tag given to a specific file, or namely, it’s media type. JPEGs, for example, are listed as image/jpeg, GIFS as image/gif, MP3 as audio/mpeg, etc. If you have a very common file, chances are it’s going to have a media type.

This is where things get a bit messy. If a server is unsure what a file’s media type is, it will either state the file doesn’t have one or it will report back as application/octet-stream, or basically a binary file. These include applications (exe), disk images (iso/img) and so on. While this normally doesn’t hurt a user experience, since in either case, when you request a file, you get that file, but it can have some effects underneath the surface.

Remember that earlier I said GZIP, a compression method, makes transferring files not only faster but also makes them use less bandwidth? Well, under most conditions, you have to tell it two things:

  1. Which files or file types are okay to compress?
  2. What is the minimal size we should compress?

Most servers typically do not know what a KTX or even a GLTF file is (let alone GLB files, GLTF’s binaries), so they will typically associate them with application/octet-stream, which are typically not included in a default list of files to compress because most binary files cannot really be compressed any further and certain setups can stress a server more than it should have to be if it’s constantly compressing virtually every file type. So while my tests on finding out GZIP is KTX’s best friend, it does mean I had to open a big doorway I wasn’t exactly happy with: allowing application/octet-stream to be compressed.

Thankfully, things are slow and it’s easy to get away with it for now, but later on, such an open situation could result in some issues and isn’t very easy to scale. Despite looking around, I couldn’t find an easy way to simply specify extensions to follow within the settings, and this issue also affects the High Fidelity marketplace, meaning that some optimizations are not being used when they are completely available, but with also a good reason as to why such measures are not in place yet.

So, long story aside, what is the solution?

Upon checking iana.org, I came across some MIME types just for the above-mentioned files! In fact, GLTF and GLB are already proposed and registered (model/gltf+json and model/gltf-binary respectively) in addition to KTX (image/ktx). There also appears to be a general tag for models as well (model/mesh) but any experts can correct me if this is wrong. Thankfully, at least for nginx, it is easy to also add in extra MIME types, so I can stop using application/octet-stream.

So my overall thing for this discussion is if such MIME Types should be used more often, since this could help in aiding everyone in the long run with quicker load times (4.8MB avatar becoming almost 2.7MB) while also preventing possible exploitation (finding additional ways to stress the server) and better organized (KTX files are treated as images and not binaries).