On August 21, 2018, Tavis Ormandy of Google's Project Zero disclosed security issues with the way that ImageMagick delegates work to Ghostscript.

This vunerability has also been published as VU#332928 by the Computer Emergency Response Team. Many operating systems ship a version of ImageMagick, and according to the CERT report, virtually all of them are affected. Transloadit builds its own versions of ImageMagick, but as a security researcher by the nickname of Lawn responsibly disclosed on August 22 at 19:14 UTC+1, Transloadit was also affected.

By uploading a maliciously malformed image, exploiters would be able to run arbitrary commands on our encoding machines.

As soon as the support team realized the scope of this vulnerability, our Security Response Team was informed and started their investigation at 20:24 UTC+1.

We learned that the recommended workaround involved disabling the processing of PS, EPS, PDF, and XPS, which would obviously have severe consequenses for many of our customers.

Our four security engineers continued their investigation and came up with two alternative approaches. After some deliberation, scanning files for this new type of malicious content before handing them off to ImageMagick was decided to be the best solution for our situation.

By 21:50 UTC+1, a patch to this end was written and we started testing it. We resolved an unrelated issue with our CI, preventing deploys, and by 23:33 UTC+1, the patch was in production and confirmed to be working by both our SRT and the independent vulnerability researcher. We were no longer vulnerable to this issue, while being able to continue processing valid PS, EPS, PDF, and XPS files.

While one of our security engineers was writing the patch, the remaining three continued investigating the severity of the issue for Transloadit. By 21:58 UTC+1, it was determined that for Transloadit, this incident can be labeled with severity: low, thanks to earlier efforts combating ImageMagick-related vulnerabilites:

We have finalized and deployed a big update so that our encoding machines (that would be most susceptible to these kind of attacks) now run without any secrets. They can take jobs out of a queue via read-only, and export them via append-only. All other secrets and privileges have been stripped.

In the face of this incident, we're thankful for these earlier efforts and also to Lawn and Sasha Usenko, Head of Security at Coursera, for disclosing this issue with us prompty and responsibly.

If you have any questions, comments or concerns around this issue or the way we handled it, please do get in touch.