In the recent past, we have been experiencing some slowdowns, elevated queue times and error rates. Apart from Route 53, RDS and LAN outages we experienced on EC2, some of these issues were also caused by people sending too many requests at once, for example by importing a large library of videos or images. Today we are fixing that by introducing rate limiting.

Generally speaking, one customer's behavior should not affect anyone else's. Since all our customers still share a single job queue, we cannot guarantee that without using rate limits. We already tackled most problems by introducing priority queues (a customer with X jobs in queue will have their next jobs appended to the queue, while jobs by other customers are prepended) and reducing scale times to 2 minutes per octa-core machines, etc..

But no matter how fast we can scale, there are always limits in our physical world. We experienced large batch imports in the past, where people were importing thousands of videos at once, or tens of thousands of images. This caused our platform to scale up to 100 machines, but Assemblies from other customers could still execute for as long as 90 minutes, which is not acceptable at all.

Besides valid mass imports, buggy integration code can also cause too many requests to be sent in a very short period of time. This can lead to large bills at the end of the month, which the customer did not plan for.

In short, we are dealing with problems by limiting the requests you can send.

For now, we only limit the number of Assemblies that you can create to 250 per minute. We feel that this is more than enough for reasonable usage. We will closely evaluate the impact of this measure and introduce more limits as required.

Should you ever hit the rate limit, you will receive a 413 RATE_LIMIT_REACHED error. The error JSON has a subkey info.retryIn, which tells you the number of seconds you need to wait before you can create your next Assembly. By providing this info, we hope to help our customers with smooth integration and we want to make sure that your import doesn't throttle even for a second longer than necessary.

Remember that request limits are a feature to protect our platform from (mostly invalid) excessive use, ensuring a smoother ride for everybody. We also allow different rate limits on a per customer basis. So if you feel you need a higher rate limit, please create a support ticket and I will happily raise it.