Batch CloudFront invalidations to avoid rate limiting

[LawnGnome]

As reported on Zulip, we are starting to see the odd rate limiting failure when invalidating sparse index entries when large numbers of crates are published at the same time.

I suspect we may need some sort of simple batching solution to handle this.

[syphar]

at docs.rs we had to do this from the start, but we're doing wildcard invalidations, where the limit is much lower (15 parallelly running invalidations).

Also we needed to keep track of the running requests since in our case they can take between 4 and 10 minutes to finish, so we're also regularly checking the status of these running requests to know when we can enqueue another one.

I assume you're invalidating single URLs, not wildcards, so your limits & the speed hopefully looks differently.

[Turbo87]

my understanding was that the number of requests doesnt matter so much and what it more relevant is the number of total paths that are being invalidated, so I'm not sure whether batching would actually help. I might be misunderstanding though, so probably worth clarifying with the AWS folks.

another alternative would be to move the sparse index behind the Fastly CDN since they don't appear to have such low rate limits for the invalidations. might be easier in the short term than building a complicated batching solution.

[syphar]

my understanding was that the number of requests doesnt matter so much and what it more relevant is the number of total paths that are being invalidated

yes, that's it.

see https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cloudfront-limits.html#limits-invalidations
or rust-lang/docs.rs#1871

In case of our wildcard invalidations, it doesn't matter if I'm doing 10 in one request or 10 in single requests.

Which is why I have to track in-progress invalidations to see when I have a free invalidation spot again.

This is also a scaling issue pending to happen, when the releases per day are much higher.

I assume we have to migrate to fastly before that.

[walterhpearce]

Is there a compelling reason the index was kept on cloudfront? If there was not, also migrating it to Fastly seems like a win-win.

[jdno]

The index was set up before or maybe just around the same time that we started exploring Fastly, so we didn't consider Fastly in the infrastructure design back then. I don't think there is a technical reason why it's not on Fastly...