Skip to content

Conversation

@mak
Copy link
Contributor

@mak mak commented Aug 25, 2025

This is a proof of concept of using refcount gc rather than mark&sweep

idea here is twofold,
first we use redis EXPIRE to automatically remove tasks, this save as gc a trip and need for listing everything
second we maintain a set of used objects in redis with counter, once the counter goes to zero we can safely remove object from s3

refcount is incremented every time task is created and decremented when task finishes

This is untested and meant to be an discussion opener rather than complete solution ;]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant