Skip to content

Conversation

TaylanUB
Copy link

@TaylanUB TaylanUB commented Oct 5, 2025

This is a COMPLETELY UNTESTED patch after which I've NOT EVEN TRIED TO COMPILE the module.

I've never worked on Nginx modules before and don't write much C. I'll have to see if I can figure out how to even compile and test this. No idea how long I'll be struggling with that, so I thought I'd already throw the untested patch in here.

EDIT: Seems to work fine; see below.

This should also offer a slight performance boost because it avoids having to
perform an allocation for every single cache entry being walked.
@TaylanUB
Copy link
Author

TaylanUB commented Oct 5, 2025

This appears to be working quite well on my end.

@TaylanUB TaylanUB changed the title [WIP, UNTESTED] Fix memory leak when using partial (wildcard) keys. Fix memory leak when using partial (wildcard) keys. Oct 5, 2025
@TaylanUB
Copy link
Author

TaylanUB commented Oct 5, 2025

This has been running on my high-traffic production server (bg3.wiki) for the past 8 or so hours now, and seems to have completely fixed our memory overuse issue that was requiring periodic reloads of Nginx to avoid system-wide OOMs.

Testing indicates that the wildcard purging works as expected, and there's no error logs or any other indications of a problem.

Purging performance is still not great, since the code opens every single file in the cache (about 30K files in our case) but that's another matter I'm thinking of trying to tackle separately.

Would appreciate a merge of this! :-)

@TaylanUB
Copy link
Author

Meanwhile, here's steps to compile and deploy this yourself if you really need it right now:

  • Clone my fork of this module and check out the commit corresponding to this bug-fix: 960d2370f64061ef9449ff14f163aafb8ad1208c
  • Clone the main nginx repository, and switch to the branch corresponding to the Nginx version you use. For example, if you're on Debian 13, your Nginx version should be 1.26.X so you should check out the stable-1.26 branch.
  • Run nginx -V on the machine on which you use Nginx, to get the full original compilation parameters.
  • In the working directory of the main Nginx repository, run ./auto/configure with those full parameters, and this one added to the end: --add-dynamic-module=../ngx_cache_purge. This assumes that the module is in a directory called ngx_cache_purge, alongside the main nginx source directory, together in the same parent directory.
  • Run make modules. This should produce the file: objs/ngx_http_cache_purge_module.so
  • On the machine running Nginx, either overwrite the original .so directly, or uninstall it and put the new .so into some other place from which Nginx can load it.
  • To avoid having to restart Nginx entirely: Disable the module in your Nginx config (and any configuration relying on it), reload Nginx, then enable the module again and reload Nginx again. This will make it close and re-open the .so so it actually starts using the new version.

Example shell session:

This is an example; do not copy-paste commands you don't understand, or imitate exactly what I did, because your situation may be different. You could break your Nginx installation!

$ mkdir -p ~/src/nginx
$ cd ~/src/nginx
$ git clone git@github.com:TaylanUB/ngx_cache_purge.git
$ cd ngx_cache_purge
$ git checkout 960d2370f64061ef9449ff14f163aafb8ad1208c
$ cd ..
$ git clone git@github.com:nginx/nginx.git
$ cd nginx
$ ssh myserver 'nginx -V'
nginx version: nginx/1.26.3
built with OpenSSL 3.5.1 1 Jul 2025
TLS SNI support enabled
configure arguments: {{CONFIGURE_ARGUMENTS}}
$ git checkout stable-1.26
$ ./auto/configure {{CONFIGURE_ARGUMENTS}} --add-dynamic-module=../ngx_cache_purge
$ make modules
$ scp objs/ngx_http_cache_purge_module.so myserver:
$ ssh myserver
server # mods=/usr/local/lib/nginx/modules/
server # mkdir -p "$mods"
server # mv ngx_http_cache_purge_module.so "$mods"
server # cd /etc/nginx
server # rm sites-enabled/mwfc-purge # vhost that handles PURGE
server # apt-get autoremove --purge libnginx-mod-http-cache-purge
server # systemctl reload nginx
server # echo "load_module $mods/ngx_http_cache_purge_module.so;" > modules-available/http-cache-purge.conf
server # ln -s ../modules-available/http-cache-purge.conf modules-enabled/
server # ln -s ../sites-available/mwfc-purge sites-enabled/
server # systemctl reload nginx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant