How to use the cudatbx module

[Josephine-Young]

I noticed that there exists a "cudatbx" directory inside the cctbx project, but there isn't much docs content describing it. What are the prerequisites or hardware requirements to run this module? What role does cudatbx play in this project?

[rtviii]

I also want to know this.

[bkpoon]

That code is for development and contains a CUDA implementation for the direct summation calculation of structure factors (and something similar for solution scattering). The module is not distributed in any binaries, so you must build it from source. You will need nvcc already in your path and the GCC version must be 13 or lower because our conda environments are built with GCC 13. You should also have the conda command also available. The following steps were tested on Rocky Linux 9 with CUDA 12.8 and GCC 11.5.

mkdir cctbx-dev
cd cctbx-dev
wget https://raw.githubusercontent.com/cctbx/cctbx_project/master/libtbx/auto_build/bootstrap.py
python bootstrap.py --builder=cctbx --no-boost-src --use-conda --python=311 --config-flags="--enable_cuda" --nproc=64
cd build
source setpaths.sh
libtbx.configure cudatbx
make

And to test (starting from previous commands)

cd ..
mkdir test
cd test
cudatbx.python ../modules/cctbx_project/cudatbx/scattering/tst_direct_summation.py

That test file compares the CUDA implementation with the CPU implementation.

[rtviii]

Thanks. Are you guys develop gpu-accelerated versions of many other modules?

[bkpoon]

No, unless there is an application that could really benefit. Generally, our workflows are not dominated by a single calculation that is rate limiting. For cudatbx, we were simulating some images which was dominated by the structure factor calculation.

But the underlying framework is there. That is, we can call CUDA code from Python using our existing data structures and all the data handling and copying is handled internally.

[phyy-nx]

Thanks. Are you guys develop gpu-accelerated versions of many other modules?

We have a cuda implementation of nanoBragg here:
https://github.com/cctbx/cctbx_project/tree/master/simtbx/nanoBragg

nanoBragg, originally by @jmholton, is used for simulating diffraction patterns, and we used it a lot in these works:
https://pmc.ncbi.nlm.nih.gov/articles/PMC7642780/
https://pubmed.ncbi.nlm.nih.gov/32038048/
https://pmc.ncbi.nlm.nih.gov/articles/PMC7642780/