GSoC 2025: DX file support for density visualizations

[PardhavMaradani]

Hi @BradyAJohnston and @yuxuanzhuang ,

I want to next get started on the density visualization part outlined in the GSoC 2025 proposal.

MN already has support for a density entity. Today, mrcfile is being used to read in .map files (either plain or compressed) to create the density object with the corresponding surface / wire density style. This is the only format that is currently supported. Support for OpenDX .dx files was prototyped in #438.

MDAnalysis supports exporting density results to a .dx file as described in the Calculating the solvent density around a protein example of the user guide. The other use case for .dx files is what is outlined in #438, where tools like APBS can generate charge densities and export them to a .dx format.

The bmda prototype had support for .dx and .vdb files. Here are a couple of examples (for both the cases mentioned above) that show the UI controls:

The interface allows control of the ISO value (with different possible ranges), separate colors for the positive and negative ISO values (needed for charge densities), ability to slice along a given axis - only +x prototyped there. The prototype considered density as a component - because that is how nglview was adding it for visualization. Because of this, it was tied to the universe - it would get hidden if the universe got hidden, deleted with it, etc - essentially linked to it. This doesn't quite fit in with the MN model because there is a separate entity for densities.

To keep this least disruptive, I was thinking of the following approach:

• Add a new Method type called something like Density (DX) under MN Import tab
• This section allows the import of .dx files and takes the inputs of default iso value, the iso value range, the style type (surface and wire - maybe I'll start with just surface)
• Have separate style nodes like Style Density ISO Surface and Style Density ISO Wire - similar to the existing ones, but with support for the options mentioned above - like the iso value, the +/- colors, slicing along +/- of x,y,z axes etc
• Expose the Density entities in the Molecular Nodes panel of the 3D viewport by showing the single style node configurations - this will work for both current density stuff as well as the new ones
• There is currently no API interface for density entity like what exists for Molecule and what we added for Trajectories as part of GSoC - I will make it similar to Molecule entity

Please let me know if you have any thoughts on the above and if it's ok to get started as described. Thanks

[BradyAJohnston]

This mostly sounds good. I figure for import though the existing import operator could be expanded to just handled .dx and .vdb import as well rather than a separate import, as they will all end up as the same Density entity anyway?

For authoring new style nodes, collaborative node creation is difficult but not impossible. We basically just have to keep a completely linear git history for the nodes file.

[PardhavMaradani]

I figure for import though the existing import operator could be expanded to just handled .dx and .vdb import as well rather than a separate import, as they will all end up as the same Density entity anyway?

Given that there will be new import options (like iso value range, defaults, etc), I thought it'd be cleaner to keep them independent and not break any existing stuff. I also noticed that the import methods to entities is not a direct one-to-one mapping, so thought this would be ok. Please let me know if you think otherwise.

For authoring new style nodes, ...

In the prototype, the density style node was completely created in code - ie., not imported from the data file. Did you want to try something like that or just stick with the node created in the data file?

[BradyAJohnston]

The current density import takes a guess at a good cutoff (iso value) and uses that as an initial display value. As this can always be changed after import, I figure this can just be the same approach for the other density import as well? Sometimes the cutoff isn't known when importing until a user can see / adjust anyway.

For the new styles, if you are happy to build them via code I am happy with that. I might keep them in code or move into the data file later on but both are easier if you initially build them in code.

[PardhavMaradani]

The current density import takes a guess at a good cutoff (iso value) and uses that as an initial display value. As this can always be changed after import, I figure this can just be the same approach for the other density import as well?

Ok, I will use the same import and not add any new options. For the above examples (in the pictures), the solvent density one had a range of -1.5 to 1.5 (float) and the apbs example of 1 to 100 (int) - these were set as the min/max values along with the type, which also impact the step value of how the slider changes. I can keep them all floats with infinite range (like how it is today with Threshold), but note that this might not be very usable. Update: I see that the initial threshold is being set using np.quantile - I will use the same and see how it works in these cases and maybe see if the type could be updated based on the data type as well.

For the new styles, if you are happy to build them via code I am happy with that

Ok, I will set them up in code so it allows greater flexibility.

[PardhavMaradani]

Hi @BradyAJohnston , do we need any backward compatibility for the vdb file support? The latest MN code switched to openvdb since 4.4, so this will only work with blender 4.4+. In the prototype I had a check to use pyopenvdb for 4.x prior and openvdb since 4.4+. Just wanted to check before switching to only use openvdb. Thanks

I later had something like to handle the differences:

    if ovdb:
        vgrid.transform = vdb.createLinearTransform(voxelSize=1)
        vgrid.transform.preScale(np.array(d.delta) * world_scale)
        vgrid.transform.postTranslate(np.array(d.origin) * world_scale)
    else:
        vgrid.transform.scale(np.array(d.delta) * world_scale)
        vgrid.transform.translate(np.array(d.origin) * world_scale)

[BradyAJohnston]

Sorry have been absent for a few days, should get to the PRs over the next couple of days.

Don't worry about targeting backwards compatibility for now. With MN I only target current Blender version so you can just use openvdb.

[PardhavMaradani]

Hi @BradyAJohnston , another question:

MN is currently using mrcfile to read the .map (MRC2014 file format) files.

For DX files, MDAnalysis is using gridDataFormats library (part of MDAnalysis) to write the .dx files. My prototype and your previous exploration also used this library for loading the grids and converting to vdb files. gridDataFormats also supports MRC2014 file format (and also has gzip support for compressed .gz files) as you can see here - it uses mrcfile internally

I verified that the test map file (tests/data/emd_24805.map.gz) can be loaded fine with this library as you can see below:

Would it make sense to move to this library (which is already a dependency of MDAnalysis and will exist) so that we have a uniform support for both map files, dx files and all other supported formats? What do you think?

[BradyAJohnston]

Yep I'm happy with this idea to simplify and improve things.

[PardhavMaradani]

Hi @BradyAJohnston , I added the custom font support as part of the pending changes I already have with annotations:

Here are the changes I plan to create a new PR with:

• Add custom font configuration for text (works for both viewport and renders)
• Add molecule info annotation (basic info)
• Add density info and density grid axes annotations (see image below)
• Separate annotation types per entity (to avoid any name collisions)
• Add vertical spacing support for multiline text
• Tests for new molecule and density annotations

Here is an example of the two new density annotations - this also shows the custom font used for the info (bottom left):

These changes are on top of PR #917 because the annotations need access to the grid object etc. Given that #917 isn't merged yet, should I wait or go ahead with a new PR? There will be merge conflicts because of some changes in common places, hence checking. Thanks

[BradyAJohnston]

Up to you if you'd like to open the PR before merge. I've just left some comments so we can merge soon, sorry for delay on it!

[PardhavMaradani]

Closing this as changes are as part of PRs #916, #917, #918 and #920