GSoC 2025: Annotations support for entities

[PardhavMaradani]

Hi @BradyAJohnston and @yuxuanzhuang ,

I got started on the annotations support. I am planning to do this in such a way that this is common for all entities and not just Trajectories - though it is for Trajectories that I will be adding the basic annotations initially.

Here is the example GUI from the prototype that shows various options that will be supported both through the GUI and API:

Here is the high level approach I have in mind:

I will add a BaseAnnotation class that is completely independent and includes all the blf, gpu and other Blender code (for the drawing and rendering support) - this will include all the basic methods to draw text (labels) and lines anywhere in the 3d and 2d (viewport) space. All our high level annotations can be built using these basic methods. Entities will extend this to provide access to the entity specific details - for example, TrajectoryAnnotation(BaseAnnotation), which provides access to the underlying entity - the universe, blender object, etc.

I will add an AnnotationsManager class that is responsible for the overall management of annotations - including ones we provided and custom ones that users can add (see later). This will include all the blender property management that is needed for both API and GUI, the viewport draw handler, etc. This will provide APIs to add different annotation types, delete annotations, get annotations etc. This will also provide iterable and subscriptable access to the added annotations through the API. I will make a base class that has all this functionality so that it can be reused across different entities.

What shows up in the UI (and through specific API) are the different annotation types that we support. For example, for Trajectories, the prototype supported the following:

• atom_info - basic atom info - name and optional res id, seg id
• bond_angles - bond details between 3 atoms - lengths, angle
• com - center of mass of a selection
• com_distance - distance between two center of masses
• canonical_dihedrals - canonical dihedral details of a given resid

All annotation types will have the following configurable attributes (in addition to the input attributes that are annotation specific) like what is shown in the image above: font color, font size, text alignment, text rotation, offset x/y (viewport), line color, line width, arrow size, pointer length and others that control all aspects of what is drawn.

I also want users to be able to create their own annotation types with ease (and not have to modify MN code) - thereby getting the corresponding API and the GUI support. To allow this, I want to build the above Trajectory annotation types the same way that users would create their custom annotation types. This will allow us to build with extensibility in mind right from the beginning. This is also a bit similar to how MDAnalysis provides a AnalysisBase for creating custom analysis classes.

For example, the atom_info annotation type could look something like:

class AtomInfo(TrajectoryAnnotation):
    annotation_type = "atom_info"

    selection: str
    show_resid: bool
    show_segid: bool

    def __init__(self, selection: str, show_resid: bool = False, show_segid: bool = False):
        super().__init__()
        self.selection = selection
        self.show_resid = show_resid
        self.show_segid = show_segid

    def draw(self):
        atom_group = self.universe.select_atoms(self.selection)
        for atom in atom_group:
            text = atom.name
            if self.show_resid:
                text += "|res " + str(atom.resid) + ", " + atom.resname
            if self.show_segid:
                text += "|seg " + a.segid
            self.draw_text_3d(atom.position, text)

You will already see a similarity to Blender's panels and operators. Any classes derived from TrajectoryAnnotation will get auto-registered using metaclass's __new__ or __init_subclass__ methods with the AnnotationsManager. There will also be a register and unregister for manually controlling the same. The annotation_type class attribute will specify the annotation type - this will also be a value in the dropdown in the GUI. A corresponding API method (here add_atom_info) will be added in the AnnotationManager and it will return the AtomInfo object using a simple factory pattern in the manager class. Python's annotations in this class (selection: str, etc) will be used to create the corresponding input blender properties. These will show up in the GUI along with the rest of the common annotation configurations (like font color, font size, etc). All of these will also be exposed to the API as dynamic attributes linked to the internal blender properties so that both the API and GUI are in sync. The draw method is called during the viewport draw handler to actually draw the annotation in the viewport or in renders.

I did some very quick prototyping of the above approach, but will need to explore a bit more to make sure I'm not missing anything. I will update this discussion if any of this changes.

The actual APIs for the end users would look something like:

ant = trajectory_entity.annotations.add_atom_info(selection="name CA", ... # to add the `atom_info` annotation type
ant.show_resid = ... # annotation specific inputs
ant.font_size = ... # common configuration (across all annotations)
ant.visible = True / False # annotation specific visibility
trajectory_entity.annotations.remove(... # to remove annotation by object or name
trajectory_entity.annotations.get(... # get annotation object by name - same as what is returned during add
trajectory_entity.annotations[... # for subscriptable access to annotations by name
for ant in trajectory_entity.annotations: # for iterable access to annotations
trajectory_entity.annotations.visible = True / False # to control visibility of all annotations

I will ensure that the common code is separated out so that this can be easily extended to other entities like Molecule etc later.

I will update this discussion as I keep making progress.

[BradyAJohnston]

Reading through your descriptions it all looks good! Re-usability of draw code would be very beneficial and it seems like you are on top of it. Looking forward to see what you cook up :)

[PardhavMaradani]

I got the basic skeleton of the API and GUI interface working with the extensible annotation classes.

I want to run it by you guys before I proceed with further changes. I haven't done any of the actual drawing related to annotations yet, but there aren't any unknowns there as of now (based on prototype experience).

Here is a short video that shows the progress so far:

annotations-api-gui-interface.mp4

As you can see from the above, the whole registration process of custom annotation classes (the ones we write as well as the ones that users can write) work fine. The dynamic methods added for annotation types (like add_atom_info) have code completions and can take in user inputs as well. The API interface and GUI are in sync as expected. The video doesn't show this, but the API interface allows all other annotation properties (the common ones like 'text_color`, etc) as well - as shown below:

User created annotations also work fine - as seen in the BondInfo class in the video. It is auto-registered and the drop down for this annotation type in the GUI is also dynamic. The user inputs (specified as python annotations) in the class are the inputs for this new annotation type. Blender does not allow operators with dynamic properties - this is because operators have to be registered and the properties have to be known upfront. To get this working in our case, I had to create temporary dynamic operators with custom properties to get the add annotations working from the GUI. This also works fine.

The initial approach I thought for annotations (which is the same as what was in the prototype) is to use Blender properties to save all of the annotation specific data. Though Blender provides ways to create dynamic properties (which we need to support extensibility), it still seems like a lot of property management that we will have to deal with and wasn't quite straightforward. One other approach I had in mind was to use Geometry Nodes for this - essentially use the nodes themselves to store data as opposed to properties. We already have nice interfaces to expose and manage these - both for API and GUI. So, it seemed natural to re-use those. This is what you might have noticed towards the end of the video above. We could potentially create a new modifier etc, but I thought it'd be easy and maybe beneficial in the future too to use the existing tree. There is just one AnnotationsGroup node that shows up in the main tree, which internally holds all the annotation nodes. This is linked to the join geometry just to avoid it getting deleted during arrange_tree. The internal nodes are also linked to a join geometry similar to style nodes to allow ordered iteration. These nodes should have no impact on the rest of the stuff. They will be iterated in the draw code to draw out the annotations. All of this seems to work pretty well in my testing so far.

Are you guys ok with me going down this path? It might seem unconventional to use geometry nodes for annotations, but maybe it is a good thing and fits in with the overall MN theme of using nodes? With this, a lot of what we have for nodes can be re-used, where as with properties, it will be something new that we will have to figure out (given the dynamic nature we are seeking now) how to expose those and see if there are any other unknowns. Could you please share your thoughts and let me know if I can continue this way?

[BradyAJohnston]

Thanks for putting the initial example together.

Just to be clear about the example, with the nodes that are being added aren't outputting any geometry themselves, just being used for storing data that is then use in the for drawing for the GPU overlay? It might stray too far from the usual geometry nodes UX, but I think we can certainly explore the idea for a bit more first.

Rather than placing them inside their own node group, could you potentially have the nodes at the top level? Then a user could interact the their properties theoretically right, which would then additionally change the overall drawing code? This could be an interesting method for controlling aspects outside of the node group from inside. I'm not 100% that we should be doing this though, so still a bit cautious to go down this approach.

[PardhavMaradani]

Just to be clear about the example, with the nodes that are being added aren't outputting any geometry themselves, just being used for storing data that is then use in the for drawing for the GPU overlay?

Hi Brady, yes, that is correct. These nodes don't output any geometry. They are used for storing the different annotation type data and are evaluated by us in our draw code to read that data and generate the overlays.

It might stray too far from the usual geometry nodes UX, but I think we can certainly explore the idea for a bit more first.

Does this mean I can go ahead with this approach? Could you please confirm?

Let me highlight a few other issues with the regular properties based approach. To use in any UIList, we need a CollectionProperty that is essentially an array of a PropertyGroup. We cannot have different types of property groups within the same collection. In the prototype of earlier annotations the different annotation types were hardcoded, so the property group had all the options relevant for all the annotation types and they were selectively displayed in the UI. This will no longer be the case when we have to support dynamic user generated annotations. The alternative to this would be to have separate collections for each of the annotation type properties (ie., as separate property groups). These cannot be put together in a common way in the UI or exposed through the APIs. I couldn't quite see how this could be made to work - so there are quite a few unknowns left. Even if we figured out a way (which at least for now, seems unlikely), we'd need a consistent way to expose them for the APIs (like what we have with the GeometryNodeInterFace). It is due to all these unknowns I explored the node based approach, which seems ideal for these dynamic scenarios. It is definitely a novel approach to using Geometry Nodes, but we get everything we want to support and also can re-use (and make better) existing code.

Rather than placing them inside their own node group, could you potentially have the nodes at the top level? Then a user could interact the their properties theoretically right, which would then additionally change the overall drawing code?

Yes, these could potentially be placed at the top level - similar to the style nodes. Just as the current style interfaces from tree is returning the list of style nodes (based on the node starting with Style ), an annotations interface could use the annotation nodes. These nodes are not evaluated by Blender, but by us, that's the only difference. Yes, the values in these nodes (which are updated by the UI and API) can also be updated by the users to impact the overall drawing code too. One potential downside to pulling them out is that the overall group specific features (like the visible property for all annotations currently) will be lost. For now, I think we could keep it all tucked in the node group for now and maybe consider exposing them later when we add more GUI in the geometry node n-panel, etc.

[PardhavMaradani]

Does this mean I can go ahead with this approach? Could you please confirm?

Hi Brady, I went ahead with the node based approach after further testing as it seemed to be the most flexible option. I created PR #891 with these changes. These changes are the base supporting changes needed to add the remaining annotations support.

[BradyAJohnston]

I'm commented on the PR, but I think we need to return to the property CollectionProperty approach. As you have said we can't make them dynamically with different properties, but having all possible properties and then selectively displaying those that are relevant for the particular case in the UI seems like a more "blender-like" approach than storing data in nodes. I think this is the approach that should be taken instead of the nodes.

[PardhavMaradani]

Closing this discussion. Discussed changes are in PRs #891, #900, #901, #903, #905, #906, #907, #909 and #910