Skip to content

Conversation

drbh
Copy link
Collaborator

@drbh drbh commented Aug 7, 2025

This PR includes a subclass of the ModuleType type to mark dynamically loaded kernels as _KernelModuleType

uv run typed-kernels.py
# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "numpy",
#     "torch==2.7.0",
#     "kernels",
# ]
#
# [tool.uv.sources]
# kernels = { git = "https://github.com/huggingface/kernels.git", branch = "type-kernel-module" }
# ///
import torch

from kernels import get_kernel

# Download optimized kernels from the Hugging Face hub
activation = get_kernel("kernels-community/activation")

print(torch)
print(activation)
print(activation.is_kernel)
print(type(activation))

output

Fetching 7 files: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00, 121826.26it/s]
<module 'torch' from '/home/ubuntu/.cache/uv/environments-v2/kern-type2-f896b3bc741a85a9/lib/python3.13/site-packages/torch/__init__.py'>
<kernel_module 'activation_1579b6109768b104' from '/home/ubuntu/.cache/huggingface/hub/models--kernels-community--activation/snapshots/2fafa6a3a38ccb57a1a98419047cf7816ecbc071/build/torch27-cxx11-cu126-x86_64-linux/activation/__init__.py'>
True
<class 'kernel_module'>

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

module = importlib.util.module_from_spec(spec)
if module is None:
raise ImportError(f"Cannot load module {module_name} from spec")
module.__class__ = _KernelModuleType
Copy link
Member

@danieldk danieldk Aug 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I am a bit worried that there is some module graph processing code that directly matches on ModuleType and will break. Maybe it's safer to directly inject the attributes/methods for now without changing the class (we can always do it in the future without additional breakage)?

Maybe it's also worth storing the LayerRepository/LocalRepository besides the module name , since that provides full information about where the module comes from.

We also need to add to the spec that any method_name and is_kernel attributes are overridden and probably document this attributes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants