Add GPUPlugin for building vector indices on GPU #135545

mayya-sharipova · 2025-09-26T14:53:24Z

Add GPUPlugin that build vector indices on GPU.
Currently available for "hnsw" and "int8_hnws" types.

It uses Nvidia cuvs library that should be available on a node.
GPUPlugin is under the feature flag for 9.2 release.

- Created GPUVectorsFormat that write/read of flat vectors - Added a new index_options: gpu for dense_vector field that is under the feature flag

./gradlew ":x-pack:plugin:gpu:yamlRestTest" --tests "org.elasticsearch.xpack.gpu.GPUYamlTestSuiteIT.test {p0=gpu/10_basic/*}"

…ors format.

First save Cagra index to hnswlib format on disk. Read this disk file to serialize to Lucene HNSW format.

This reverts commit ec3330e.

Also remove index_options.type:gpu

Plugins can provide VectorsFormatProvider that provides new KnnVectorsFormat for different VectorIndexTypes. If there formats provided by plugins they are used instead of standard

index.vectors.indexing.use_gpu has 3 options: - auto (null) default: use gpu indexing when available - false: don't use gpu indexing - true: use gpu indexing and if not available, throw an error

…upported

Elasticsearch

…VS resources (#132670)

…ix (#132832) This PR updates cuvs-java dependency to 25.10 (I left 25.08 and updated its verification metadata to the final version for convenience in case we want to go back). It uses CuVSMatrix as a way to transfer data efficiently from GPU memory to the Java heap directly (and then to a Lucene file). I tried to keep changes at a minimum, but some restructuring was necessary (e.g. resource management need to be done at a upper level - we need to keep hold of the resource until we finished reading the CuVSMatrix).

ChrisHegarty · 2025-09-29T13:19:55Z

I can't tell, but when the test-gpu label is added, do we get all the usual CI test + the testing on GPU? Or does it replace existing tests?

ChrisHegarty · 2025-09-29T13:24:29Z

x-pack/plugin/gpu/src/internalClusterTest/java/org/elasticsearch/plugin/gpu/GPUIndexIT.java

+
+    @BeforeClass
+    public static void checkGPUSupport() {
+        assumeTrue("cuvs not supported", GPUSupport.isSupported(false));


Do tests need to check the feature flag too? I just curious what these do when tested against test-release ?

I agree, we should have a couple of tests that asserts that you cannot create GPU things with the feature flag off.

Added more tests in 543fafe

I've added "test-release" label for the PR. So I guess it will test when FF is disabled?

It is strange when I put "test-release", elasticsearch-ci would not run with a message "Pipeline upload rejected: You can only change the pipeline of a running build". So I've eventually deleted "test-release" label.

For the record, we have such tests in GPUPluginInitializationIT

benwtrent

I think this looks good!

I didn't review the native integrations, just format stuff.

I would like tests that verify that you cannot utilize the format if es.vectors_indexing_use_gpu_feature_flag_enable isn't set :).

…tings" This reverts commit 304bc68.

mayya-sharipova · 2025-09-30T01:27:07Z

@elasticsearchmachine test this please

mayya-sharipova · 2025-09-30T01:50:17Z

@elasticsearchmachine test this please

ChrisHegarty · 2025-09-30T08:23:15Z

...k/plugin/gpu/src/main/java/org/elasticsearch/xpack/gpu/codec/ES92GpuHnswSQVectorsFormat.java

+ * HNSW graph is built on GPU, while scalar quantization and search is performed on CPU.
+ */
+public class ES92GpuHnswSQVectorsFormat extends KnnVectorsFormat {
+    public static final String NAME = "Lucene99HnswVectorsFormat";


Should the name be ES814ScalarQuantizedVectorsFormat ? I'm surprised that tests do not fail with this. Maybe I'm wrong, no?

ES814ScalarQuantizedVectorsFormat is a flat format that is used inside it, but by itself it is still Lucene99HnswVectorsFormat.

Lucene99HnswVectorsFormat doesn't concern itself how flat format stores data.

Oh, ok then. I honestly confuse myself about these names all the time! Sorry for the noise.

x-pack/plugin/gpu/src/main/java/org/elasticsearch/xpack/gpu/GPUPlugin.java

ChrisHegarty · 2025-09-30T08:43:18Z

@elasticsearchmachine test this please

ChrisHegarty · 2025-09-30T08:44:54Z

I temporarily removed the test-gpu label and replaced it with test-release. I just want to get a CI run with release testing, and we can remove test-release and restore test-gpu.

mayya-sharipova and others added 30 commits June 17, 2025 14:41

Iter1

3af34ab

- Created GPUVectorsFormat that write/read of flat vectors - Added a new index_options: gpu for dense_vector field that is under the feature flag

move GPU plugin to x-pack

d0da0d4

move GPUIndexType to it's own top-level class, and spotless

34b0ce8

Add yml rest test that can be called:

d3db656

./gradlew ":x-pack:plugin:gpu:yamlRestTest" --tests "org.elasticsearch.xpack.gpu.GPUYamlTestSuiteIT.test {p0=gpu/10_basic/*}"

Reload KnnVectorsFormat to allow plugins to declare a custom knn vect…

82f1ca5

…ors format.

Add dependency on cuVS

5207bed

Add a supported check for cuVS - can be used in more tests eventually

d0ee7e1

revert accidental commit

9b6fde2

Small edits

2ac22b3

Build cagra index (iter1)

8cee75e

Build cagra index (iter2)

3f05ba0

Write Cagra index to Lucene HNSW format

2ac9d8b

First save Cagra index to hnswlib format on disk. Read this disk file to serialize to Lucene HNSW format.

Enable GPU indexing test in KnnIndexTester

4826da6

Fix merging in GPU index writer

7c85493

Attempt to use Java 22 DatasetImpl with MemorySegment

ec3330e

Revert "Attempt to use Java 22 DatasetImpl with MemorySegment"

723bb5a

This reverts commit ec3330e.

Merge remote-tracking branch 'upstream/main' into es-gpu

5cb3829

Also remove index_options.type:gpu

Fix spotless

0ee27d9

Use off-heap Dataset when merging vector data

d27b62f

Plugins can provide VectorsFormatProvider

e90fcd7

Plugins can provide VectorsFormatProvider that provides new KnnVectorsFormat for different VectorIndexTypes. If there formats provided by plugins they are used instead of standard

Fix bugs

9f2c96f

Introduce an index level setting: index.vectors.indexing.use_gpu

4390d55

index.vectors.indexing.use_gpu has 3 options: - auto (null) default: use gpu indexing when available - false: don't use gpu indexing - true: use gpu indexing and if not available, throw an error

Merge branch 'main' into es-gpu

949f8af

Fix initialization of DatasetUtilTests to avoid instance check if uns…

8f22e28

…upported

Add unknown value test to IndexSettingsTests for GPU

5386fb8

Add GPU Integration test

7a2cd83

Add more tests

1f742f4

Adjust Cagra Index degree to fit the default performance of

9ad0367

Elasticsearch

Add CuVSResourceManager for controlling parallelism and pooling of cu…

446fba7

…VS resources (#132670)

ChrisHegarty reviewed Sep 29, 2025

View reviewed changes

benwtrent approved these changes Sep 29, 2025

View reviewed changes

mayya-sharipova added 5 commits September 29, 2025 14:00

Merge branch 'main' into es-gpu

b88f2b6

Revert "Introduce feature flag "vectors_indexing_use_gpu" in IndexSet…

3cb4b71

…tings" This reverts commit 304bc68.

Move "index.vectors.indexing.use_gpu" setting under GPUPlugin

2d51f5a

Merge branch 'main' into es-gpu

57d37e4

Add test when FF is disabled

543fafe

mayya-sharipova added the test-release Trigger CI checks against release build label Sep 29, 2025

mayya-sharipova added 2 commits September 29, 2025 17:21

Merge branch 'main' into es-gpu

dba71de

Rename formats to Lucene99HnswVectorsFormat

6d843e1

Merge branch 'main' into es-gpu

b8c190e

mayya-sharipova removed the test-release Trigger CI checks against release build label Sep 30, 2025

mayya-sharipova and others added 2 commits September 29, 2025 21:54

Merge branch 'main' into es-gpu

f221ff5

Update cuvs snapshot version to 25.10.0

e66aca5

ChrisHegarty reviewed Sep 30, 2025

View reviewed changes

Merge branch 'main' into es-gpu

f324889

ChrisHegarty reviewed Sep 30, 2025

View reviewed changes

x-pack/plugin/gpu/src/main/java/org/elasticsearch/xpack/gpu/GPUPlugin.java Outdated Show resolved Hide resolved

ChrisHegarty added test-release Trigger CI checks against release build and removed test-gpu Run tests using a GPU labels Sep 30, 2025

mayya-sharipova added 2 commits September 30, 2025 06:58

Rename feature flag to gpu_vectors_indexing

f926084

Merge branch 'main' into es-gpu

d48d11d

ChrisHegarty approved these changes Sep 30, 2025

View reviewed changes

mayya-sharipova added the test-gpu Run tests using a GPU label Sep 30, 2025

mayya-sharipova merged commit 9faa068 into main Sep 30, 2025
36 checks passed

mayya-sharipova deleted the es-gpu branch September 30, 2025 13:09

Add GPUPlugin for building vector indices on GPU #135545

Add GPUPlugin for building vector indices on GPU #135545

Uh oh!

Conversation

mayya-sharipova commented Sep 26, 2025

Uh oh!

ChrisHegarty commented Sep 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benwtrent left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mayya-sharipova commented Sep 30, 2025

Uh oh!

mayya-sharipova commented Sep 30, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mayya-sharipova Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ChrisHegarty commented Sep 30, 2025

Uh oh!

ChrisHegarty commented Sep 30, 2025

Uh oh!

Uh oh!

Uh oh!

benwtrent left a comment •

edited

Loading

mayya-sharipova Sep 30, 2025 •

edited

Loading