Loading Multiple Indexes from index_store

[systemz64]

Hi
I have test application where I am generating both a tree index and a vector index. I am naming these specifically as follows:

_` tree_index = TreeIndex(
nodes,
storage_context=storage_context,
index_id="tree_index",
show_progress=True # Shows a progress bar during ingestion
)
#tree_index.index_id = "tree_index"
logger.info("Tree Index created/updated in ChromaDB.")

# Then, create a VectorStoreIndex for similarity search
logger.info("Starting ingestion: Creating VectorStoreIndex with semantic chunking via transformations...")
vector_index = VectorStoreIndex(
    nodes,
    storage_context=storage_context,
    index_id="vector_index",
    show_progress=True
)`_

Right now I am using ChromaDB and SimpleIndexStore to test. The index_store.json file shows this:
{"index_store/data": {"3078825e-2aad-47c9-baa1-fdd22c8e8071": {"type": "tree", "data": "{"index_id": "

There does not seem to be a way get the store to use the name I have provided which makes loading the index really hard.
Would anyone have a suggestion around this?
Thx

[dosubot[bot]]

Hi @systemz64! I'm Dosu and I’m helping the llama_index team.

Currently, there is no documented way to enforce that your custom index names (like "tree_index" and "vector_index") are used as the top-level keys in the index_store.json file when using ChromaDB and SimpleIndexStore. The storage mechanism uses a key-value store abstraction, and by default, it appears to generate UUIDs as keys for storing index data, regardless of the index_id you provide when creating the index objects. This is why you see UUIDs instead of your chosen names in the JSON file, which can make manual loading or referencing by name difficult [source].

The index_id you specify is stored inside the index struct, but the key used in the JSON file is not guaranteed to match it. The relevant storage classes (KVIndexStore, SimpleIndexStore) use the index_id as the key when adding index structs, but the actual persistence logic may still generate or use UUIDs as the top-level keys [source].

Workarounds:

• You can maintain an external mapping between your custom names and the UUIDs generated in the index_store.json file. This way, you can look up the correct UUID when you need to load a specific index by name.
• When loading indexes, you can iterate over the stored index structs and check the index_id field inside each struct to find the one matching your desired name.

Summary:
There is no built-in configuration or method to force the use of your custom names as the keys in index_store.json. You will need to implement a mapping or lookup mechanism in your application to associate your names with the stored UUIDs.

If you need more details on how to implement such a lookup or want to see example code, let me know!

To reply, just mention my name @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}