Creating Vector store and building a RAG pipeline in clickhouse

[sandeep1404]

Dear team,

I have my data table in clickhouse database, and this table is being updated everyday. Now I want to build a RAG pipeline so that I can able to retrieve any user requested data, I have few questions :

1.How to create embeddings for all the columns in my table and store this in vectordatabase in clickhouse, since my data is being updated every day and it has more than 10 billion rows, how huge does the vector store is ?
2.I have checked the documentation that there is an inbuilt method to do the vector search In clickhouse, is that a feasible method.
3. How to build a proper RAG pipeline for my continues updates in the data, I am looking for a solution where whenever my data is updated in clickhouse the embeddings for new data should be created automatically, instead of embedding the complete database.
So far I started working with the notebooks from the GitHub reference for my own data https://github.com/ClickHouse/examples/tree/main/blog-examples/llama-index/hacknernews_app but I cannot able to get the response, it was giving few errors, are there any references or notebooks where I can find the end to end RAG pipeline design using clickhouse DB, with my updated data, can someone help me answering my queries, any help to solve my use case is highly appreciated, Thank you in advance