Open
Description
I want to use map_batches update the database obtained by langchain FAISS, but I can not get correct answer.
Is it because the distributed approach is not suitable for this kind of update ?
from langchain.vectorstores import FAISS
db = FAISS.from_texts(["start"],embedding_model)
def update_db(batch):
global db
db.add_texts(batch['text'])
# log the db
print(len(db.docstore._dict)) # 2
return {}
demo_data.map_batches(
update_db,
batch_size=10,
compute=ActorPoolStrategy(size=1)).count()
print("-----")
print(len(db.docstore._dict)) # 1
Metadata
Metadata
Assignees
Labels
No labels