flow tests of tiered with dbpedia to run bm (including flat buffer limit) #366
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Base automatically changed from
tiered_index_enable_sync_mode
to
feature_HNSW_tiered_index
May 7, 2023 09:38
* small modification to bf batch iterator * remove promise of perfect score in HNSW multi batch * implement batch iterator for tiered and some needed helpers * fix for merge results * make the iterator a nested class, fix and modify logic * added first unit test * some fixes and more tests * another test * first overlapping vector tests * fix a bug on reallocation * added an edge cases test * added comments * added `BY_SCORE_THEN_ID` order and sorter * make BF batch iterator use it in select-base search * modification to the BI to handle resize while alive, and use BY_SCORE_THEN_ID * added dynamic parallel test * move iterator from generic vec_sim_tiered to hnsw_tiered * leak fixes * fix clang build * minor test refactor * review fixes * decrease index size * move some array logic to arr_cpp.h * after rebase fixes * review fixes
* implemented `rangeQuery` for VecSimTieredIndex, ... including needed utility functions * renaming `merge_results.h` and moving `filter_results` to it * fix build * first test and some fixes * improved test and added a parallel test * fix a bug where we safely get (from `safeGetEntryPoint`) the old entry point but then we get the new max level when trying to search using the old one * fix tests * Update comments * review fixes * after rebase fixes * added a general comment on tiered index's guarantees
…s requires changing a bit the logic of updating jobs id after removing a vector from flat buffer and swap ids. (#367)
meiravgri
force-pushed
the
meiravg_bufferlimit_bm
branch
from
1d73f9f to
8adef94
Compare
…on to python bindings
…meiravg_bufferlimit_bm
added marked deleted pybind max cpu = 8 taking alon's fixes for th on the pending swap jobs
added memory prints to hnsw resize
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe the changes in the pull request
A clear and concise description of what the PR is solving.
Which issues this PR fixes
Main objects this PR modified
Mark if applicable