[8.2] [MOD-11650] Fix Out-of-Bounds Write in Vector Preprocessing #794
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* rename dataSize->getStoredDataSize * missing rename * format * add input blob size to AbstractIndexInitParams in VecSimIndexAbstract move data members to private when possible DONT FIX LEAK YET Factories: move NewAbstractInitParams to a general location (new file factory_utils) replace it in all factories * try sanitizer * run sanitizer * runanyway * rervt task unit test use sanitizer in pull request * fix leak in input bob size * fix int8/uint8 elementSizeEstimation tests This caused a buffer overflow because setup<TieredIndexParams> didn't set the data type to INT8, creating a float32 index instead. DataBlock::addElement() tries to copy dim * sizeof(float) bytes, but the allocated buffer is only dim * sizeof(int8) bytes, causing a read overflow. * run codecov with sanitizer (also intek) * some fixes and assertion * fix uint8 * fix again * fix possible warning abour divison by zero by checking quantBits with constexpr * TO REVERT! fix leaks that will be moved to a separate PR it was failing only with codecov becuase only there we use FP64_TESTS=1 Prevent template deduction errors in GenerateAndAddVector by making data_t parameter non-deducible Used std::type_identity<data_t>::type for the value parameter to force explicit template specification (e.g., GenerateAndAddVector<double>()) instead of allowing compiler to incorrectly deduce int from literal values, which caused buffer overflows when index expected different data types. * Revert "TO REVERT!" This reverts commit af844ec. * rever ci changes * calculate EstimateElementSize accroding to the stored vector size add tests * revert unrelated change in cmake.san * add batch itertor blob correctness to int8 tests fix getQueryBlob in tiered add getHNSWIterator to tiered batch itertor if its BUILD_TESTS * apply suggesting (cherry picked from commit 9fb223a)
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 8.2 #794 +/- ##
==========================================
- Coverage 96.67% 96.67% -0.01%
==========================================
Files 125 126 +1
Lines 7680 7663 -17
==========================================
- Hits 7425 7408 -17
Misses 255 255 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
GuyAv46
changed the title
GuyAv46
approved these changes
Oct 9, 2025
ofiryanai
approved these changes
Oct 9, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
backport #784 to 8.2