Fulltext index: Initial version

[kunga]

Make a working prototype of fulltext index (WITH (layout=flat, tokenizer=whitespace))

Internal design doc

Vector index history: azevaykin, MBkkt

#23698

Syntax

ALTER TABLE texts
ADD INDEX index
GLOBAL USING fulltext
ON (text)
WITH (
    layout = "flat"
    tokenizer = "whitespace",
    use_filter_lowercase = true
)

Steps

• Add public api (see Create vector index in SchemeShard #4967)
  • Initial public api for fulltext index #23381
  • Support multiple columns in fulltext index #24439
  • Extend support of multiple columns in fulltext index #25337
• Add scheme objects (see Create vector index in SchemeShard #4967)
  • Add fulltext index feature flag #23701
  • Support fulltext index on SchemeShard #23739
• Add base helpers
  • Add base fulltext helpers #24589
• Support CREATE / ALTER TABLE ADD INDEX fulltext (see ADD Vector index USING syntax #6602)
  • Pass index settings as is
  • Pass index settings as is #24606
  • Handle fulltext index type in YQL
  • Handle fulltext index type in CREATE / ALTER TABLE #25670
  • Test fulltext scheme queries with QueryService #26112
• Add index build on an existing table
  • DataShard fulltext index build scan #25028
  • Build fulltext index on SchemeShard #25811
• Write KQP tests
  • Create table / alter table
  • Select
  • Insert/Update/Delete
  • Test all different combinations of key, text, data, cover columns
• Show fulltext index parameters in UI
• Make simple SELECT statements work
• Make simple INSERT/UPDATE/DELETE statements work
• Add backup support (see https://t.me/c/1952638471/8935)
• Add UTF8 support
• Implement SHOW CREATE TABLE (see create_table_formatter.cpp)