lvector_create_index

Signature:CREATE PROCEDURE lvector_create_index(table_name LVARCHAR, index_type lvarchar, num_partitions INTEGER);

Purpose: Creates or recreates a corresponding LanceDB/lvector index.

Examples:

EXECUTE PROCEDURE lvector_create_index('docs_demo', '', 1);
EXECUTE PROCEDURE lvector_create_index('docs_demo', 'ivf_flat', 1);
EXECUTE PROCEDURE lvector_create_index('docs_demo', 'ivf_pq', 16);
EXECUTE PROCEDURE lvector_create_index('docs_demo', 'ivf_hnsw_pq', 1);
EXECUTE PROCEDURE lvector_create_index('docs_demo', 'ivf_hnsw_sq', 1);
EXECUTE PROCEDURE lvector_create_index('docs_demo', 'btree', 0);
EXECUTE PROCEDURE lvector_create_index('docs_demo', 'bitmap', 0);

Additional details:

Lower num_partition values seem to perform best.Specifying a blank index type ('') will result in a type being automatically chosen by the system.All ivf_* index types are created on the 'lvector' column.All other index types are created on the 'metadata' column.Your mileage may vary with index type choices, but speaking generally:

Ivf_pq is typically used with large numbers of vectors (greater than half a million). Ivf_hnsw* is typically used when accuracy is critical.

Note: Choose your index type based on the number of vectors in your table and your accuracy vs. speed requirements.