Vector indexes

• [fix] Enabled TCP_NODELAY for auto-embedding HTTP connections to reduce latency for requests with small payloads

Core

• [fea] Reworked the UPDATE query engine. The new implementation supports updates with multidimensional arrays and arrays of vectors (for example: UPDATE ns SET vecs_arr = [[1.0, 1.1], [0.0, 0.7], [[0.5, 0.1], [0.3, 1.0]]]). It also fixes 24 issues with updates to array elements by position or wildcard, arrays of objects, and uninitialized indexed fields
• [fea] Added explicit sorting to MERGE queries. MERGE now works with any selection, not just full-text queries
• [fea] Added deduplication to MERGE queries. May break compatibility: automatic deduplication changes the results in some cases
• [fea] Added merged_query_id() for sort expressions. It lets you explicitly prioritize subquery results in MERGE
• [fea] Optimized collision handling for string hash indexes, reducing unnecessary string comparisons
• [fea] Added the items_deletion, items_modification, and wait_replication states to #activitystats for UPDATE/DELETE queries, transactions, and RAFT cluster replication
• [fix] Fixed a possible hang in UnsubscribeUpdates when closing an updates subscription with pending events
• [fix] Fixed event loss and unnecessary queue retention when subscribers join or leave
• [fix] Fixed event delivery after queue overflow. Event-loss notifications no longer repeat indefinitely and are delivered to all subscription streams regardless of their filters
• [fix] Fixed metadata key compatibility for the serial() precept. Both the old key format and the new one introduced after v5.11.0 are now recognized correctly

Fulltext

• [fea] Reduced full-text index memory usage during builds and background cleanup by limiting excess buffer capacity and releasing unused space
• [fea] Reduced peak memory usage for full-text queries by using more compact temporary structures to combine term matches
• [fea] Added indexing_struct_details to #memstats with a more detailed breakdown of text index memory usage. See the full list of fields in the TextIndexMemoryStats schema
• [fix] Improved BM25 ranking accuracy when deleted or updated documents have not yet been cleaned up from the full-text index
• [fix] Fixed highlight/snippet for arrays of certain lengths
• [fix] Fixed phrase search in individual fields of composite text indexes and in arrays

Replication

• [fix] Fixed a possible temporary split-brain in RAFT clusters during overlapping leader elections. A node could start a new election after already accepting another leader
• [fix] Fixed error handling for snapshots during initial RAFT leader synchronization. Previously, this could terminate the process in rare cases

Reindexer server

• [fea] Added IPv6 address support when displaying connected clients in #clientsstats
• [fix] WAL queries are now rejected for CSV, Protobuf, and MsgPack output formats
• [fix] Fixed transaction lifetime handling in gRPC. Active transactions no longer expire prematurely
• [fix] Fixed a possible heap-use-after-free when stopping the RPC server while a client connection is closing

Reindexer tool

• [fea] SQL errors are now displayed when attempting to autocomplete an invalid query
• [fix] Fixed CLI autocompletion for backslash commands, empty input, and commands with leading whitespace

Build

• [fea] CMake now detects and uses ccache and mold when available

Face

• [fea] Changed the script of the "Save" button availability for the forms
• [fea] Added new ft_cleanup_timeout_ms field to the "Namespace settings"
• [fea] Added new deleted_docs_optimization_threshold field to the full-text indexes config
• [fix] Changed some Russian labels in accordance with the documentation

Core

• [fea] Optimized transaction commit. Transactions with CJSON items now commit 10-20% faster
• [fea] The namespace write lock is now released before the storage flush during write operations
• [fea] Reduced PK index memory usage by ~20%
• [fea] Added support for subqueries inside joined queries
• [fix] Fixed Explain timings for queries with MERGE. Previously, JOIN preselect time was calculated incorrectly for such queries
• [fix] Fixed several rare issues with document migration in storage after a PK index change
• [fix] Fixed * and / operator precedence in UPDATE SET expressions
• [fix] Fixed total.data_size calculation for indexes in #memstats. Previously, it could become negative in some cases
• [fix] Combining EQUAL_POSITION with ALLSET conditions is now prohibited (such queries used to work incorrectly)
• [fix] SQL conditions with NULL now follow the Query API rules: <, <=, >, >=, and LIKE with NULL return an error instead of silently working as IS NULL. EMPTY and NOT NULL are now allowed only after IS
• [fix] Fixed a data race on transaction commit while multithreaded vector inserts were still running
• [fix] Text functions (snippet(), highlight(), etc.) now return an error when applied to a field without a full-text index
• [fix] Fixed validation of #lsn conditions inside nested queries (MERGE/JOIN/subqueries)
• [fix] A failed index update (UpdateIndex) no longer drops the original index, even if the error occurs while converting data to the new index type
• [upd] Removed the deprecated replication.data_hash field from #memstats. Use replication.checksum instead

Fulltext

• [fea] Reworked incremental builds for text indexes. Accumulated updates no longer trigger a full blocking index rebuild: documents deleted from the index are now cleaned up in the background instead. Cleanup is controlled by the new deleted_docs_optimization_threshold option in the text index config and by the ft_cleanup_timeout_ms option in the namespaces section of #config
• [fea] Improved handling of queries typed in the wrong keyboard layout
• [fix] Fixed snippets for array indexes where only some of the array elements match the query

Vector indexes

• [fea] Added support for OpenAI-compatible embedding APIs in auto-embedding (protocol.type: "openai")
• [fea] Query-builder and DSL UPDATE results no longer include float vector fields unless they are requested explicitly (SelectAllFields() or vectors()), the same as SELECT. SQL UPDATE and DELETE still return vector fields. See Float vector fields in selection results
• [fix] Improved handling of invalid responses from the embedding service

Replication

• [fea] DELETE queries, and the delete record written when an UPDATE changes the primary key, now store only primary key fields in the WAL and in the replication payload. This cuts WAL size and replication traffic for large documents
• [fea] Namespace meta (PutMeta / DeleteMeta) and WAL records are now written asynchronously and do not interfere with background storage flush
• [fea] Reduced the number of namespace resyncs on intermediate nodes in cascade replication

Reindexer server

• [fea] Added runtime log level management via the HTTP API
• [fea] Added individual log levels for the core, server, http, rpc, and grpc loggers. See logging configuration and RX_*_LOGLEVEL Docker variables
• [fea] Added a separate gRPC request log (grpclog / --grpclog, disabled by default)
• [fea] HTTP/RPC/gRPC request logs now use the error level for failed requests and info for successful ones. Previously, all requests were logged at info
• [fea] Improved memory release after large requests in RPC/HTTP connections and in the C++ client. This also reduces memory footprint for replication Leader with a large number of Followers
• [fix] Fixed handling of relative paths in HTTP requests for static files
• [fix] Fixed a server crash when an invalid path was passed to --config

Reindexer tool

• [fea] Relaxed format requirements for dumps restored via -f/--filename

Go connector

• [fix] ExecSQL and ExecSQLToJSON now take the namespace from UPDATE and TRUNCATE for metrics and tracing. Those statements have no FROM, so the namespace label used to stay empty

Build

• [fea] Added pre-built packages for Ubuntu 26.04 and AltLinux p11
• [upd] Stopped publishing pre-built packages for Ubuntu 22.04 and AltLinux p10

Face

• [fea] Added the openai protocol type to the auto-embedding config on the vector index config page
• [fea] Added per-component runtime log level settings to the Database config page
• [fix] Fixed the replication table layout

Extra credits

We thank Vladimir Babin (@chiliec) for responsibly reporting a security issue that has been addressed in this release.

Core

• [fea] Added support for nested left/inner joins. Warning: For builtin builds, Query/QueryResults serialization was migrated to a new format. Updated bindings are required for builtin mode to work correctly. Full compatibility is preserved for cproto/ucproto
• [fea] Improved heuristics for scan order and sort index selection when using Distinct under mixed read/write workloads
• [fea] Optimized cost/iteration estimation when planning B-tree scans under mixed read/write workloads
• [fix] Fixed #memstats calculation for indexes. Since v5.15.0, idset_plain_size could overflow
• [fix] Fixed a race when cancelling background index optimization. Since v5.15.0, background optimization could finish in an incomplete state and cause exceptions during select queries
• [fix] Fixed a crash on int64 overflow for string indexes with collate_mode: numeric
• [fix] Fixed a crash when a subquery and an inner join were placed in the same parentheses
• [fix] Fixed a crash in forced sort for composite indexes
• [fix] Improved number parsing in UPDATE SET expressions. Combinations like 9ddd are now treated as field names, not numbers

Fulltext

• [fea] Reworked the in-memory storage and incremental build layer for text indexes. The new implementation significantly improves index performance under mixed read/write workloads
• [fea] Removed max_rebuild_steps and max_step_size from the text index config: the new logic is based on the share of deleted documents in the index (rebuild when 50%+ of documents are deleted)
• [fea] Added total_vdocs, removed_vdocs, and vdocs_compactions fields to #memstats for text indexes
• [fea] Improved variant generation for wrong keyboard layout. The new version produces fewer potentially irrelevant variants
• [fea] Improved term splitting (EnableTermsSplit). Letter/digit boundary splitting now works regardless of term length (e.g. season1 produces the season 1 variant)
• [fix] Fixed double penalty application to suffix ranks

Vector indexes

• [fea] Added built-in circuit breaker logic for auto-embedding. Warning: The circuit breaker is enabled by default and may affect previously configured embedders

Replication

• [fix] Fixed a race between proxying logic and RAFT leader elections. Previously, this race could lead to errors like Request was proxied to follower node

Reindexer server

• [fea] Improved SQL suggest generation via /db/{database}/suggest in some cases
• [fea] HTTP endpoint /db/{database}/suggest now returns the position in characters, not bytes
• [fix] Fixed a connection object leak when the client closed the connection immediately without sending any requests (typical for HAProxy keep-alive probes)
• [fix] Fixed error reporting when trying to use protobuf with multidimensional arrays

Reindexer tool

• [fea] Added a progress bar when creating a database dump. See reindexer tool options (--no-progress-meter)
• [fea] Added CLI commands for index management (\index add / \index update / \index drop)
• [fea] The output option now also affects the format of aggregation results (previously always printed as a table)

Go connector

• [fea] Removed panic from the public user API (except for dedicated calls like MustCommit, where panic is part of the contract). Safe alternative versions have been added for specific methods where panic is part of the contract
• [fix] Updated the go-json module. The old version had a race and could panic when working with aggregation results (on the first call)

Face

• [fea] Added Circuit breaker fields to the vector index config page
• [fea] Removed MaxRebuildSteps and MaxStepSize fields from the text index config page
• [fea] Changed the enable_kb_layout field on the text index config page to support new enum values
• [fix] Fixed the Git documentation link

Core

• [fea] Optimized ID-set intersection logic during queries. The new implementation adapts better to different ID distributions within sets
• [fea] Added heap-based intersection for geo-queries with a large number of points. In certain scenarios, this can provide up to a 10x speedup (see the RStarRTreePointDWithin/1% benchmark)
• [fea] Reduced the impact of background index optimization on write operations by increasing the granularity of optimization cancellation checks. Previously, background index optimization could trigger latency spikes for write operations
• [fea] Made stack traces more informative for MUSL builds
• [fea] Slightly optimized indexes memory layout under mixed write/read workloads
• [fix] Fixed a bug with row invalidation in the joined cache
• [fix] Added explicit error in case of fulltext index inside JOIN's ON clause. Previously those queries did not work correctly in the most of the cases

Fulltext

• [fix] Fixed possible crash in long words (256+ bytes) indexing

Vector indexes

• [fea] Added an optional streaming fetch for HNSW indexes. In this case, the query planner will rely on the query's final limit/offset instead of the k value

Replication

• [fea] Added stricter checks for coroutine stack sizes
• [fix] Fixed a rare issue where a RAFT cluster leader received a false error from Ping while another coroutine was handling an exception, causing it to fall back to the candidate role

Go connector

• [fea] Optimized encoding/decoding logic for binary structures related to queries and query results
• [fix] Fixed NextObj and joined-fields oncorrect interactions: previously it could generate panic even on the correct data

Build

• [fea] Added WITH_UBSAN build option
• [fea] Updated the base Docker image to alpine:3.23

Face

• [fea] Changed format for the Keep diacritics option (fulltext indexes)
• [fix] Fixed Replication table column width

Core

• [fea] Improved heuristic for scan order selection when a multi-condition query requires sorting but background index optimization has not run (typical for mixed read/write workloads)
• [fea] Improved planner logic for scan order selection when explicit sorting is not required
• [fea] Replaced hash value mixing logic for composite indexes. The new implementation significantly reduces collision probability on insert into hash-composite indexes (mean insert time reduced by 5-20% depending on composite value composition)
• [fix] Fixed checksum mismatch error when loading data from disk after removing a UUID index

Fulltext

• [fea] Added term splitting (handles missing whitespace). For example, query leonardodicaprio can now match a document with Leonardo Di'Caprio. See EnableTermsSplit and SplitProc in the fulltext index config
• [fix] Fixed search for terms ending with a dot-character (.) — previously this conflicted with incorrect keyboard layout search
• [fix] Fixed EnableTranslit and EnableKbLayout flag behavior — since v5.12.0, transliteration and wrong keyboard layout search could not be disabled by setting these flags to False

Vector indexes

• [fea] Added support for vector arrays in auto-embedding on insert. Auto-embedding can now be used for data chunking on the embedding service side

Reindexer server

• [fea] Added HTTP endpoints POST /api/v1/query/convert/dsl and POST /api/v1/query/convert/sql to convert and prettify SQL/DSL queries
• [fea] HTTP endpoint GET /db/{database}/suggest now returns parsing error details
• [fix] Fixed --max-http-req=0 flag behavior. A zero value now sets the maximum request size to unlimited

Reindexer tool

• [fea] Added dry-run mode to validate a dump file without applying it. More on validation
• [fea] Added progress bar display while applying a dump

Go connector

• [fea] Optimized CJSON encoding/decoding
• [fea] Optimized Query building process

Build/Deploy

• [fea] Added pre-built packages for Fedora 43/44
• [upd] Deprecated Fedora 41/42 pre-built packages

Face

• [fea] Added queued_namespace_syncs column to Statistics table
• [fea] Implemented Backend-side Prettify for Frontend: the new prettify version not only formats the query but also places parentheses according to actual operator precedence
• [fea] Improved error interpretation for SQL editor
• [fea] Added EnableTermsSplit and SplitProc to Fulltext config
• [fix] Fixed default value for UUID field

Core

• [fea] Optimized grouping equal positions comparator
• [fea] Added implicit conversion between scalar values and single-value composites
• [fea] Added extendible hashing into hash-indexes. This slightly increases mean insertion time, but makes it much more stable, allowing to avoid huge latency spikes on resizing
• [fea] Improved estimate cost calculation for equal_positions
• [fix] Disallowed creating a sparse index over JSON fields with type 'object'. Previously this behavior could cause a few critical bugs and inconsistent states, and was mostly unusable. This may break compatibility if you have object fields in your sparse indexes
• [fix] Fixed data migration on PK update, when some of the documents are marked as 'deleted'
• [fix] Fixed support for subqueries on the left side of WhereExpressions

Fulltext

• [fea] Reworked initial term variants generation logic. Now the search engine uses variants from previous steps to generate more variants on the current step. For example, variants received from the typos-handling mechanism will be used for stemming, etc.
• [fea] Changed ranking for multiword synonyms. Now rank divides proportionally between all the new terms
• [fea] Changed logic around base ranking config. Now full match will always have the best rank, and all the other base ranks will be proportionally decreased according to the configured values. This may break compatibility in some cases, because from now on there is no way to make full-match rank lower than any other base rank
• [fea] MinRelevancy parameter in config has been deprecated. Added new MinRank parameter, which covers the full ranking range
• [fea] Deprecated FuzzyTextIndex and related configs were removed completely
• [fix] Fixed crash during IS NOT NULL filtering condition and EnablePreselectBeforeFt index option interaction

Vector indexes

• [fea] Added support for multiple vectors in a single field (i.e. array vector indexes)
• [fix] Fixed crash in dequantizing of empty (null) vectors during quantization config update

Reindexer server

• [fea] Added memory limit for max response size (default value is 1 GB) to avoid unexpected OOMs. Limit may be changed via net.max_http_rsp_size yaml-config option, --max-http-rsp CLI flag or RX_MAX_HTTP_RSP Docker env
• [fea] Optimized tags synchronization for Update/Delete queries (new logic allows reducing response sizes by cutting off CJSON tags dictionary)
• [fea] Updated swagger to v5.32

Go connector

• [fea] Bumped dependency versions and updated min Go version (v1.24.0)
• [fea] Optimized some of the CJSON marshaling logic
• [fea] Updated content returned in IsSortable and Conditions fields of the IndexDescription object. Now it corresponds to the actual index capabilities
• [upd] Updated fulltext config structure according to changes in core fulltext indexes

Face

• [fea] Added vertical resize for the JSON preview window on the Index edit page
• [fea] Added new fields to the Vector index settings

Core

• [fix] Fixed crash during error handling in IndexUpdate when PK is missing
• [fix] Fixed composite values validation in ALLSET operator
• [fix] Fixed composite indexes error handling in equal_position
• [fix] Fixed negative radius validation in DWithin condition for geo index

Vector indexes

• [fix] Fixed disk ANN cache for composite primary keys. Previously it could lead to crash on startup
• [fix] Fixed KNN search with radius for quantized HNSW index

Go connector

• [fix] Fixed type tags handling for empty slices

Core

• [fea] Added now()-function support into WHERE-clause. Now it may be used both in UPDATE SET and WHERE clauses
• [fea] Added flat_array_len()-function into UPDATE SET. Now it may be used both in UPDATE SET and WHERE clauses
• [fea] Added checksum field into #memstats-namespaces as better alternative for datahash
• [fea] Changed grouping logic for equal_position. New syntax/logic has better match with standard json-paths and also supports nested arrays in explicit way
• [fix] Fixed possible memory leak during composite-indexes substitution inside WHERE-clauses (in cases, when int->string convertion was performed before composite substitution)
• [fix] Fixed SQL parsing for queries with combination of or inner join(...) and left join(...)
• [fix] Fixed storage data migration, when Primary key index was changed
• [fix] Fixed 2D points convertion on WHERE-clause (previously it could led to crashes on assertion)
• [fix] Added explicit check for rtree Primary keys. Geo-indexes can not be PK anymore
• [fix] Fixed forced sort errors handling for KNN-queries, when query has LIMIT and OFFSET
• [fix] Fixed UUID->string conversions for nested arrays on UUID-index deletion

Fulltext

• [fea] Added optional terms boost, that allows to set rank multiplier for specific terms

Vector indexes

• [fea] Added 8 bit scalar quantization for HNSW-index. Read more...
• [fea] Added more effective vectorized implementations for L2, IP and cosine metrics.

Replication

• [fea] Added checksum check instead of datahash - checksum implementation has lower collisions rate and higher impact from each document's field
• [fix] Fixed some rare case, when temporary namespace could remain alive after replication error

Reindexer server

• [fea] Changed FilterDef in Query DSL: some of the fields were marked as deprecated and left_expression/right_expression were as more unified alternatives for better functions support and future filtering expressions development

Go connector

• [fea] Added unified WhereExpressions method for better functions support and future filtering expressions development
• [fix] Fixed deserialization crash for queries, where inner join stays before equal_position in brackets

Face

• [fea] Added Explain visualization for queries with MERGE
• [fea] Added Boost for specific fulltext terms into fulltext config tab
• [fix] Fixed up/down buttons for custom field on pagination section
• [fix] Fixed the issue related to the page opening in a new window from left bar
• [fix] Fixed the issue related to the page opening in a new window from Namespace tabs

Fulltext

• [fix] Fixed possible heap-use-after in composite fulltext indexes, created over non-indexed fields
• [fix] Fixed composite fulltext index cache invalidation after UPDATE-queries
• [fix] Fixed deleted docs handling, when selection results exceeding merge_limit and there are multiple build steps in incremental index

Vector indexes

• [fix] Fixed possible buffer overflow in transactions logic in case of multithreading insertion into HNSW