Text index metadata path passes the Distributed table name to mergeTreeTextIndex → "There is no index with name ..." (BAD_ARGUMENTS) #2880

Summary

When a source points at a Distributed table, all metadata queries that use the text-index fast path fail. Index discovery correctly resolves Distributed → local table, but index reads pass the Distributed name to mergeTreeTextIndex(...), which only accepts a MergeTree table.

Version: HyperDX 2.32.0 (also present in 2.34.0 and main), ClickHouse 26.7.3.19, multi-shard cluster.

What happens

Source = observability.otel_traces (Distributed → observability.otel_traces_local, which has INDEX idx_span_attr_items SpanAttributeItems TYPE text(tokenizer = 'array')).

HyperDX generates:

SELECT splitByString('=', token)[1] AS key
FROM mergeTreeTextIndex('observability', 'otel_traces', 'idx_span_attr_items')
WHERE part_name IN (SELECT name FROM system.parts
                    WHERE database = 'observability' AND table = 'otel_traces' AND active = 1 AND ...)
GROUP BY key HAVING key != '' LIMIT 1000

Code: 36. DB::Exception: There is no index with name 'idx_span_attr_items'. (BAD_ARGUMENTS)

Root cause

In packages/common-utils/src/core/metadata.ts, two places keep the Distributed table name:

1. mergeTreeTextIndex(databaseName, tableName, index) at L708, L744, L1204, L1281 — while getSkipIndices already resolves the local table + cluster via getDistributedTableArgs (feat: Support fetching table metadata for Distributed tables #1920, feat: Support fetching distributed table metadata with cluster() #1944).
2. partsOverlapFilter (L618) filters system.parts by the same name — a Distributed table has no parts there, so even with the index name fixed the predicate would be empty.

Impact

The error is only console.warn-ed, so the UI shows nothing — but there is no fallback for the affected columns:

• getMapKeys returns [] from its catch block (L726) instead of falling through to the MV/scan tiers → no SpanAttributes['…'] keys in autocomplete or the field picker.
• determineKeyValueFetchingStrategy assigns one strategy per column, so columns routed to the text index never reach the existing kv rollup MV → no filter values for those columns.
• Affects logs too (otel_logs is Distributed; otel_logs_local has idx_log_attr_items, idx_res_attr_items, idx_scope_attr_items, idx_trace_id).

Search/filtering still works — the has(SpanAttributeItems, 'key=value') rewrite runs as normal SQL on the shards. Only metadata is broken.

Proposed fix

Use the already-resolved local table and wrap the table function in cluster(), and build partsOverlapFilter on the local table name. Verified on a live 26.7 cluster (returns 277 keys where the current query throws):

SELECT splitByString('=', token)[1] AS key
FROM cluster('{cluster}', mergeTreeTextIndex('observability','otel_traces_local','idx_span_attr_items'))
WHERE part_name IN (SELECT name FROM system.parts
                    WHERE database = 'observability' AND table = 'otel_traces_local' AND active = 1)
GROUP BY key HAVING key != ''

Passing {cluster} as a literal works — querySkipIndices already relies on it.

Separately, the getMapKeys catch blocks should fall through to the MV/scan tiers instead of returning [], so a text-index failure degrades gracefully.