[core] Validate the timestamp field type for merge_map_with_keytime - #10367
Open
jackylee-ch wants to merge 1 commit into
Open
jackylee-ch wants to merge 1 commit into
jackylee-ch wants to merge 1 commit into
Conversation
FieldMergeMapWithKeyTimeAggFactory validated the MAP/ROW shape and that the configured timestamp field exists, but never its type. The merge path reads it with InternalRow#getString and compares lexicographically, so a non-string timestamp field -- declared BIGINT/TIMESTAMP/INT, which is natural for a "ts" field -- was accepted at DDL and only failed at the first merge/compaction with a ClassCastException that names neither the field nor the function. The aggregation docs already mandate a string timestamp field. Validate it is CHAR/VARCHAR at factory creation, matching the type checks the other aggregate factories already perform.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
FieldMergeMapWithKeyTimeAggFactoryvalidates that the aggregated field is aMAPwhose value is a
ROWwith at least two fields, and that the configured timestamp fieldexists — but it never checks that field's type. The merge path
(
FieldMergeMapWithKeyTimeAgg) reads the timestamp withInternalRow#getStringandcompares it lexicographically, so a non-string timestamp field — declared
BIGINT/TIMESTAMP/INT, which is a natural choice for a "ts" field — is accepted at DDL andonly fails at the first merge/compaction with a
ClassCastExceptionthat names neitherthe field nor the function.
The aggregation docs already mandate a string timestamp field. This validates it is
CHAR/VARCHARat factory creation, matching the type checks the other aggregatefactories (sum, product, …) already perform, so the misconfiguration fails fast with a
clear message.
Tests
FieldAggregatorTest#testFieldMergeMapWithKeyTimeAggFactoryRejectsNonStringTsFieldasserts a
BIGINTtimestamp field is rejected;…AcceptsStringTsFieldasserts a stringfield still builds.
API and Format
No change.
Documentation
No change.