fix: Feature freshness logic updated - #6765
Merged
Merged
Conversation
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6765 +/- ##
==========================================
+ Coverage 46.94% 47.09% +0.14%
==========================================
Files 419 419
Lines 51839 51878 +39
Branches 7515 7525 +10
==========================================
+ Hits 24338 24430 +92
+ Misses 25766 25700 -66
- Partials 1735 1748 +13
... and 1 file with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
ntkathole
reviewed
Aug 21, 2026
ntkathole
approved these changes
Aug 21, 2026
Signed-off-by: Jitendra Yejare <11752425+jyejare@users.noreply.github.com>
…_view Move the _get_max_timestamp call to compute_metrics and pass the result into _compute_for_feature_view, eliminating a duplicate data-source query per feature view. Signed-off-by: Jitendra Yejare <11752425+jyejare@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com>
ntkathole
force-pushed
the
fix_freshness
branch
from
August 21, 2026 15:48
21a5936 to
68afd8e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
The monitoring UI Freshness column previously used
metric_date(the DQM computation window start) as a proxy for data freshness. This was misleading —metric_datereflects when metrics were computed, not how fresh the underlying data is.This PR introduces a new
max_event_timestampfield that storesMAX(event_timestamp)from the actual data source. The UI now displays age relative to the newest source event, giving an accurate picture of data staleness.Changes
Backend —
monitoring_service.py_as_utc_datetime()helper to normalize timestamp-like values to timezone-aware UTC datetimes._newest_event_in_window()to clamp the source's max timestamp to the computation window._save_computed_metrics,_save_log_metrics) now propagatemax_event_timestamp.max_event_timestampacross contributing feature views.Backend —
monitoring_utils.pyMONITORING_TIMESTAMP_FIELDSconstant to DRY-up timestamp field lists across offline stores.max_event_timestampto all three column definition lists (feature, feature_view, feature_service).normalize_monitoring_rownow handles pandas NaT/NaN for the new field.Schema — all offline stores (BigQuery, Snowflake, Redshift, Postgres, Spark, Oracle, Dask, DuckDB)
max_event_timestampcolumn to CREATE TABLE DDLs for all three monitoring tables.ALTER TABLE … ADD COLUMNmigration for existing tables.UI —
FeatureMetricsTable.tsx/useMonitoringApi.tsmax_event_timestampwith a fallback tometric_date.renderfunction signature ((_val, item)) so the fullFeatureMetricobject is passed to the freshness logic.Number.isNaNguards for robustness.Docs
feature-monitoring.mdwithmax_event_timestampin the example response and clarified the freshness semantics.Which issue(s) this PR fixes:
Fixes inaccurate freshness display in the monitoring UI — the column previously showed
metric_date(computation window) instead of actual data recency.Checks
git commit -s)Testing Strategy
Tests added
tests/unit/monitoring/test_feature_freshness.py— unit tests for_newest_event_in_window(inside window, clamped, before window) and end-to-endauto_computeverification thatmax_event_timestampis persisted.tests/integration/monitoring/test_monitoring_integration.py— extendedtest_auto_compute_uses_pushdown_for_max_timestampto verifymax_event_timestampin saved metrics.MAX(event_timestamp)from source parquet files.Misc
The new column is nullable and added via
ADD COLUMN IF NOT EXISTSmigrations, so this is backward-compatible with existing monitoring tables.