Sitelet https://github.com/feast-dev/feast/pull/6765
Skip to content

fix: Feature freshness logic updated - #6765

Merged
ntkathole merged 2 commits into
feast-dev:masterfrom
jyejare:fix_freshness
Aug 21, 2026
Merged

fix: Feature freshness logic updated#6765
ntkathole merged 2 commits into
feast-dev:masterfrom
jyejare:fix_freshness

Conversation

@jyejare

@jyejare jyejare commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

What this PR does / why we need it:

The monitoring UI Freshness column previously used metric_date (the DQM computation window start) as a proxy for data freshness. This was misleading — metric_date reflects when metrics were computed, not how fresh the underlying data is.

This PR introduces a new max_event_timestamp field that stores MAX(event_timestamp) from the actual data source. The UI now displays age relative to the newest source event, giving an accurate picture of data staleness.

Changes

Backend — monitoring_service.py

  • Added _as_utc_datetime() helper to normalize timestamp-like values to timezone-aware UTC datetimes.
  • Added _newest_event_in_window() to clamp the source's max timestamp to the computation window.
  • All metric save paths (_save_computed_metrics, _save_log_metrics) now propagate max_event_timestamp.
  • Feature service metrics aggregate max_event_timestamp across contributing feature views.

Backend — monitoring_utils.py

  • Added MONITORING_TIMESTAMP_FIELDS constant to DRY-up timestamp field lists across offline stores.
  • Added max_event_timestamp to all three column definition lists (feature, feature_view, feature_service).
  • normalize_monitoring_row now handles pandas NaT/NaN for the new field.

Schema — all offline stores (BigQuery, Snowflake, Redshift, Postgres, Spark, Oracle, Dask, DuckDB)

  • Added max_event_timestamp column to CREATE TABLE DDLs for all three monitoring tables.
  • Added ALTER TABLE … ADD COLUMN migration for existing tables.

UI — FeatureMetricsTable.tsx / useMonitoringApi.ts

  • Freshness column now uses max_event_timestamp with a fallback to metric_date.
  • Fixed render function signature ((_val, item)) so the full FeatureMetric object is passed to the freshness logic.
  • Added Number.isNaN guards for robustness.
  • Updated tooltip to show "Newest event" instead of "Data from".

Docs

  • Updated feature-monitoring.md with max_event_timestamp in the example response and clarified the freshness semantics.

Which issue(s) this PR fixes:

Fixes inaccurate freshness display in the monitoring UI — the column previously showed metric_date (computation window) instead of actual data recency.

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests

Tests added

  • tests/unit/monitoring/test_feature_freshness.py — unit tests for _newest_event_in_window (inside window, clamped, before window) and end-to-end auto_compute verification that max_event_timestamp is persisted.
  • tests/integration/monitoring/test_monitoring_integration.py — extended test_auto_compute_uses_pushdown_for_max_timestamp to verify max_event_timestamp in saved metrics.
  • Manual — verified on a live cluster with Dask offline store that freshness correctly reflects MAX(event_timestamp) from source parquet files.

Misc

The new column is nullable and added via ADD COLUMN IF NOT EXISTS migrations, so this is backward-compatible with existing monitoring tables.

@jyejare
jyejare requested review from a team and sudohainguyen as code owners August 20, 2026 19:24
@jyejare jyejare changed the title fix: feature freshness logic updated fix: Feature freshness logic updated Aug 20, 2026
@codecov-commenter

codecov-commenter commented Aug 20, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 20.93023% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 47.09%. Comparing base (03dac6f) to head (68afd8e).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
sdk/python/feast/monitoring/monitoring_service.py 40.00% 10 Missing and 2 partials ⚠️
sdk/python/feast/monitoring/monitoring_utils.py 14.28% 6 Missing ⚠️
...ffline_stores/contrib/spark_offline_store/spark.py 0.00% 5 Missing ⚠️
sdk/python/feast/infra/offline_stores/bigquery.py 0.00% 3 Missing ⚠️
...line_stores/contrib/oracle_offline_store/oracle.py 0.00% 3 Missing ⚠️
sdk/python/feast/infra/offline_stores/snowflake.py 0.00% 2 Missing ⚠️
..._stores/contrib/postgres_offline_store/postgres.py 0.00% 1 Missing ⚠️
sdk/python/feast/infra/offline_stores/dask.py 0.00% 1 Missing ⚠️
sdk/python/feast/infra/offline_stores/duckdb.py 0.00% 1 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6765      +/-   ##
==========================================
+ Coverage   46.94%   47.09%   +0.14%     
==========================================
  Files         419      419              
  Lines       51839    51878      +39     
  Branches     7515     7525      +10     
==========================================
+ Hits        24338    24430      +92     
+ Misses      25766    25700      -66     
- Partials     1735     1748      +13     
Flag Coverage Δ
go-feature-server 30.58% <ø> (ø)
python-unit 48.40% <20.93%> (+0.15%) ⬆️
Files with missing lines Coverage Δ
sdk/python/feast/infra/offline_stores/redshift.py 39.00% <ø> (ø)
..._stores/contrib/postgres_offline_store/postgres.py 46.15% <0.00%> (-0.13%) ⬇️
sdk/python/feast/infra/offline_stores/dask.py 52.38% <0.00%> (ø)
sdk/python/feast/infra/offline_stores/duckdb.py 37.18% <0.00%> (ø)
sdk/python/feast/infra/offline_stores/snowflake.py 37.52% <0.00%> (-0.15%) ⬇️
sdk/python/feast/infra/offline_stores/bigquery.py 48.80% <0.00%> (-0.16%) ⬇️
...line_stores/contrib/oracle_offline_store/oracle.py 39.38% <0.00%> (-0.37%) ⬇️
...ffline_stores/contrib/spark_offline_store/spark.py 36.00% <0.00%> (-0.26%) ⬇️
sdk/python/feast/monitoring/monitoring_utils.py 48.61% <14.28%> (+0.12%) ⬆️
sdk/python/feast/monitoring/monitoring_service.py 33.18% <40.00%> (+18.70%) ⬆️

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 03dac6f...68afd8e. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread sdk/python/feast/monitoring/monitoring_service.py Outdated
jyejare and others added 2 commits August 21, 2026 21:18
Signed-off-by: Jitendra Yejare <11752425+jyejare@users.noreply.github.com>
…_view

Move the _get_max_timestamp call to compute_metrics and pass the result
into _compute_for_feature_view, eliminating a duplicate data-source query
per feature view.

Signed-off-by: Jitendra Yejare <11752425+jyejare@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ntkathole
ntkathole merged commit f0bc070 into feast-dev:master Aug 21, 2026
21 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants