chore: Support "offline" source builds of the `feature-server` image by tchughesiv · Pull Request #5193 · feast-dev/feast

tchughesiv · 2025-03-27T15:18:11Z

What this PR does / why we need it:

We should be able to build the feature-server image entirely from source, dependencies and all. This PR almost entirely accomplishes that. With this PR we also add a new limited pip dependency group, among others, which we'll use going forward in our feature-server image builds and for operator e2e tests.

In order to accomplish the offline builds from source w/ pip, we use a tool called cachi2, which pre-fetches the sdists/archives and enables us to mount the fetched sources into an offline build container.

From a linux-x64 machine, the following command will complete an offline build from source -

$ source ./infra/scripts/offline-build.sh

The following pip modules are not yet built from source, many due to maturin related source build issues -

# we'll need to pip install anything that tries to build/use maturin from source
############
# feast deps
pydantic_core==2.27.2
watchfiles==1.0.4
rpds-py==0.23.1
# snowflake dep
cryptography==44.0.2
# aws dep
aiohttp==3.11.14
# psycopg dep
psycopg-binary==3.2.5
# milvus dep
milvus-lite==2.4.12
############

Misc

Completed build from source available here - quay.io/tchughesiv/feature-server:0.47.0-sdist

Signed-off-by: Tommy Hughes <tohughes@redhat.com>

tchughesiv · 2025-04-02T14:03:09Z

@ntkathole i've made the requested changes and am now running another build to ensure nothing breaks. thanks

ntkathole · 2025-04-02T14:09:47Z

@ntkathole i've made the requested changes and am now running another build to ensure nothing breaks. thanks

a small comment, otherwise, Looks good to me !

tchughesiv · 2025-04-02T17:18:03Z

@ntkathole the build was successful with your requested changes

ntkathole

Looks good to me!

franciscojavierarceo

lgtm

franciscojavierarceo · 2025-03-28T10:24:52Z

+        os: [ ubuntu-latest ]
    env:
-      PYTHON: 3.11
+      OS: ${{ matrix.os }}


Why this change?

because i've enabled this job to run against multiple python versions

…east-dev#5193) * support offline source builds for feature-server image Signed-off-by: Tommy Hughes <tohughes@redhat.com> * offline build review changes Signed-off-by: Tommy Hughes <tohughes@redhat.com> --------- Signed-off-by: Tommy Hughes <tohughes@redhat.com> Signed-off-by: Jacob Weinhold <29459386+j-wine@users.noreply.github.com>

The `azure` extra pinned `pymssql<2.3.3`, which resolves to 2.3.2. That release ships no macOS arm64 wheel, so `make install-python-dependencies-dev` fails building pymssql from source on Apple Silicon (feast-dev#5636). The `<2.3.3` cap was added in feast-dev#5193 because pymssql 2.3.3 dropped its Linux wheels, which the offline feature-server image build relies on. pymssql 2.3.6+ restored the Linux wheels and added macOS arm64 wheels, so `>=2.3.6` fixes Apple Silicon dev installs without regressing the offline build. Regenerated the py3.10/3.11/3.12 CI requirement locks (pymssql 2.3.2 -> 2.3.13) and the pixi lock. pymssql has no runtime dependencies, so no other packages change. Fixes feast-dev#5636

The `azure` extra pinned `pymssql<2.3.3`, which resolves to 2.3.2. That release ships no macOS arm64 wheel, so `make install-python-dependencies-dev` fails building pymssql from source on Apple Silicon (#5636). The `<2.3.3` cap was added in #5193 because pymssql 2.3.3 dropped its Linux wheels, which the offline feature-server image build relies on. pymssql 2.3.6+ restored the Linux wheels and added macOS arm64 wheels, so `>=2.3.6` fixes Apple Silicon dev installs without regressing the offline build. Regenerated the py3.10/3.11/3.12 CI requirement locks (pymssql 2.3.2 -> 2.3.13) and the pixi lock. pymssql has no runtime dependencies, so no other packages change. Fixes #5636

The `azure` extra pinned `pymssql<2.3.3`, which resolves to 2.3.2. That release ships no macOS arm64 wheel, so `make install-python-dependencies-dev` fails building pymssql from source on Apple Silicon (feast-dev#5636). The `<2.3.3` cap was added in feast-dev#5193 because pymssql 2.3.3 dropped its Linux wheels, which the offline feature-server image build relies on. pymssql 2.3.6+ restored the Linux wheels and added macOS arm64 wheels, so `>=2.3.6` fixes Apple Silicon dev installs without regressing the offline build. Regenerated the py3.10/3.11/3.12 CI requirement locks (pymssql 2.3.2 -> 2.3.13) and the pixi lock. pymssql has no runtime dependencies, so no other packages change. Fixes feast-dev#5636

# [0.66.0](v0.65.0...v0.66.0) (2026-08-21) ### Bug Fixes * Add connection pre-warming for DynamoDB async client ([89240fa](89240fa)), closes [#6060](#6060) * Add remote registry client extra ([#6697](#6697)) ([b8dfcb0](b8dfcb0)) * Address review feedback on FIPS cipher suite configuration ([4a35fba](4a35fba)) * Allow remote-registry first apply for new projects ([39d408d](39d408d)) * Avoid importing feast.feature_store at mcp_server import time ([ddb2e9a](ddb2e9a)) * Bump pymssql to >=2.3.6 for macOS arm64 wheel support ([181eb35](181eb35)), closes [#5636](#5636) [#5193](#5193) [#5636](#5636) * Call ApplySavedDataset RPC instead of ApplyFeatureService in RemoteRegistry.apply_saved_dataset() ([934d341](934d341)) * Catch missing dbt parser dependency in dbt CLI commands ([#6534](#6534)) ([3c2ae3c](3c2ae3c)) * Default authentication to kubernetes auth ([6a4690a](6a4690a)) * Defer feature-freshness thread to post-fork to avoid Gunicorn deadlock ([#6648](#6648)) ([104ad10](104ad10)), closes [#6647](#6647) * Do not pass undeclared feature view columns to ODFV UDFs ([#6527](#6527)) ([75b9463](75b9463)) * downgrade mcp pin to 1.29.0 and fix CI lockfiles and unit tests ([98e5bca](98e5bca)), closes [#6706](#6706) * Feast apply silently ignoring ttl updates to None or timedelta(0) ([#6709](#6709)) ([97b0f25](97b0f25)), closes [#6703](#6703) * Fix mypy TorchTensor type alias error ([#6712](#6712)) ([34de6fa](34de6fa)), closes [#5563](#5563) * Fixed data source creation form gaps ([5d0f7d6](5d0f7d6)) * Handle parameterized and complex Trino types in type map ([326554d](326554d)) * Isolate default user permissions ([e37adbf](e37adbf)) * Isolate projection join key maps ([d1c709d](d1c709d)) * Map Postgres real to FLOAT instead of DOUBLE ([62db435](62db435)) * Merge shared ODFV source projections in feature resolution ([d269946](d269946)), closes [#6621](#6621) * More exhaustive athena types ([a9aaefc](a9aaefc)) * Normalize SQL registry read_path to the psycopg3 driver like path ([#6644](#6644)) ([996c6ea](996c6ea)), closes [#6643](#6643) * **operator:** add spec.services.onlineStore.disabled to opt out of the online store ([d81d4e3](d81d4e3)), closes [#6586](#6586) * Preinstall DuckDB delta extension for tests ([fd4d49d](fd4d49d)), closes [#6743](#6743) * Preserve event-time ordering within Redis online_write_batch ([40fb788](40fb788)), closes [#5163](#5163) * Prevent mutation of cached feature resolution results ([ea17419](ea17419)) * Remote feastRef FeatureStore fails first apply for a new feastProject ([9affee5](9affee5)) * Remove inert subjectaccessreviews and reorganize RBAC rules ([f771ea4](f771ea4)) * Report single-feature-view spark_application materialization success ([a9219d9](a9219d9)), closes [#6673](#6673) * Reset the global security manager after the permissions fixture ([7667215](7667215)) * Resolve kserve with pip --dry-run instead of installing it ([01da132](01da132)), closes [#6732](#6732) * Resolve write_to_offline_store feature view with a single registry lookup ([a42dc85](a42dc85)), closes [#4235](#4235) * Return False from __eq__ on cross-type comparison ([#6637](#6637)) ([0f149a9](0f149a9)), closes [#6636](#6636) * Reuse IdP-issued client tokens until near expiry ([602d752](602d752)) * Reuse the OIDC JWKS client across requests ([#6683](#6683)) ([a1e6fc2](a1e6fc2)) * Separate CronJob and feature-server ServiceAccounts ([398f643](398f643)) * Serialize UnixTimestamp proto values as raw int64 in remote online store transport ([1e7134f](1e7134f)) * Set FIPS cipher suites before pyarrow.flight import to prevent crash on IBM Power ([979b82a](979b82a)) * Support Entra ID (Azure AD) token claims in OIDC auth ([#6631](#6631)) ([f843c63](f843c63)) * UDF/ODFV source rehydrate (+ Postgres / online cache) ([#6655](#6655)) ([5fd7af7](5fd7af7)) * Updated projects-list.json in order to display newly added projects ([#6657](#6657)) ([3a6a103](3a6a103)) * Use correct image name in multi-arch imagetools push step ([faf85e0](faf85e0)) * Use join keys instead of entity names in ODFV materialization ([#6645](#6645)) ([abffebc](abffebc)), closes [#5965](#5965) * use matching proto class per feature view list in SqliteOnlineStore.plan() ([adb8c1c](adb8c1c)), closes [#6658](#6658) * Widen Athena integer type mapping for unsigned ints ([3425783](3425783)) ### Features * Add ConnectionRef to DataSource for pluggable external credential resolution ([28bde01](28bde01)) * Add Feature Service Create in UI ([0399380](0399380)) * Add hybrid to ValidOfflineStoreDBStorePersistenceTypes for HybridOfflineStore support ([#6707](#6707)) ([310ab51](310ab51)), closes [#6701](#6701) * Add MLflow integration support to Feast operator ([#6611](#6611)) ([52999f1](52999f1)) * Add opt-in filter_by_created_timestamp cutoff to get_historical_features ([#6617](#6617)) ([79b33ce](79b33ce)), closes [#6615](#6615) * Add optional OIDC token audience and issuer verification ([#6670](#6670)) ([ef307c6](ef307c6)) * Add packaged feature repository support to Feast Operator ([8112b1e](8112b1e)), closes [#6598](#6598) * add plan() support to DynamoDBOnlineStore ([51ce982](51ce982)), closes [#6658](#6658) [#6659](#6659) * Added optional namespace/colleciton to datasets ([165fcf2](165fcf2)) * Added SQL registry schema_mode and registry create command ([#6704](#6704)) ([037c4cd](037c4cd)) * Allow users to have protected project on shared registry ([f9923bc](f9923bc)) * Apply Intermediate TLS defaults on API fallback and handle transient errors ([#6587](#6587)) ([43ae993](43ae993)) * **cli:** Updated feast init demo by adding rag template ([#5946](#5946)) ([c8628eb](c8628eb)), closes [#5264](#5264) * Expose the OIDC JWKS tunables through the operator ([#6690](#6690)) ([fef4e78](fef4e78)), closes [#6683](#6683) * Making feast vector store with open ai search api compatible ([#6121](#6121)) ([54da19a](54da19a)) * Multi-arch publish for feast operator image ([b221036](b221036)) * OpenLineage lineage enhancements - full object coverage, richer UI, and API-level sync ([#6719](#6719)) ([120a868](120a868)) * **operator:** Add spec.services.initImage for init container image override ([#6598](#6598)) ([ca355cb](ca355cb)) * Pass optional OIDC audience and issuer through the operator ([#6677](#6677)) ([a13ed7b](a13ed7b)), closes [#6670](#6670) * **server:** Remote Materialization ([#6649](#6649)) ([b7ae488](b7ae488)), closes [#4526](#4526) * Support Lineage configs via operator ([bf1e54a](bf1e54a)) * Updated datasets UI to support grouping ([7ae64ec](7ae64ec))

tchughesiv added the ok-to-test label Mar 27, 2025

tchughesiv changed the title ~~chore: Support building feature-server images from source~~ chore: Support for building feature-server images from source Mar 27, 2025

tchughesiv changed the title ~~chore: Support for building feature-server images from source~~ chore: Support for building the feature-server image from source Mar 27, 2025

tchughesiv force-pushed the noBinary branch 3 times, most recently from 504eed8 to 005db0f Compare March 27, 2025 19:49

tchughesiv marked this pull request as ready for review March 27, 2025 20:24

tchughesiv requested a review from a team as a code owner March 27, 2025 20:24

tchughesiv changed the title ~~chore: Support for building the feature-server image from source~~ chore: Support offline source builds of the feature-server image Mar 27, 2025

tchughesiv changed the title ~~chore: Support offline source builds of the feature-server image~~ chore: Support "offline" source builds of the feature-server image Mar 27, 2025

tchughesiv marked this pull request as draft March 27, 2025 20:31

tchughesiv force-pushed the noBinary branch 3 times, most recently from 4355d95 to f4ccc80 Compare March 27, 2025 22:41

tchughesiv marked this pull request as ready for review March 28, 2025 01:41

tchughesiv requested review from franciscojavierarceo and redhatHameed March 28, 2025 01:42

tchughesiv marked this pull request as draft March 28, 2025 16:14

tchughesiv force-pushed the noBinary branch from b7d51e9 to 250e8ec Compare March 28, 2025 17:05

tchughesiv marked this pull request as ready for review March 28, 2025 20:16

tchughesiv force-pushed the noBinary branch from 250e8ec to 6f6b647 Compare March 30, 2025 14:24

jyejare reviewed Apr 1, 2025

View reviewed changes

Comment thread Makefile Outdated

tchughesiv force-pushed the noBinary branch 5 times, most recently from a2a8f41 to d362522 Compare April 1, 2025 15:56

support offline source builds for feature-server image

fdb664e

Signed-off-by: Tommy Hughes <tohughes@redhat.com>

tchughesiv force-pushed the noBinary branch from d362522 to fdb664e Compare April 1, 2025 16:00

ntkathole reviewed Apr 2, 2025

View reviewed changes

Comment thread sdk/python/feast/infra/feature_servers/multicloud/offline/Dockerfile.builder.arrow Outdated

ntkathole reviewed Apr 2, 2025

View reviewed changes

Comment thread sdk/python/feast/infra/feature_servers/multicloud/offline/Dockerfile.builder.arrow Outdated

ntkathole reviewed Apr 2, 2025

View reviewed changes

Comment thread infra/scripts/offline-build.sh Outdated

offline build review changes

3e33393

Signed-off-by: Tommy Hughes <tohughes@redhat.com>

ntkathole reviewed Apr 2, 2025

View reviewed changes

Comment thread sdk/python/feast/infra/feature_servers/multicloud/Dockerfile.dev

ntkathole approved these changes Apr 2, 2025

View reviewed changes

franciscojavierarceo approved these changes Apr 3, 2025

View reviewed changes

franciscojavierarceo enabled auto-merge (squash) April 3, 2025 12:37

tchughesiv removed the request for review from redhatHameed April 3, 2025 12:40

tchughesiv disabled auto-merge April 3, 2025 12:42

tchughesiv enabled auto-merge (squash) April 3, 2025 12:43

franciscojavierarceo disabled auto-merge April 3, 2025 12:51

franciscojavierarceo merged commit cc1fcad into feast-dev:master Apr 3, 2025

This was referenced Apr 4, 2025

ci: Revert matrix changes to linter GH Action #5218

Merged

fix: Resolve no pq wrapper import issue #5240

Merged

tchughesiv mentioned this pull request Apr 15, 2025

ci: Maturin-related builds from source #5270

Merged

This was referenced May 1, 2025

chore: Offline source image builds, use hermeto generic fetcher for arrow deps #5318

Merged

chore: Add rpms & milvus-lite / ibis binaries to hermeto fetcher for offline builds #5333

Merged

adarshsm mentioned this pull request Jul 30, 2026

fix: Bump pymssql to >=2.3.6 for macOS arm64 wheel support #6668

Merged

4 tasks

Navigation Menu

Uh oh!

chore: Support "offline" source builds of the `feature-server` image - #5193

chore: Support "offline" source builds of the `feature-server` image#5193
franciscojavierarceo merged 2 commits into
feast-dev:masterfrom
tchughesiv:noBinary

tchughesiv commented Mar 27, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tchughesiv commented Apr 2, 2025

Uh oh!

Uh oh!

ntkathole commented Apr 2, 2025

Uh oh!

tchughesiv commented Apr 2, 2025

Uh oh!

ntkathole left a comment

Uh oh!

franciscojavierarceo left a comment

Uh oh!

franciscojavierarceo Mar 28, 2025

Uh oh!

tchughesiv Apr 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

tchughesiv commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Misc

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tchughesiv commented Apr 2, 2025

Uh oh!

Uh oh!

ntkathole commented Apr 2, 2025

Uh oh!

tchughesiv commented Apr 2, 2025

Uh oh!

ntkathole left a comment

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo left a comment

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo Mar 28, 2025

Choose a reason for hiding this comment

Uh oh!

tchughesiv Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tchughesiv commented Mar 27, 2025 •

edited

Loading