New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
all: entirely remove minio support #45311
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
IMPORTANT: Please note this change is NOT active for customers until the
CHANGELOG is updated and clearly states the removal of this for a specific
version of Sourcegraph.
This change entirely removes minio support from our codebase, in favor
of the new Apache-licensed `sourcegraph/blobstore` Docker image:
* All deployment types already no longer use minio, including:
* Single container `sourcegraph/server`
* Docker Compose
* Pure Docker
* Kubernetes deployment types (helm, kustomize, vanilla, etc.)
* AMI/image-based 'one click' deployments and script-install deployments
* Removed the Docker image so we no longer _distribute_ any minio code for
new Sourcegraph versions in the future.
* Removed the ability of Sourcegraph to even configure/use minio in any form.
* Updated third-party license information to reflect the change.
Helps #44254
Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
|
Codenotify: Notifying subscribers in CODENOTIFY files for diff 710ae45...1ba9b22.
|
|
Codenotify: Notifying subscribers in OWNERS files for diff 710ae45...1ba9b22.
|
efritz
reviewed
Dec 7, 2022
| @@ -10,5 +10,5 @@ Zoekt,Apache License 2.0,https://github.com/sourcegraph/zoekt | |||
| Comby,Apache License 2.0,https://github.com/comby-tools/comby/ | |||
| Alpine Linux,GNU General Public License,https://alpinelinux.org/ | |||
| Tini,MIT License,https://github.com/krallin/tini | |||
| Minio,GNU Affero General Public License v3.0,https://min.io/ | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
efritz
approved these changes
Dec 7, 2022
Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
slimsag
added a commit
that referenced
this pull request
Dec 7, 2022
* all: entirely remove minio support
IMPORTANT: Please note this change is NOT active for customers until the
CHANGELOG is updated and clearly states the removal of this for a specific
version of Sourcegraph.
This change entirely removes minio support from our codebase, in favor
of the new Apache-licensed `sourcegraph/blobstore` Docker image:
* All deployment types already no longer use minio, including:
* Single container `sourcegraph/server`
* Docker Compose
* Pure Docker
* Kubernetes deployment types (helm, kustomize, vanilla, etc.)
* AMI/image-based 'one click' deployments and script-install deployments
* Removed the Docker image so we no longer _distribute_ any minio code for
new Sourcegraph versions in the future.
* Removed the ability of Sourcegraph to even configure/use minio in any form.
* Updated third-party license information to reflect the change.
Helps #44254
Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
sashaostrikov
pushed a commit
that referenced
this pull request
Dec 7, 2022
* all: entirely remove minio support
IMPORTANT: Please note this change is NOT active for customers until the
CHANGELOG is updated and clearly states the removal of this for a specific
version of Sourcegraph.
This change entirely removes minio support from our codebase, in favor
of the new Apache-licensed `sourcegraph/blobstore` Docker image:
* All deployment types already no longer use minio, including:
* Single container `sourcegraph/server`
* Docker Compose
* Pure Docker
* Kubernetes deployment types (helm, kustomize, vanilla, etc.)
* AMI/image-based 'one click' deployments and script-install deployments
* Removed the Docker image so we no longer _distribute_ any minio code for
new Sourcegraph versions in the future.
* Removed the ability of Sourcegraph to even configure/use minio in any form.
* Updated third-party license information to reflect the change.
Helps #44254
Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
slimsag
added a commit
that referenced
this pull request
Dec 7, 2022
* all: introduce sourcegraph/blobstore; add option to cmd/server (#44255) * all: add new sg blobstore backend to supersede minio * cmd/server: optionally run sg blobstore backend over minio * lsifuploadstore: add minio back to help text * sg generate * uploadstore: add DeleteObjects / NewListObjectsV2Paginator to s3API interface * uploadstore: regenerate mocks * uploadstore: add gcsAPI.Objects listing API * uploadstore: regenerate mocks * uploadstore: add Store.ExpireObjects stub implementation (thanks eric) * uploadstore: implement ExpireObjects in GCS/S3/Lazy interfaces * worker: integrate lsifuploadstore expirer * uploadstore: update test API calls * uploadstore: add TODO for myself * uploadstore: batch S3 object deletions * uploadstore: use operation logger instead of log15 * cmd/server: use sglog * cmd/server: fix image build * blobstore: build s3proxy from source ourselves; use our base alpine image (#44976) * blobstore: build s3proxy from source ourselves; use our base alpine image * sg.config: remove ineffective arguments * cmd/server: remove minio support (default to blobstore) (#44979) * cmd/server: remove minio support (default to blobstore) * revert both blob store PRs (#44991) * Revert "blobstore: build s3proxy from source ourselves; use our base alpine image (#44976)" This reverts commit c5181d0. * Revert "cmd/server: remove minio support (default to blobstore) (#44979)" This reverts commit e49509f. * server: use sg/log for minio * blobstore: bring changes back (#45014) * bring back blob store PRs (#44991) This reverts commit 5c5ccff. * blobstore: add missing env var * Revert "blobstore: bring changes back (#45014)" (#45024) This reverts commit e03ca0f. * cmd/server: correct usage of blobstore backend (#45029) * Revert "Revert "blobstore: bring changes back (#45014)" (#45024)" This reverts commit afc1201. * cmd/server: correct usage of blobstore backend * cmd/server: correct s3proxy authorization method * cmd/server: fix blobstore port * cmd/server: correct blobstore env vars / fix CI (#45037) cmd/server: correct blobstore env vars * dev: cache blobstore image building in dev (#45065) * {blobstore, uploadstore}: general cleanup (#45042) * blobstore: default S3PROXY_AUTHORIZATION=none * cmd/server: write blobstore env var overrides + default values more cleanly * uploadstore: remove lifecycle configuration in favor of self-managing expiration * blobstore: use port 9000 by default (#45105) All deployments will have blobstore serve on port 9000, so default to this in the Dockerfile so it doesn't need to be configured everywhere. Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com> * make dev blobstore data/log dirs writable (#45124) For me, the ~/.sourcegraph-dev/data/blobstore directory was chmod 775, which meant that the blobstore Docker container could not write to it. This caused errors: - `failed to create bucket: operation error S3: CreateBucket, https response error StatusCode: 403` from the precise-code-intel-worker service - `sendSimpleErrorResponse: 403 AccessDenied Forbidden` in the blobstore Docker container logs Making these directories world-writable fixes the problem. * fix s3proxy (blobstore) port in sg config (#45117) The blobstore Docker image specifies a container listen port of 9000 (`S3PROXY_ENDPOINT="http://0.0.0.0:9000"`), but this `docker run --publish` flag in sg.config.yaml points to a container port of 80. This commit updates sg.config.yaml to use the correct port (9000). * blobstore: fix /data file permissions (#45226) As Quinn noted in an earlier PR, if `/data` is not writable by the container then we encounter these errors: - `failed to create bucket: operation error S3: CreateBucket, https response error StatusCode: 403` from the precise-code-intel-worker service - `sendSimpleErrorResponse: 403 AccessDenied Forbidden` in the blobstore Docker container logs Docker inherits the permissions of the folder if it is created in the Dockerfile and I forgot to include the creation of the `/data` folder previously. Fixing this means we don't e.g. have to manually correct permissions in Docker Compose or pure-docker deployment types. * all: entirely remove minio support (#45311) * all: entirely remove minio support IMPORTANT: Please note this change is NOT active for customers until the CHANGELOG is updated and clearly states the removal of this for a specific version of Sourcegraph. This change entirely removes minio support from our codebase, in favor of the new Apache-licensed `sourcegraph/blobstore` Docker image: * All deployment types already no longer use minio, including: * Single container `sourcegraph/server` * Docker Compose * Pure Docker * Kubernetes deployment types (helm, kustomize, vanilla, etc.) * AMI/image-based 'one click' deployments and script-install deployments * Removed the Docker image so we no longer _distribute_ any minio code for new Sourcegraph versions in the future. * Removed the ability of Sourcegraph to even configure/use minio in any form. * Updated third-party license information to reflect the change. Helps #44254 Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com> Co-authored-by: Keegan Carruthers-Smith <keegan.csmith@gmail.com> Co-authored-by: Kalan <51868853+kalanchan@users.noreply.github.com> Co-authored-by: Noah S-C <noah@sourcegraph.com> Co-authored-by: Quinn Slack <quinn@slack.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
IMPORTANT: Please note this change is NOT active for customers until the CHANGELOG is updated and clearly states the removal of this for a specific version of Sourcegraph.
This change entirely removes minio support from our codebase, in favor of the new Apache-licensed
sourcegraph/blobstoreDocker image:sourcegraph/server* Docker Compose * Pure Docker * Kubernetes deployment types (helm, kustomize, vanilla, etc.) * AMI/image-based 'one click' deployments and script-install deploymentsHelps #44254
Signed-off-by: Stephen Gutekanst stephen@sourcegraph.com
Test plan
Existing tests / CI