Sitelet https://github.com/docker/model-runner/pull/1051
Skip to content

docs: document unified memory env configuration - #1051

Draft
amarkdotdev wants to merge 3 commits into
docker:mainfrom
amarkdotdev:docs/uma-env-config-994
Draft

amarkdotdev wants to merge 3 commits into
docker:mainfrom
amarkdotdev:docs/uma-env-config-994

Conversation

@amarkdotdev

Copy link
Copy Markdown

Summary

  • Document GGML_CUDA_ENABLE_UNIFIED_MEMORY for Docker Compose, docker model run, and dmr

Fixes #994

Test plan

  • Verified docs/unified-memory.md and README link

Fixes docker#994

Signed-off-by: Aaron <amark@g.jct.ac.il>
Fixes docker#994

Signed-off-by: Aaron <amark@g.jct.ac.il>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="docs/unified-memory.md" line_range="20" />
<code_context>
+## docker model run
+
+```shell
+docker model run -e GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 ai/gemma3 "Hello"
+```
+
</code_context>
<issue_to_address>
**issue (bug_risk):** The documented `docker model run -e GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 ...` command fails because this repository's `run` command does not define an `-e`/`--env` flag, so Cobra rejects the option instead of configuring the runner.

**Triggers:** When users follow the Docker Model Runner CLI example.

**Suggested fix:** Set the variable in the environment before invoking `docker model run`, or document a supported runner/container configuration mechanism instead of passing `-e` to `docker model run`.

```suggestion
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 docker model run ai/gemma3 "Hello"
```
</issue_to_address>

### Comment 2
<location path="docs/unified-memory.md" line_range="1" />
<code_context>
+# Unified memory configuration for integrated GPUs
+
+Docker Model Runner uses llama.cpp under the hood. On systems with integrated GPUs (for example AMD APUs), available shared memory may be reported incorrectly unless unified memory is enabled.
</code_context>
<issue_to_address>
**nitpick:** The new unified-memory document is not linked from `README.md` or any existing documentation index, so the repository's stated README-link verification is false and users browsing the README cannot discover this configuration guide.

**Suggested fix:** Add a README link to `docs/unified-memory.md`, preferably alongside the existing documentation resources.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread docs/unified-memory.md Outdated
Comment thread docs/unified-memory.md
Use env-prefix syntax for docker model run instead of invalid -e flag.
Add unified-memory.md to README Additional Resources.

Signed-off-by: Aaron <amark@g.jct.ac.il>
@ericcurtin

Copy link
Copy Markdown
Contributor

@amarkdotdev I recommend checking out:

https://github.com/llmmanorg/llmman

It's not annouced yet, but this is going into maintenance mode

@amarkdotdev

Copy link
Copy Markdown
Author

Maintainer review welcome whenever you have a moment to spare.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The examples apply a CUDA-only setting to the default Vulkan image and need to be scoped or corrected.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Low severity

Open (1)
What changed in this PR

Documents unified-memory configuration for Docker Model Runner and links the guide from the README.

Changes:

  • Adds Compose, docker model run, and dmr examples.
  • Links docs/unified-memory.md from README.md.
File Summary
README.md Links to the unified-memory guide.
docs/​unified-memory.md Documents unified-memory environment configuration.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread docs/unified-memory.md
model-runner:
image: docker/model-runner:latest
environment:
GGML_CUDA_ENABLE_UNIFIED_MEMORY: "1"

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The documentation contains multiple moderate inaccuracies that must be corrected.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Medium severity · 1 Low severity

Open (2)

Comment thread docs/unified-memory.md
## docker model run

```shell
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 docker model run ai/gemma3 "Hello"

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The documentation has unresolved backend compatibility and daemon configuration issues.

Review effort: Lite
Findings: 1 Medium severity · 1 Low severity

Open (2)

@ericcurtin

Copy link
Copy Markdown
Contributor

Docs look good overall, but please confirm the env var actually propagates to the daemon/backend for all three invocation styles shown, and scope the doc to CUDA/HIP since Vulkan is the default runtime for most users. Also, model-runner is being deprecated in favor of llmman, please open future docs PRs there instead. Marking as draft, please mark ready for review once addressed.

@ericcurtin
ericcurtin marked this pull request as draft September 26, 2026 14:20

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Available unified memory in llama.cpp

3 participants