Repository navigation
[wimp] Turn on multithreading for wimp. - #191747
Conversation
After flutter#190048, we no longer have multthreading issues with wimp, so we should turn multithreading on. This addresses flutter#178749
This ensures we can share resources on the picture-to-image path.
There was a problem hiding this comment.
Code Review
This pull request refactors GetImpellerTexture to accept a ContentContext reference instead of a std::shared_ptr<Context>, updating associated display list code, image decoders, and unit tests. It also addresses an EXIF orientation issue in renderer.dart by cloning VideoFrame to ImageBitmap, and refactors GetRasterThread in surface.cc to avoid function-local static initialization locks under Emscripten. Feedback points out a potential race condition in the new GetRasterThread implementation and flags a const_cast on ContentContext in images_impeller.cc as a design issue.
|
Golden file changes have been found for this pull request. Click here to view and triage (e.g. because this is an intentional change). If you are still iterating on this change and are not ready to resolve the images on the Flutter Gold dashboard, consider marking this PR as a draft pull request above. You will still be able to view image results on the dashboard, commenting will be silenced, and the check will not try to resolve itself until marked ready for review. For more guidance, visit Writing a golden file test for Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. |
There was a problem hiding this comment.
The changes here make sense to me. Could the issue mentioned in the header also be avoided by adding an argument to DisplayListToTexture, something like reset_transients?
There was a problem hiding this comment.
Also, does the precaching have any memory or runtime performance hits?
There was a problem hiding this comment.
Could the issue mentioned in the header also be avoided by adding an argument to DisplayListToTexture, something like
reset_transients
I considered this, and this is one thing Gemini suggested. However, I think this solution is better, because it's more similar to what is done on native. On the native side, when an image is created, a message is kicked off to the raster thread to populate the texture, so it always happens in its own separate render pass. The problem I was running into here was that DisplayListToTexture was happening mid-frame, and the glyph atlas and other stateful bits in impeller were confused by that. We can try to add a reset_transients parameter and hope that fixes the problem and makes impeller work re-entrantly this way, but there might be a bunch of unknown gotchas with this, since this isn't a flow that is exercised in an existing system. I'd rather just make this work in the more hardened/supported way, where we just make sure all the images are available as textures before we start actually drawing the frame.
This has no memory or runtime performance hit, because it's just hoisting work from the middle of the frame to before we start the frame. It's the same amount of work and memory either way.
| if (image) { | ||
| image->asImpellerImage()->GetCachedTexture(context_); | ||
| } |
There was a problem hiding this comment.
In RasterizeImage, you check to see if image->asImpellerImage() is null before grabbing the impeller texture - is it possible for asImpellerImage() to return null here?
There was a problem hiding this comment.
In practice, no. All images that will exist in wimp are impeller images. I'll go ahead and add an assert though.
| final bool needsClone = | ||
| isVideoFrame || !transferOwnership || (isMultiThreaded && !_isTransferable(object)); | ||
| if (needsClone) { | ||
| textureSource = (await createImageBitmap( | ||
| object, | ||
| bounds: (x: 0, y: 0, width: width, height: height), | ||
| )).toJSAnyShallow; | ||
| textureSource = | ||
| (await (isVideoFrame | ||
| ? createImageBitmap(object) | ||
| : createImageBitmap(object, bounds: (x: 0, y: 0, width: width, height: height)))) | ||
| .toJSAnyShallow; |
There was a problem hiding this comment.
This is potentially doing a copy of each video frame, right? Could we possibly try to preallocate a few textures to copy into?
There was a problem hiding this comment.
The ImageBitmap APIs provided by the browser are a little higher-level than that, so there isn't a 1:1 relationship between a createImageBitmap call and an allocation of a texture. Under the hood, the browser does a lot of clever under-the-hood resource pooling, and it's possible that createImageBitmap might actually not create a new texture at all if it's the right format.
walley892
left a comment
There was a problem hiding this comment.
Mostly LGTM, a few comments.
Could you also add a blurb to the description about the image precache?
flutter/flutter@0cbd1a4...70797e1 2026-09-02 bkonyi@google.com [analysis] Upgrade package:analyzer to 14.3.0 and enable custom plugin compilation (flutter/flutter#191590) 2026-09-02 engine-flutter-autoroll@skia.org Roll Dart SDK from d8d7869d8031 to 9a8fbde80f6c (1 revision) (flutter/flutter#192179) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from dec311244db4 to ae22da2e8308 (1 revision) (flutter/flutter#192177) 2026-09-02 bkonyi@google.com [tool] Migrate DoctorCommand to modular dependency injection (flutter/flutter#190758) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from 482b7e285244 to dec311244db4 (3 revisions) (flutter/flutter#192161) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from 28447e9fb8d5 to 482b7e285244 (3 revisions) (flutter/flutter#192157) 2026-09-02 34465683+rkishan516@users.noreply.github.com feat: Add placeholder to DecorationImage (flutter/flutter#191528) 2026-09-02 papmodern14@gmail.com [Android] Do not schedule engine frames on a detached FlutterJNI (flutter/flutter#191204) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from 3a242fb7cdb3 to 28447e9fb8d5 (1 revision) (flutter/flutter#192145) 2026-09-02 engine-flutter-autoroll@skia.org Roll Dart SDK from 9164def35347 to d8d7869d8031 (2 revisions) (flutter/flutter#192144) 2026-09-02 74458687+anazr9@users.noreply.github.com Add FadeInImageTransition.fadeInOver to fade the image in over the placeholder (flutter/flutter#186246) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from 63d13ec4df6d to 3a242fb7cdb3 (9 revisions) (flutter/flutter#192138) 2026-09-02 bkonyi@google.com [flutter_tools] Add --no-plugins flag and disable plugins for benchmark (flutter/flutter#192129) 2026-09-02 bkonyi@google.com [tool] Migrate ChannelCommand to modular dependency injection (flutter/flutter#190751) 2026-09-01 46920873+gabrimatic@users.noreply.github.com Add mouseCursor to RawScrollbar (flutter/flutter#185750) 2026-09-01 ksanaullah383.khan@gmail.com Fix self-comparison and assert typos in TestSemantics (flutter/flutter#191938) 2026-09-01 robert.ancell@canonical.com [Linux] Wait for frames to be rendered before using them in another context (flutter/flutter#192094) 2026-09-01 jacksongardner@google.com [wimp] Turn on multithreading for wimp. (flutter/flutter#191747) 2026-09-01 engine-flutter-autoroll@skia.org Roll Skia from 3911a1fe7f7a to 63d13ec4df6d (6 revisions) (flutter/flutter#192125) 2026-09-01 jason-simmons@users.noreply.github.com Remove the redundant flutter/shell/platform/android:robolectric_tests build target (flutter/flutter#192117) 2026-09-01 AfzalivE@users.noreply.github.com [iOS] Preserve semantics parents after reparenting (flutter/flutter#189686) 2026-09-01 bkonyi@google.com [flutter_tools] Report preview reload timing analytics in LspPreviewDetector (flutter/flutter#192120) 2026-09-01 bkonyi@google.com [flutter_tools] Implement Templates slice and flutter create integration (flutter/flutter#191748) 2026-09-01 louisehsu@google.com Uiscene migrate add2app hosts (flutter/flutter#191847) 2026-09-01 engine-flutter-autoroll@skia.org Roll Packages from d642322 to 7a7912f (8 revisions) (flutter/flutter#192119) 2026-09-01 zhongliu88889@gmail.com [web] Anchor flt-semantics-host at 0,0 to fix WebKit semantics offset (flutter/flutter#190486) 2026-09-01 jason-simmons@users.noreply.github.com Replace Shell::WaitForFirstFrame with an asynchronous API that matches the Shell threading model (flutter/flutter#191841) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC stuartmorgan@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
…r#12727) flutter/flutter@0cbd1a4...70797e1 2026-09-02 bkonyi@google.com [analysis] Upgrade package:analyzer to 14.3.0 and enable custom plugin compilation (flutter/flutter#191590) 2026-09-02 engine-flutter-autoroll@skia.org Roll Dart SDK from d8d7869d8031 to 9a8fbde80f6c (1 revision) (flutter/flutter#192179) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from dec311244db4 to ae22da2e8308 (1 revision) (flutter/flutter#192177) 2026-09-02 bkonyi@google.com [tool] Migrate DoctorCommand to modular dependency injection (flutter/flutter#190758) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from 482b7e285244 to dec311244db4 (3 revisions) (flutter/flutter#192161) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from 28447e9fb8d5 to 482b7e285244 (3 revisions) (flutter/flutter#192157) 2026-09-02 34465683+rkishan516@users.noreply.github.com feat: Add placeholder to DecorationImage (flutter/flutter#191528) 2026-09-02 papmodern14@gmail.com [Android] Do not schedule engine frames on a detached FlutterJNI (flutter/flutter#191204) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from 3a242fb7cdb3 to 28447e9fb8d5 (1 revision) (flutter/flutter#192145) 2026-09-02 engine-flutter-autoroll@skia.org Roll Dart SDK from 9164def35347 to d8d7869d8031 (2 revisions) (flutter/flutter#192144) 2026-09-02 74458687+anazr9@users.noreply.github.com Add FadeInImageTransition.fadeInOver to fade the image in over the placeholder (flutter/flutter#186246) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from 63d13ec4df6d to 3a242fb7cdb3 (9 revisions) (flutter/flutter#192138) 2026-09-02 bkonyi@google.com [flutter_tools] Add --no-plugins flag and disable plugins for benchmark (flutter/flutter#192129) 2026-09-02 bkonyi@google.com [tool] Migrate ChannelCommand to modular dependency injection (flutter/flutter#190751) 2026-09-01 46920873+gabrimatic@users.noreply.github.com Add mouseCursor to RawScrollbar (flutter/flutter#185750) 2026-09-01 ksanaullah383.khan@gmail.com Fix self-comparison and assert typos in TestSemantics (flutter/flutter#191938) 2026-09-01 robert.ancell@canonical.com [Linux] Wait for frames to be rendered before using them in another context (flutter/flutter#192094) 2026-09-01 jacksongardner@google.com [wimp] Turn on multithreading for wimp. (flutter/flutter#191747) 2026-09-01 engine-flutter-autoroll@skia.org Roll Skia from 3911a1fe7f7a to 63d13ec4df6d (6 revisions) (flutter/flutter#192125) 2026-09-01 jason-simmons@users.noreply.github.com Remove the redundant flutter/shell/platform/android:robolectric_tests build target (flutter/flutter#192117) 2026-09-01 AfzalivE@users.noreply.github.com [iOS] Preserve semantics parents after reparenting (flutter/flutter#189686) 2026-09-01 bkonyi@google.com [flutter_tools] Report preview reload timing analytics in LspPreviewDetector (flutter/flutter#192120) 2026-09-01 bkonyi@google.com [flutter_tools] Implement Templates slice and flutter create integration (flutter/flutter#191748) 2026-09-01 louisehsu@google.com Uiscene migrate add2app hosts (flutter/flutter#191847) 2026-09-01 engine-flutter-autoroll@skia.org Roll Packages from d642322 to 7a7912f (8 revisions) (flutter/flutter#192119) 2026-09-01 zhongliu88889@gmail.com [web] Anchor flt-semantics-host at 0,0 to fix WebKit semantics offset (flutter/flutter#190486) 2026-09-01 jason-simmons@users.noreply.github.com Replace Shell::WaitForFirstFrame with an asynchronous API that matches the Shell threading model (flutter/flutter#191841) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC stuartmorgan@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
…r#12727) flutter/flutter@0cbd1a4...70797e1 2026-09-02 bkonyi@google.com [analysis] Upgrade package:analyzer to 14.3.0 and enable custom plugin compilation (flutter/flutter#191590) 2026-09-02 engine-flutter-autoroll@skia.org Roll Dart SDK from d8d7869d8031 to 9a8fbde80f6c (1 revision) (flutter/flutter#192179) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from dec311244db4 to ae22da2e8308 (1 revision) (flutter/flutter#192177) 2026-09-02 bkonyi@google.com [tool] Migrate DoctorCommand to modular dependency injection (flutter/flutter#190758) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from 482b7e285244 to dec311244db4 (3 revisions) (flutter/flutter#192161) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from 28447e9fb8d5 to 482b7e285244 (3 revisions) (flutter/flutter#192157) 2026-09-02 34465683+rkishan516@users.noreply.github.com feat: Add placeholder to DecorationImage (flutter/flutter#191528) 2026-09-02 papmodern14@gmail.com [Android] Do not schedule engine frames on a detached FlutterJNI (flutter/flutter#191204) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from 3a242fb7cdb3 to 28447e9fb8d5 (1 revision) (flutter/flutter#192145) 2026-09-02 engine-flutter-autoroll@skia.org Roll Dart SDK from 9164def35347 to d8d7869d8031 (2 revisions) (flutter/flutter#192144) 2026-09-02 74458687+anazr9@users.noreply.github.com Add FadeInImageTransition.fadeInOver to fade the image in over the placeholder (flutter/flutter#186246) 2026-09-02 engine-flutter-autoroll@skia.org Roll Skia from 63d13ec4df6d to 3a242fb7cdb3 (9 revisions) (flutter/flutter#192138) 2026-09-02 bkonyi@google.com [flutter_tools] Add --no-plugins flag and disable plugins for benchmark (flutter/flutter#192129) 2026-09-02 bkonyi@google.com [tool] Migrate ChannelCommand to modular dependency injection (flutter/flutter#190751) 2026-09-01 46920873+gabrimatic@users.noreply.github.com Add mouseCursor to RawScrollbar (flutter/flutter#185750) 2026-09-01 ksanaullah383.khan@gmail.com Fix self-comparison and assert typos in TestSemantics (flutter/flutter#191938) 2026-09-01 robert.ancell@canonical.com [Linux] Wait for frames to be rendered before using them in another context (flutter/flutter#192094) 2026-09-01 jacksongardner@google.com [wimp] Turn on multithreading for wimp. (flutter/flutter#191747) 2026-09-01 engine-flutter-autoroll@skia.org Roll Skia from 3911a1fe7f7a to 63d13ec4df6d (6 revisions) (flutter/flutter#192125) 2026-09-01 jason-simmons@users.noreply.github.com Remove the redundant flutter/shell/platform/android:robolectric_tests build target (flutter/flutter#192117) 2026-09-01 AfzalivE@users.noreply.github.com [iOS] Preserve semantics parents after reparenting (flutter/flutter#189686) 2026-09-01 bkonyi@google.com [flutter_tools] Report preview reload timing analytics in LspPreviewDetector (flutter/flutter#192120) 2026-09-01 bkonyi@google.com [flutter_tools] Implement Templates slice and flutter create integration (flutter/flutter#191748) 2026-09-01 louisehsu@google.com Uiscene migrate add2app hosts (flutter/flutter#191847) 2026-09-01 engine-flutter-autoroll@skia.org Roll Packages from d642322 to 7a7912f (8 revisions) (flutter/flutter#192119) 2026-09-01 zhongliu88889@gmail.com [web] Anchor flt-semantics-host at 0,0 to fix WebKit semantics offset (flutter/flutter#190486) 2026-09-01 jason-simmons@users.noreply.github.com Replace Shell::WaitForFirstFrame with an asynchronous API that matches the Shell threading model (flutter/flutter#191841) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC stuartmorgan@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
…nt inner skip
- Dispose `Paragraph` and `Picture` inside `drawText` and register `addTearDown(image.dispose)`.
- Remove the redundant inner `skip: isSafari || isFirefox` on `Renders tab as space instead of tofu` (already applied on the enclosing `group('Text', ..., skip: isSafari || isFirefox)`).
- Correct historical commit attribution in PR notes (flutter#175442 introduced `DlWimpImageFromPicture::GetImpellerTexture` with a null `TypographerContext`, flutter#183913 added the `!isWimp` skip, and flutter#191747 refactored `GetImpellerTexture` to use `ImpellerRenderContext`).
…193426) Unskips `Renders tab as space instead of tofu` in `engine/src/flutter/lib/web_ui/test/ui/text_test.dart` on Wimp (`chrome-dart2wasm-wimp-ui`) and strengthens the test assertions: - When `DlWimpImageFromPicture::GetImpellerTexture` was introduced in flutter#175442 (`e0f544bf620`), it constructed a temporary `impeller::AiksContext` with a `nullptr` `TypographerContext`, causing `picture.toImage()` to drop text runs and return blank images (which caused `matchImage(tabImage, tofuImage)` to fail when the `!isWimp` skip was added in flutter#183913 / `2490c22d5be`). - flutter#191747 (`87c02198123`) refactored `DlWimpImageFromPicture::GetImpellerTexture` to render via `ImpellerRenderContext` (which owns the shared `impeller::ContentContext` and `TypographerContextSkia::Make()`), restoring `.notdef` glyph rendering in `picture.toImage()`. - Removes the `!isWimp` guard so `Renders tab as space instead of tofu` runs on both Skwasm and Wimp. - Adds an assertion that unassigned non-control codepoints (`\u{0378}`) also render a `.notdef` tofu box distinct from `''`, and disposes `Paragraph`, `Picture`, and `Image` resources created inside `drawText`. Fixes flutter#183944 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant in-code documentation (doc comments with `///`). - [x] If this PR introduces a new feature or capability, I created and linked a website documentation issue or PR in [flutter/website] (or verified none is needed). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [flutter/website]: https://github.com/flutter/website [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
This turns multithreaded wimp on. This fixes a few issues that popped up when diagnosing problems related to multithreading:
VideoFrameobjects to the web worker, make anImageBitmapout of it. This normalizes the EXIF rotation of the image so that we don't end up with problems transferrring it to a texture.ContentContextwhen generating the impeller texture. This makes sure that paths like the picture-to-image code path can share resources like the typographer context and cached runtime effects.ContentContextwhen rendering pictures to images now, we do a pre-pass that renders pictures to their image representations before starting the rendering of the frame and puts them in the image cache. This avoids having nested render passes inside of a frame, which can cause problems for the state of the glyph atlas in theContentContext