Repository navigation
[Impeller][GLES] Rebind textures before buffer uploads - #192158
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a texture rebind before binding the target texture in BlitCopyBufferToTextureCommandGLES::Encode to resolve an issue on Mali GPUs where deleted shared texture names are reused. Feedback suggests replacing GL_NONE with 0 to avoid a conceptual type mismatch and align with idiomatic OpenGL/GLES usage.
| const auto& gl = reactor.GetProcTable(); | ||
| // Force a rebind after deleted shared texture names can be reused on Mali. | ||
| // See https://github.com/flutter/flutter/issues/190640. | ||
| gl.BindTexture(texture_type, GL_NONE); |
There was a problem hiding this comment.
In OpenGL/GLES, the default texture name (representing no texture bound) is 0. GL_NONE is a GLenum constant (typically used for draw/read buffer settings, etc.) and passing it to a parameter expecting a GLuint texture ID is a conceptual type mismatch, even though both evaluate to 0. Using 0 is more idiomatic and matches the expectation in the unit tests (which expect 0u).
| gl.BindTexture(texture_type, GL_NONE); | |
| gl.BindTexture(texture_type, 0); |
There was a problem hiding this comment.
Thanks. Updated this to use 0u in a7d9451 so the texture-name type and intent are explicit and consistent with the unit-test expectation.
gaaclarke
left a comment
There was a problem hiding this comment.
Can we try this instead? The current patch may still leave paths open to the mali crash. This patch, instead of preemptively clearing out the name, does the cleanup after the texture is used and addresses the case from texture_gles.cc:
diff --git a/engine/src/flutter/impeller/renderer/backend/gles/blit_command_gles.cc b/engine/src/flutter/impeller/renderer/backend/gles/blit_command_gles.cc
index 80e415a7e4..44d320be48 100644
--- a/engine/src/flutter/impeller/renderer/backend/gles/blit_command_gles.cc
+++ b/engine/src/flutter/impeller/renderer/backend/gles/blit_command_gles.cc
@@ -211,6 +211,11 @@ bool BlitCopyBufferToTextureCommandGLES::Encode(
}
const auto& gl = reactor.GetProcTable();
gl.BindTexture(texture_type, gl_handle.value());
+ // Clean up texture binding so the worker/upload context doesn't retain a
+ // dangling reference to a shared texture that may be deleted by another context.
+ // See https://github.com/flutter/flutter/issues/190640.
+ fml::ScopedCleanupClosure unbind([&gl, texture_type]() { gl.BindTexture(texture_type, 0u); });
+
const GLvoid* tex_data =
source.GetBuffer()->OnGetContents() + source.GetRange().offset;
diff --git a/engine/src/flutter/impeller/renderer/backend/gles/texture_gles.cc b/engine/src/flutter/impeller/renderer/backend/gles/texture_gles.cc
index af7fa58d4a..9dfcba9586 100644
--- a/engine/src/flutter/impeller/renderer/backend/gles/texture_gles.cc
+++ b/engine/src/flutter/impeller/renderer/backend/gles/texture_gles.cc
@@ -10,6 +10,7 @@
#include "flutter/fml/logging.h"
#include "flutter/fml/mapping.h"
#include "flutter/fml/trace_event.h"
+#include "flutter/fml/closure.h"
#include "impeller/base/allocation.h"
#include "impeller/base/validation.h"
#include "impeller/core/formats.h"
@@ -316,6 +317,10 @@ bool TextureGLES::OnSetContents(std::shared_ptr<const fml::Mapping> mapping,
}
const auto& gl = reactor.GetProcTable();
gl.BindTexture(texture_type, gl_handle.value());
+ // Clean up texture binding so the worker/upload context doesn't retain a
+ // dangling reference to a shared texture that may be deleted by another context.
+ // See https://github.com/flutter/flutter/issues/190640.
+ fml::ScopedCleanupClosure unbind([&gl, texture_type]() { gl.BindTexture(texture_type, 0u); });
const GLvoid* tex_data = nullptr;
if (mapping) {
tex_data = mapping->GetMapping();It's a bit of a shame to incur the overhead of this for one bad driver, but hopefully it isn't so bad that we have to conditionally do it.
|
Thanks — I like the cleanup direction and agree that leaving texture bindings behind at self-contained upload boundaries is worth addressing. My one concern is that post-use unbinding at these two sites may not repair a stale same-name binding that is already present when the next blit upload starts. In the deterministic trace, the old name 7 binding was created through TextureGLES::InitializeContentsIfNecessary and framebuffer attachment, rather than OnSetContents. If that deleted object is still represented by binding 7 in the resource context when glGenTextures returns 7 again, the first BindTexture(target, 7) in the proposed sequence may already be the operation that the driver treats as no state transition. The cleanup at the end would then be too late for that upload. That said, this is inferred from the trace rather than confirmed driver internals. I will test your exact two-site cleanup without the current pre-bind workaround against the deterministic physical-device reproduction and report the result here before deciding which form to keep. |
|
Thanks for the suggestion — I tested the exact cleanup-only variant: the current pre-bind was removed, and post-use The APK/lib SHA and the crashing tombstone Build ID matched, so there was no stale-build ambiguity. It still crashed in the same Scenario 9 after about 5.13 seconds; I did not repeat the run. The preceding owner, logical handle 311/name 8, was created through By contrast, the pre-bind The public evidence is summarized here. |
|
I completed a diagnostics-free performance A/B for the unconditional pre-bind. The same frozen app commit ( The target was Pixel 5 ( Using the full paginated sample series, the run-percentile mean deltas were CPU p50 +0.104 percentage points, p90 -0.132 percentage points, and p99 +0.060 percentage points. End-to-end iterations were 1553.7 → 1518.7 (-2.25%), but same-variant run-to-run variation was substantial, so the bind cost remains inconclusive. There was no consistent sampled CPU regression signal in this workload. The complete table, artifact identities, and limitations are documented here: |
gaaclarke
left a comment
There was a problem hiding this comment.
lgtm. I think maybe we could have come up with something cleaner but the device is rare. The overhead for the extra set seem negligible so we can avoid putting it behind a workaround branch.
|
@flar can you be a secondary reviewer on this please |
|
Updated proposal: would you consider applying the extra pre-bind to native GLES on Mali-G models with a numeric model identifier of 76 or lower, rather than restricting it to G52/G71/G72/G76 individually or applying it to all GPUs? This scopes the workaround, not Impeller support. In the production records we have reviewed, this crash group includes G52, G71, G72, and G76; we have not identified a report on a Mali-G model numbered above 76. That is an observation from our available records, not evidence of sufficient exposure or proof that higher-numbered models are unaffected. Why broaden the proposal? Driver versions can differ by manufacturer firmware and Android release. A passing test on one OS build does not establish that the same GPU model is unaffected on older firmware. A broader numerical cutoff would include untested models within the range instead of assuming they are safe merely because we lack reports. Evidence so far:
Illustrative code, assuming the renderer has been parsed into an optional numeric Mali-G model identifier: // Compute once during GLES description initialization.
// Parse the complete model token: G720 is 720, not G72.
needs_texture_upload_prebind_ =
is_native_gles && !is_angle && mali_g_model.has_value() &&
*mali_g_model > 0 && *mali_g_model <= 76;
// In BlitCopyBufferToTextureCommandGLES::Encode:
if (description.NeedsTextureUploadPrebind()) {
gl.BindTexture(texture_type, 0u);
}
gl.BindTexture(texture_type, gl_handle.value());The cutoff is explicitly numeric, not an architectural or chronological ordering: for example, G57 and G68 would be included, while G77 and G710 would not. It does not classify Mali-T-series or unknown renderer names. ANGLE would remain excluded. This is a conservative coverage proposal within that range, not proof that every included model is defective or that the cutoff is optimal. Limitations: GPU labels in these device tests are model/SoC-based, without captured runtime GL renderer/driver strings. The G71 run differs in ABI/toolchain from the earlier arm64 baseline and is not a controlled A/B. We have not yet validated pre-bind enabled versus disabled on G71. A shared FBO/abort signature does not prove the same deleted-name-reuse mechanism on every family. The extra bind cost would remain on every direct buffer-to-texture upload on included GPUs; this proposal does not establish a performance upper bound or redesign other GLES binding paths. |
gaaclarke
left a comment
There was a problem hiding this comment.
@dfdgsdfg there is a broken test you'll have to address.
[1519/9318] CommandBufferGLES.BufferToTextureBlitCanBeSubmittedBeforeContextCurrent (51 ms)
[INFO:flutter/testing/test_timeout_listener.cc(75)] Test timeout of 300 seconds per test case will be enforced.
�[0;33mNote: Google Test filter = CommandBufferGLES.BufferToTextureBlitCanBeSubmittedBeforeContextCurrent
�[m�[0;32m[==========] �[mRunning 1 test from 1 test suite.
�[0;32m[----------] �[mGlobal test environment set-up.
�[0;32m[----------] �[m1 test from CommandBufferGLES
�[0;32m[ RUN ] �[mCommandBufferGLES.BufferToTextureBlitCanBeSubmittedBeforeContextCurrent
unknown file: Failure
Unexpected mock function call - returning directly.
Function call: BindTexture(3553, 0)
Google Mock tried the following 1 expectation, but it didn't match:
../../../flutter/impeller/renderer/backend/gles/command_buffer_gles_unittests.cc:193: EXPECT_CALL(mock_gles_impl_ref, BindTexture(0x0DE1, kTextureHandle))...
Expected arg #1: is equal to 1234
Actual: 0
Expected: to be called at least once
Actual: never called - unsatisfied and active
03b12e8 to
d00e967
Compare
|
Arm has confirmed this as a driver erratum. I filed it on the Arm developer forum and Arm replied with the errata writeup they're preparing:
Two things follow from that framing, and they bear on your original reviewcomment.
Per your original suggestion, I'll extend this PR to bind the resource before the attach in |
Arm identified the reproduced texture-name reuse failure as EN_ID 1,792,661, affecting Bifrost and Valhall r17p0 through r23p0 and fixed in r24p0. Cache the workaround decision from native GLES renderer and driver strings, retaining a conservative fallback for unidentifiable Arm G-series drivers. Cover driver boundaries, exclusions, malformed versions, and bind-before-upload ordering.
|
Arm has now identified the correct erratum for our API sequence: EN_ID 1,792,661, affecting Bifrost and Valhall drivers r17p0 through r23p0, with a fix in r24p0. They confirmed that binding texture zero before rebinding the desired name reliably avoids this issue. Based on that confirmation, I have kept the pre-upload zero-bind and have not added the post-use unbind in the two upload functions you suggested ( For now, commit cd0b145 targets the driver range Arm identified: on native GLES with a Mali-G / Immortalis-G renderer, the extra zero-bind applies to r17–r23, and is omitted for known releases before r17 or from r24 onward. If that driver version cannot be determined, the workaround remains enabled conservatively. OEM backports within the affected range cannot be detected. This also corrects my earlier comment promising an extension to @gaaclarke @flar Could you confirm which scope you would prefer? If you would like the workaround applied unconditionally across drivers, or the proposed post-use unbind added to both upload functions, I'm happy to revise the patch accordingly. Otherwise, could you please re-apply the Validation: all 19 driver-scope regression cases pass; related GLES suites report 90 passed, 1 skipped. Host build and pre-push formatting checks passed. The new driver selector has not yet received physical-device validation; the prior device A/B results cover the zero-bind workaround itself. |
I was fine with applying it across drivers for now. If we are going to target specific drivers though I would want that added to |
|
When you're ready for a rereview, just press the "Request re-review" button by my name. |
|
@gaaclarke Updated in 5b2763f: moved the driver guard to Rebuilt with synced dependencies: related GLES tests passed (90 passed, 1 skipped), including all 19 driver-scope cases. Formatting checks also passed. Could you please take another look and re-apply the |
Addresses #190640
Reproduction
Public deterministic reproduction: dfdgsdfg/mali-crash-app. The workload and the historical matrix table are documented in the README.
Evidence
Confirmed facts from the diagnostic engine and the same production-like scenario:
isTexture=falsefor that name.FramebufferTexture2D:GL_INVALID_OPERATION, followed by a missing framebuffer attachment.The trace matrix 7268086803036215683, force-rebind pass matrix 7248171571680282013, and verify-3 matrix 9163761808229023214 are the key Test Lab runs.
Inference: the trace and A/B results are consistent with a deleted object remaining bound and Mali treating a rebind to the same numeric name as no state transition. This is an inference about the driver behavior, not a claim that the GLES implementation violates the object-name specification.
Change
The direct GLES buffer-to-texture upload path now performs one extra zero bind immediately before binding the destination texture. The final binding state is unchanged. The RGBA unit test uses deterministic texture name
7and preserves the ordered0then7binding sequence.This is a minimal workaround and does not supersede #190655: that PR covers a different make-current failure path, while this reproduction had a successful/current resource context. #190655 was not directly tested here.
Physical validation
Using a Flutter 3.44.8-based diagnostic engine with the same app, Scenario 9, and Mali device:
GL_INVALID_OPERATION, FBO failure,isTexture=false, or fatal signal observations.The latest-master patch received the host validation below; the physical Test Lab binaries were built from the Flutter 3.44.8-based diagnostic engine, not from this latest-master patch.
Host validation
BlitCommandGLESTest.BlitCopyBufferToTextureCommandGLESRGBA: 1/1 passedBlitCommandGLESTest.*: 11/11 passedProcTableGLES.*: 5/5 passedReactorGLES.*: 6 passed, 1 skipped because the host does not support labellingImageDecoderNoGLTest.*: 6/6 passedimpeller_unittestsandui_unittestsbuilds passedgit diff --checkpassedScope and trade-offs
This is a narrow workaround validated for one deterministic reproduction; it is not an architectural redesign or a general fix for all GLES texture-binding paths. The current physical mechanism is inferred from the trace and A/B results; this does not prove that every Mali crash in #190640 has the same cause.
The workaround applies only to
BlitCopyBufferToTextureCommandGLES; other GLES bind/upload paths remain unchanged. It adds one additionalglBindTextureper direct buffer-to-texture upload. Affected callers include image decode, glyph atlas, small utility textures, and Flutter GPU buffer-to-texture overwrite/copy paths.The extra bind is likely small compared with texel transfer for normal uploads, but it has not been benchmarked; repeated small per-frame uploads may be sensitive. GL names come from the driver via
glGenTextures, and Flutter cannot prevent numeric name reuse.Reactor-level name-reuse prevention is not a simple alternative: delayed deletion and
glFinishadd memory and stall cost, and do not directly force the observed same-name state transition.Reviewer guidance on performance and scope, including broader workaround plumbing if needed, is welcome.