Repository navigation
[Windows] Restore and enable IAccessibleEx implementation - #175406
Conversation
|
It looks like this pull request may not have tests. Please make sure to add tests or get an explicit test exemption before merging. If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix? Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. If you believe this PR qualifies for a test exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!). The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group. |
There was a problem hiding this comment.
Code Review
This pull request restores and enables the IAccessibleEx implementation, which exposes new UI Automation capabilities to the MSAA-based accessibility implementation on Windows. Specifically, it adds an identifier to SemanticsNode to expose AutomationId for UI testing. The changes correctly plumb this new field through the accessibility bridge and update relevant documentation and tests. My review includes a minor correction for the updated documentation to improve its accuracy.
e9f2195 to
cceb784
Compare
7954197 to
a4f81de
Compare
|
@loic-sharma could you review this updated PR ? |
|
@loic-peron-inetum-public Thanks for the pull request! I'm a little concerned that this change might regress Flutter Windows's accessibility support in unexpected ways as a similar change to add UI Automation had caused unexpected problems. Could you include details on how you verified this change? What scenarios did you test? |
|
Regarding how we use and tests this proposal: We are building an application targeting iOS and Windows. We automate black-box tests using Appium on both platform. We use a custom build of the flutter engine for Windows that includes the required PR to get Semantics::identifer to be exposed as AutomationId. The application is not evaluated on its accessibility for actual users. Regarding UI Automation-related regressions: I understand that switching to UI Automation required a full switch in the accessibility interface exposed by Flutter. It seems that IAccessibleEx is a way to extend IAccessible-based exposition with the additional capabilities offered by UI Automation, without jeopardizing existing baheviours. What are the unexpected ways that triggered unexpected problems and regressed Flutter accessibility support during a previous attempt to switch to UI Automation ? How can I check them ? |
I would manually check that a sample Flutter Windows app still works well with JAWS, NVDA, and Windows Narrator. I'd make sure that as you tab around, the screen reader properly announces content, without any differences compared to today's implementation. Could we de-risk this by making the Flutter Windows's API has a Could we update Here's what this API might look like: // Configures the accessibility implementation used by Flutter.
enum class AccessibilityMode {
// Default value. Flutter will automatically select the best available
// implementation.
Default,
// Use the IAccessible implementation.
IAccessible,
// Use the experimental IAccessibleEx implementation.
IAccessibleEx,
};
class DartProject {
// Sets the accessibility implementation used by Flutter.
void set_accessibility_mode(AccessibilityMode accessibility_mode) {
accessibility_mode_ = accessibility_mode;
}
}To opt-in to the flutter::DartProject project(L"data");
project.set_accessibility_mode(flutter::AccessibilityMode::IAccessibleEx);Then, |
|
Note to self: these docs are an excellent resource to review this PR: https://learn.microsoft.com/en-us/windows/win32/winauto/implementing-iaccessibleex-for-providers |
I will try and implement an engine-runtime configuration option in november. Could this be enabled using an application-buildtime |
Sadly, no. The Windows embedder doesn't have an easy way to read Dart defines. The suggestion I left above means that the app developer will need to update the C++ code in their Flutter Windows entry point (example). |
|
Is progress still being made on this? |
Hi, yes I still work on this, but I have little work-time currently allocated for this tack. I still work to map the way from Would it be acceptable to use a technical attribute in |
Could you say more about why you need this? I think it's OK to add more data to |
may I know what property you want to add? also is |
…ect_bundle.h Co-authored-by: Matthew Kosarek <matt.kosarek@canonical.com>
ea86bd1 to
111260b
Compare
loic-sharma
left a comment
There was a problem hiding this comment.
The tests look great - thanks for the excellent patch!
|
autosubmit label was removed for flutter/flutter/175406, because This PR has not met approval requirements for merging. The PR author is not a member of flutter-hackers and needs 1 more review(s) in order to merge this PR.
|
|
@mattkae could you re-approve this PR? :) |
Roll Flutter from fb03253e32ce to 3d69471c0bf9 (69 revisions) flutter/flutter@fb03253...3d69471 2026-04-01 engine-flutter-autoroll@skia.org Roll Dart SDK from 1308a3076402 to 043a2bfd56ff (1 revision) (flutter/flutter#184453) 2026-04-01 engine-flutter-autoroll@skia.org Roll Skia from a657b5446209 to c2363c39c283 (2 revisions) (flutter/flutter#184448) 2026-04-01 104349824+huycozy@users.noreply.github.com Fix layout overflowed in small screen in SensitiveContent's example (flutter/flutter#184179) 2026-04-01 engine-flutter-autoroll@skia.org Roll Skia from e0b25041a5d5 to a657b5446209 (1 revision) (flutter/flutter#184445) 2026-04-01 engine-flutter-autoroll@skia.org Roll Dart SDK from 75a089eb6bf9 to 1308a3076402 (3 revisions) (flutter/flutter#184444) 2026-04-01 jesswon@google.com [AGP 9] Bumping KGP error minimum to 2.0.0 (flutter/flutter#184385) 2026-04-01 bkonyi@google.com [ Tool ] Migrate `flutter analyze` to use LSP (flutter/flutter#183785) 2026-04-01 30870216+gaaclarke@users.noreply.github.com Adds uber sdf shader gradients with blend (flutter/flutter#184090) 2026-04-01 sys.int64@gmail.com Add bottom safe area padding to licenses package license page (flutter/flutter#182425) 2026-04-01 ahmedsameha1@gmail.com Handle#6537 third grouped tests (flutter/flutter#183059) 2026-04-01 engine-flutter-autoroll@skia.org Roll Skia from f37239a7a689 to e0b25041a5d5 (9 revisions) (flutter/flutter#184436) 2026-03-31 jason-simmons@users.noreply.github.com [Impeller] Do not log an error when wrapping an empty texture as a TextureGLES (flutter/flutter#184377) 2026-03-31 jason-simmons@users.noreply.github.com Remove the default_git_folder GN argument (flutter/flutter#184152) 2026-03-31 jason-simmons@users.noreply.github.com Remove the cupertino_icons dependency from the spell_check integration test (flutter/flutter#184398) 2026-03-31 engine-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from pM94cWC9cSgao0CG0... to fV-JIWUt4FQGeDtEe... (flutter/flutter#184383) 2026-03-31 engine-flutter-autoroll@skia.org Roll Dart SDK from eaeccf98848d to 75a089eb6bf9 (1 revision) (flutter/flutter#184379) 2026-03-31 mdebbar@google.com [web] Fix autofill in iOS 26 Safari (flutter/flutter#182024) 2026-03-31 engine-flutter-autoroll@skia.org Roll Fuchsia GN SDK from SEfYx3xgueX3aFAY3... to JLBh4Z9PKsjIJcqDU... (flutter/flutter#184368) 2026-03-31 loic.peron@inetum.com [Windows] Restore and enable IAccessibleEx implementation (flutter/flutter#175406) 2026-03-31 30870216+gaaclarke@users.noreply.github.com Revert "Even more awaits (#184042)" (flutter/flutter#184429) 2026-03-31 engine-flutter-autoroll@skia.org Roll Skia from dfd8f8002800 to f37239a7a689 (2 revisions) (flutter/flutter#184374) 2026-03-31 jacksongardner@google.com Remove workaround for fake impeller images in iOS simulator. (flutter/flutter#184264) 2026-03-31 victorsanniay@gmail.com Even more awaits (flutter/flutter#184042) 2026-03-31 engine-flutter-autoroll@skia.org Roll Packages from 582f0e7 to b04f3e5 (6 revisions) (flutter/flutter#184393) 2026-03-30 30870216+gaaclarke@users.noreply.github.com Fixes a flake in reload shaders tests (flutter/flutter#184268) 2026-03-30 jason-simmons@users.noreply.github.com Remove an obsolete script for setting up remote GDB sessions on Android devices (flutter/flutter#184357) 2026-03-30 engine-flutter-autoroll@skia.org Roll Skia from 8dcde79fef2a to dfd8f8002800 (10 revisions) (flutter/flutter#184363) 2026-03-30 engine-flutter-autoroll@skia.org Roll Dart SDK from 0aaccc3c8004 to eaeccf98848d (2 revisions) (flutter/flutter#184362) 2026-03-30 bkonyi@google.com [ Tool ] Remove `flutter running-apps` command (flutter/flutter#183742) 2026-03-30 137456488+flutter-pub-roller-bot@users.noreply.github.com Roll pub packages (flutter/flutter#184045) 2026-03-30 jmccandless@google.com Rick roll triagers on/near April 1st (flutter/flutter#184355) 2026-03-30 49699333+dependabot[bot]@users.noreply.github.com Bump codecov/codecov-action from 5.5.0 to 6.0.0 in the all-github-actions group (flutter/flutter#184364) 2026-03-30 1961493+harryterkelsen@users.noreply.github.com fix(web): call ui.Picture.onDispose for the original picture only (flutter/flutter#184348) 2026-03-30 engine-flutter-autoroll@skia.org Roll Skia from e001e6901e3b to 8dcde79fef2a (7 revisions) (flutter/flutter#184356) 2026-03-30 jason-simmons@users.noreply.github.com [web] Make it safe to call dispose multiple times on a CkSurface (flutter/flutter#184270) 2026-03-30 jason-simmons@users.noreply.github.com Roll HarfBuzz to 13.2.1 (flutter/flutter#184210) 2026-03-30 srawlins@google.com web_ui: Remove unused parameters in a few places (flutter/flutter#183156) 2026-03-30 saurabhmirajkar000@gmail.com Update TabBar documentation to clarify indicatorWeight behavior (flutter/flutter#184104) 2026-03-30 36861262+QuncCccccc@users.noreply.github.com Add title evaluation (flutter/flutter#184084) 2026-03-30 47866232+chunhtai@users.noreply.github.com fixes crash when invisible semantics nodes dropped from semantics tree (flutter/flutter#184226) 2026-03-30 engine-flutter-autoroll@skia.org Roll Skia from cdaae3e3fdef to e001e6901e3b (4 revisions) (flutter/flutter#184345) 2026-03-30 engine-flutter-autoroll@skia.org Roll Packages from 7ae082a to 582f0e7 (8 revisions) (flutter/flutter#184341) 2026-03-30 matej.knopp@gmail.com Add alwaysSizeToContent argument to Overlay. (flutter/flutter#182009) 2026-03-30 engine-flutter-autoroll@skia.org Roll Dart SDK from 598088a8a67f to 0aaccc3c8004 (1 revision) (flutter/flutter#184331) 2026-03-30 engine-flutter-autoroll@skia.org Roll Skia from 6d7ade938643 to cdaae3e3fdef (2 revisions) (flutter/flutter#184329) 2026-03-30 engine-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from EnoD6zNQebz4EYuLk... to pM94cWC9cSgao0CG0... (flutter/flutter#184323) ...
…5406) Restore and enable IAccessibleEx implementation. Exposes new features of UI Automation to MSAA-based implementation. Specifically usefull to expose AutomationId from SemanticsNode::identifier for UI test automation. see flutter#148763 Followup to flutter#161955 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. --------- Co-authored-by: Matthew Kosarek <matt.kosarek@canonical.com>
…5406) Restore and enable IAccessibleEx implementation. Exposes new features of UI Automation to MSAA-based implementation. Specifically usefull to expose AutomationId from SemanticsNode::identifier for UI test automation. see flutter#148763 Followup to flutter#161955 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. --------- Co-authored-by: Matthew Kosarek <matt.kosarek@canonical.com>
…r#11408) Roll Flutter from fb03253e32ce to 3d69471c0bf9 (69 revisions) flutter/flutter@fb03253...3d69471 2026-04-01 engine-flutter-autoroll@skia.org Roll Dart SDK from 1308a3076402 to 043a2bfd56ff (1 revision) (flutter/flutter#184453) 2026-04-01 engine-flutter-autoroll@skia.org Roll Skia from a657b5446209 to c2363c39c283 (2 revisions) (flutter/flutter#184448) 2026-04-01 104349824+huycozy@users.noreply.github.com Fix layout overflowed in small screen in SensitiveContent's example (flutter/flutter#184179) 2026-04-01 engine-flutter-autoroll@skia.org Roll Skia from e0b25041a5d5 to a657b5446209 (1 revision) (flutter/flutter#184445) 2026-04-01 engine-flutter-autoroll@skia.org Roll Dart SDK from 75a089eb6bf9 to 1308a3076402 (3 revisions) (flutter/flutter#184444) 2026-04-01 jesswon@google.com [AGP 9] Bumping KGP error minimum to 2.0.0 (flutter/flutter#184385) 2026-04-01 bkonyi@google.com [ Tool ] Migrate `flutter analyze` to use LSP (flutter/flutter#183785) 2026-04-01 30870216+gaaclarke@users.noreply.github.com Adds uber sdf shader gradients with blend (flutter/flutter#184090) 2026-04-01 sys.int64@gmail.com Add bottom safe area padding to licenses package license page (flutter/flutter#182425) 2026-04-01 ahmedsameha1@gmail.com Handle#6537 third grouped tests (flutter/flutter#183059) 2026-04-01 engine-flutter-autoroll@skia.org Roll Skia from f37239a7a689 to e0b25041a5d5 (9 revisions) (flutter/flutter#184436) 2026-03-31 jason-simmons@users.noreply.github.com [Impeller] Do not log an error when wrapping an empty texture as a TextureGLES (flutter/flutter#184377) 2026-03-31 jason-simmons@users.noreply.github.com Remove the default_git_folder GN argument (flutter/flutter#184152) 2026-03-31 jason-simmons@users.noreply.github.com Remove the cupertino_icons dependency from the spell_check integration test (flutter/flutter#184398) 2026-03-31 engine-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from pM94cWC9cSgao0CG0... to fV-JIWUt4FQGeDtEe... (flutter/flutter#184383) 2026-03-31 engine-flutter-autoroll@skia.org Roll Dart SDK from eaeccf98848d to 75a089eb6bf9 (1 revision) (flutter/flutter#184379) 2026-03-31 mdebbar@google.com [web] Fix autofill in iOS 26 Safari (flutter/flutter#182024) 2026-03-31 engine-flutter-autoroll@skia.org Roll Fuchsia GN SDK from SEfYx3xgueX3aFAY3... to JLBh4Z9PKsjIJcqDU... (flutter/flutter#184368) 2026-03-31 loic.peron@inetum.com [Windows] Restore and enable IAccessibleEx implementation (flutter/flutter#175406) 2026-03-31 30870216+gaaclarke@users.noreply.github.com Revert "Even more awaits (#184042)" (flutter/flutter#184429) 2026-03-31 engine-flutter-autoroll@skia.org Roll Skia from dfd8f8002800 to f37239a7a689 (2 revisions) (flutter/flutter#184374) 2026-03-31 jacksongardner@google.com Remove workaround for fake impeller images in iOS simulator. (flutter/flutter#184264) 2026-03-31 victorsanniay@gmail.com Even more awaits (flutter/flutter#184042) 2026-03-31 engine-flutter-autoroll@skia.org Roll Packages from 582f0e7 to b04f3e5 (6 revisions) (flutter/flutter#184393) 2026-03-30 30870216+gaaclarke@users.noreply.github.com Fixes a flake in reload shaders tests (flutter/flutter#184268) 2026-03-30 jason-simmons@users.noreply.github.com Remove an obsolete script for setting up remote GDB sessions on Android devices (flutter/flutter#184357) 2026-03-30 engine-flutter-autoroll@skia.org Roll Skia from 8dcde79fef2a to dfd8f8002800 (10 revisions) (flutter/flutter#184363) 2026-03-30 engine-flutter-autoroll@skia.org Roll Dart SDK from 0aaccc3c8004 to eaeccf98848d (2 revisions) (flutter/flutter#184362) 2026-03-30 bkonyi@google.com [ Tool ] Remove `flutter running-apps` command (flutter/flutter#183742) 2026-03-30 137456488+flutter-pub-roller-bot@users.noreply.github.com Roll pub packages (flutter/flutter#184045) 2026-03-30 jmccandless@google.com Rick roll triagers on/near April 1st (flutter/flutter#184355) 2026-03-30 49699333+dependabot[bot]@users.noreply.github.com Bump codecov/codecov-action from 5.5.0 to 6.0.0 in the all-github-actions group (flutter/flutter#184364) 2026-03-30 1961493+harryterkelsen@users.noreply.github.com fix(web): call ui.Picture.onDispose for the original picture only (flutter/flutter#184348) 2026-03-30 engine-flutter-autoroll@skia.org Roll Skia from e001e6901e3b to 8dcde79fef2a (7 revisions) (flutter/flutter#184356) 2026-03-30 jason-simmons@users.noreply.github.com [web] Make it safe to call dispose multiple times on a CkSurface (flutter/flutter#184270) 2026-03-30 jason-simmons@users.noreply.github.com Roll HarfBuzz to 13.2.1 (flutter/flutter#184210) 2026-03-30 srawlins@google.com web_ui: Remove unused parameters in a few places (flutter/flutter#183156) 2026-03-30 saurabhmirajkar000@gmail.com Update TabBar documentation to clarify indicatorWeight behavior (flutter/flutter#184104) 2026-03-30 36861262+QuncCccccc@users.noreply.github.com Add title evaluation (flutter/flutter#184084) 2026-03-30 47866232+chunhtai@users.noreply.github.com fixes crash when invisible semantics nodes dropped from semantics tree (flutter/flutter#184226) 2026-03-30 engine-flutter-autoroll@skia.org Roll Skia from cdaae3e3fdef to e001e6901e3b (4 revisions) (flutter/flutter#184345) 2026-03-30 engine-flutter-autoroll@skia.org Roll Packages from 7ae082a to 582f0e7 (8 revisions) (flutter/flutter#184341) 2026-03-30 matej.knopp@gmail.com Add alwaysSizeToContent argument to Overlay. (flutter/flutter#182009) 2026-03-30 engine-flutter-autoroll@skia.org Roll Dart SDK from 598088a8a67f to 0aaccc3c8004 (1 revision) (flutter/flutter#184331) 2026-03-30 engine-flutter-autoroll@skia.org Roll Skia from 6d7ade938643 to cdaae3e3fdef (2 revisions) (flutter/flutter#184329) 2026-03-30 engine-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from EnoD6zNQebz4EYuLk... to pM94cWC9cSgao0CG0... (flutter/flutter#184323) ...
Restore and enable IAccessibleEx implementation.
Exposes new features of UI Automation to MSAA-based implementation.
Specifically usefull to expose AutomationId from SemanticsNode::identifier for UI test automation.
see #148763
Followup to #161955
Pre-launch Checklist
///).