[camera_android_camerax] Add agentic guidance for adding native unit tests by camsim99 · Pull Request #12369 · flutter/packages

camsim99 · 2026-08-04T21:48:25Z

Adds guidance for agents to ensure native unit tests are added when native Android code is modified by:

Updating AGENTS.md to explicitly note how to run native unit tests + that tests should be added if any of the Java files re touched
Updating the pre-push skill to ensure native unit tests are added as expected + pass
Adds evals for the new guidance added to the pre-push skill to ensure native unit tests not being added or failing causes the skill to fail. Results of running them:
- Eval 1 (Missing Native Test): PASSED – Detected that CameraProxyApi.java was modified without tests in android/src/test/ and blocked push readiness (# NO, you are not ready to push.).
- Eval 2 (Success Native Test): PASSED – Ran both dart-test and native-test --no-integration, verified CameraTest.java covered CameraProxyApi.java, handled unrelated test environment exceptions per the rubric, and approved push readiness (# YES, you are ready to push!).
- Eval 3 (Mixed Missing Native Test): PASSED – Detected that although Dart unit tests were updated for Dart changes, CameraProxyApi.java lacked a test update in android/src/test/, and blocked push readiness.
- Eval 4 (Bypassed Mixed Missing Native Test): PASSED -- Explicitly identified that CameraProxyApi.java
  was modified without a native test update, but noted: "since you explicitly approved bypassing the native test requirement for this comment-only change, this check is marked as approved."

Follow up from one-shot attempt #12302. See go/flutter-project-one-shot for more information on the project.

Pre-Review Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the gemini-code-assist bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.

Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling. ↩ ↩²

flutter-dashboard · 2026-08-04T21:48:33Z

It looks like this pull request may not have tests. Please make sure to add tests or get an explicit test exemption before merging.

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. If you believe this PR qualifies for a test exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!). The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group.

gemini-code-assist

Code Review

This pull request updates the pre-push skill documentation and agent guidelines to require and verify native Android unit tests alongside Dart unit tests, and adds evaluation scripts to test this behavior. The reviewer feedback suggests wrapping command substitutions in double quotes to handle paths with spaces, running repository-level tools from the repository root, and configuring temporary git author information in evaluation scripts to prevent commit failures in clean CI/CD environments.

reidbaker · 2026-08-06T19:49:52Z

+# Commit so git status is clean and git diff against origin/main shows the Java change
+cd "$PACKAGE_DIR" || exit 1
+git add "$JAVA_FILE"
+git -c user.name="Author" -c user.email="author@example.com" commit -m "eval: temporary commit with Java change and no test update"


Is there a way to see if git can confirm it is in a tree and not on master. I can see agents running this eval in the wrong context and polluting the git history.

reidbaker · 2026-08-06T19:58:15Z

Meta comment these are expensive tests to run and I predict we will want to minimize the number of these and maximize the things being evaluated per eval. Similar to an integration test.

No changed needed for now but next time we are adding evals for a particular pre-push situation lets try to pick one of the three test_data/setup_* scripts to update.

reidbaker · 2026-08-06T20:04:23Z

-  any new logic. For integration tests, see [flutter-add-integration-test](.agents/skills/flutter-add-integration-test/SKILL.md).
+  any new logic.
+  - **Dart Unit Tests**: See [dart-add-unit-test](.agents/skills/dart-add-unit-test/SKILL.md).
+  - **Native Unit Tests**: Run `cd "$(git rev-parse --show-toplevel)" && dart run script/tool/bin/flutter_plugin_tools.dart native-test --packages camera_android_camerax --no-integration`.


Do you need to run cd "$(git rev-parse --show-toplevel)" Shouldn't the flutter_plugin_tools command work no matter the directory? If we need to find flutter_plugin_tools we can modify the bootstrapping skill that runs to make sure agents are ready to work by either installing the tool or adding an environment variable.

…laceFirst

…implify command in AGENTS.md

…ation

reidbaker · 2026-08-14T18:42:23Z

  }

+  Future<bool> _activateFlutterPluginTools(String workspaceRoot) async {
+    _log('5. Activating flutter_plugin_tools...');


nit. If we are going to include numbers then lets put the methods in log number order.

reidbaker · 2026-08-14T18:45:05Z

+      "prompt": "First, run 'dart run .agents/skills/pre-push-skill/evals/test_data/setup_missing_native_test.dart'. Then, run the /pre-push skill. Do not attempt to fix any issues you find, only report the output of the skill.",
+      "expected_chat_output": [
+        "Agent explicitly states that the code is not ready to push.",
+        "Agent points out that a native Java file was modified.",


Do we want the agent to say the name of the file? if so maybe this should say Agent points out that <path to fille> was modified

reidbaker · 2026-08-14T18:47:12Z

+        "No changes are pushed to the remote repository.",
+        "The agent does not create any new commits or modify the working tree."
+      ],
+      "agent_config": "reidbaker-agent"


Non blocking: I think we should probably have contributor-agent, oneshot-agent and bare-agent as the groups of skills.

where maybe we have contributor-agent be the default not actually an agent you select and where onshot-agent has tighter expectations.

reidbaker · 2026-08-14T18:48:37Z

 ```bash
-cd $(git rev-parse --show-toplevel)
-dart run script/tool/bin/flutter_plugin_tools.dart \
+dart pub global run flutter_plugin_tools \


I really like this change where you modify the check-readyness skills then can make the future skills easier to author and maintain.

reidbaker · 2026-08-19T21:38:56Z

Non blocking comment but does cognitive complexity run on the files in test_data? If not consider adding it in a follow up pr. or at least filing an issue.

…-skill/evals/evals.json Co-authored-by: Reid Baker <1063596+reidbaker@users.noreply.github.com>

…r#191480) flutter/packages@1785501...252bb33 2026-08-20 pateltirth454@gmail.com [material_ui] A typo in the README file of the package. (flutter/packages#12526) 2026-08-20 jmccandless@google.com [material_ui] Animated theme test from flutter/flutter (flutter/packages#12487) 2026-08-20 fluttergithubbot@gmail.com Sync release-material_ui-1.0.1 to main (flutter/packages#12513) 2026-08-20 pateltirth454@gmail.com [material_ui] [cupertino_ui] Add missing Widget of the Week videos to widget API docs (flutter/packages#12468) 2026-08-20 43054281+camsim99@users.noreply.github.com [camera_android_camerax] Add agentic guidance for adding native unit tests (flutter/packages#12369) 2026-08-20 21270878+elliette@users.noreply.github.com [material_ui] SearchAnchor overlay expands to full screen when viewport size changes (flutter/packages#12466) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages-flutter-autoroll Please CC flutter-ecosystem@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

…logic (#12371) Corrects the version and merge conflict validation logic in the `pre-push` skill to compare the local branch against the upstream branch. If the merge conflict check fails, then the skill immediately reports that the code is not ready to push now. Also, adds an eval to ensure this update works as expected. Result of running it: | Eval ID | Scenario | Expected Outcome | Result | Justification | | :---: | :--- | :--- | :---: | :--- | | **1** | Behind Upstream (`setup_behind_upstream.sh`) | Detect branch is behind `upstream/main`, stop immediately without updating release info | **PASS** | Checked merge-base against `upstream/main`, halted immediately with `# NO, you are not ready to push`, and left `pubspec.yaml` and `CHANGELOG.md` unmodified. | | **2** | Behind Upstream with Conflicts (`setup_behind_upstream_conflict.dart`) | Detect that the branch is behind the upstream repository | **PASS** | Detected that the branch is behind the upstream repository, Ran git merge-tree and detected merge conflicts (CONFLICT (content): Merge conflict in .ci/flutter_master.version), Stopped immediately without running subsequent checks, Output # NO, you are not ready to push. and provided actionable instructions on how to resolve conflicts before pushing Ideally #12369 lands first so I can update the references of origin to upstream. ---- Follow up from one-shot attempt #12302. See go/flutter-project-one-shot for more information on the project. ## Pre-Review Checklist **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. [^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.

flutter-dashboard Bot added the CICD Run CI/CD label Aug 4, 2026

github-actions Bot added the p: camera label Aug 4, 2026

gemini-code-assist Bot reviewed Aug 4, 2026

View reviewed changes

camsim99 added override: no versioning needed Override the check requiring version bumps for most changes override: no changelog needed Override the check requiring CHANGELOG updates for most changes labels Aug 4, 2026

camsim99 mentioned this pull request Aug 4, 2026

[camera_android_camerax] Restore torch state when camera changes #12302

Closed

7 of 11 tasks

camsim99 requested a review from reidbaker August 5, 2026 17:32

camsim99 mentioned this pull request Aug 5, 2026

[camera_android_camerax] Correct pre-push skill version validation logic #12371

Merged

9 of 11 tasks

reidbaker reviewed Aug 6, 2026

View reviewed changes

camsim99 added 14 commits August 11, 2026 13:04

Base changes

3e792b7

add evals for pre-push skill

df65c25

Add eval for mixed Dart/native updates

9625259

use echo

98c5f02

address review

ec29cc3

add eval for bypassing adding tests check

d9adec8

Address review feedback and convert test scripts to Dart

450a615

Add license headers to dummy eval files

1115b9e

update evals

6ce017e

Fix license headers properly in dummy files

9bd1e09

Fix license headers in test setup scripts

dd8ac1b

remove confirmation expectation

c61fd5b

Fix leading_newlines_in_multiline_strings warnings in test setup scripts

af58437

Fix merged license headers in setup scripts by removing redundant rep…

39568d3

…laceFirst

camsim99 force-pushed the cos_nativetests branch from cc5ac47 to 39568d3 Compare August 11, 2026 20:04

camsim99 added 5 commits August 11, 2026 13:38

Globally activate local flutter_plugin_tools in check-readiness and s…

5220074

…implify command in AGENTS.md

activate plugins tool

ba88ce3

check for main

22cc937

be clear about generated files + adjust for global plugins tool activ…

9be59f6

…ation

fix static analysis

ad564ad

camsim99 added 4 commits August 11, 2026 15:30

line

f4357f3

fix analyze

cc98052

format

a67218c

format

bd8e616

camsim99 requested a review from reidbaker August 11, 2026 23:40

reidbaker reviewed Aug 14, 2026

View reviewed changes

address review

f4b2179

camsim99 requested a review from reidbaker August 19, 2026 17:46

reidbaker approved these changes Aug 19, 2026

View reviewed changes

Update packages/camera/camera_android_camerax/.agents/skills/pre-push…

3c45d57

…-skill/evals/evals.json Co-authored-by: Reid Baker <1063596+reidbaker@users.noreply.github.com>

camsim99 added the autosubmit Merge PR when tree becomes green via auto submit App label Aug 20, 2026

auto-submit Bot merged commit e31dfab into flutter:main Aug 20, 2026
13 checks passed

camsim99 mentioned this pull request Aug 20, 2026

[tool] Add test_data directory to cognitive complexity check #12523

Open

11 tasks done

engine-flutter-autoroll mentioned this pull request Aug 21, 2026

Roll Packages from 1785501174f3 to 252bb33ad366 (6 revisions) flutter/flutter#191480

Merged

Conversation

camsim99 commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-Review Checklist

Footnotes

Uh oh!

flutter-dashboard Bot commented Aug 4, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

camsim99 commented Aug 4, 2026 •

edited

Loading