Sitelet https://github.com/Strategy11/formidable-forms/pull/3359
Skip to content

Add cypress-audit Lighthouse checks for dashboard and form preview - #3359

Open
vivi-the-going-merry[bot] wants to merge 316 commits into
masterfrom
fix/issue-6683-cypress-audit
Open

vivi-the-going-merry[bot] wants to merge 316 commits into
masterfrom
fix/issue-6683-cypress-audit

Conversation

@vivi-the-going-merry

@vivi-the-going-merry vivi-the-going-merry Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Adds cypress-audit to the Cypress suite, alongside the existing cypress-axe/cypress-html-validate checks.

  • Registers lighthouse/prepareAudit in cypress.config.js and cypress-audit/commands in the support file.
  • Adds one spec per page for the first pass, matching the existing admin-a11y.cy.js / form-preview-a11y.cy.js split: the dashboard (admin-lighthouse-audit.cy.js) and the front-end form preview (form-preview-lighthouse-audit.cy.js).
  • Thresholds start permissive (0 for every category) on purpose — this run is the baseline measurement. The lighthouse task logs each page's raw category scores to the CI log; thresholds will be tightened to that measured baseline in a follow-up commit before this is ready for review.

package-lock.json's diff is npm-generated fallout from adding lighthouse (a large transitive tree) — not hand-edited.

Verified: npm ci --legacy-peer-deps --include=dev installs cleanly from the updated lockfile; eslint passes on all changed files. Behavioral verification (real Lighthouse scores) comes from this PR's own CI run with run e2e tests — no local browser/wp-env available in this environment.

Fixes Strategy11/formidable-pro#6683

@coderabbitai

coderabbitai Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: Strategy11/formidable-forms/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: c9a8f9bd-b977-4c8a-a3d5-52d582657e63

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@vivi-the-going-merry vivi-the-going-merry Bot added run analysis run e2e tests Run the Cypress end-to-end suite on this PR labels Sep 17, 2026
@deepsource-io

deepsource-io Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

DeepSource Code Review

We reviewed changes in 1b64c62...4ffffed on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.

See full review on DeepSource ↗

PR Report Card

Overall Grade   Security  

Reliability  

Complexity  

Hygiene  

Code Review Summary

Analyzer Status Updated (UTC) Details
PHP Sep 17, 2026 5:51p.m. Review ↗
JavaScript Sep 17, 2026 5:51p.m. Review ↗

Important

AI Review is run only on demand for your team. We're only showing results of static analysis review right now. To trigger AI Review, comment @deepsourcebot review on this thread.

@vivi-the-going-merry

Copy link
Copy Markdown
Contributor Author

Root-caused this run's 4/4 Cypress-shard failures — not a flake, and not caused by the browser switch itself acting alone.

Control check: the commit before --browser chrome landed (bd0be27) only failed 1 of 4 shards, and that failure was the new form-preview-lighthouse-audit.cy.js spec itself (the 403 this next commit set out to fix). After switching e2e:githubrun to --browser chrome, all 4 shards failed, each on a different, pre-existing, unrelated spec (GlobalSettings/globalSettingsUpdates.cy.js timing out on a toast close button, a cy.visit() load failure, a "2 elements matched" click, a "content not found" assertion) — none of them touch lighthouse or this PR's diff.

Mechanism: e2e:githubrun is the one script the whole sharded suite runs, so switching it to --browser chrome moved every spec in every shard off Electron, not just the two new lighthouse specs. Real Chrome under this CI runner is evidently unstable enough for the existing wp-admin timing-sensitive specs to start failing broadly. Separately, before:browser:launch in cypress.config.js calls prepareAudit(launchOptions) unconditionally — so every spec's browser launch now carries Lighthouse's CDP flags too, regardless of browser.

Fix shape (not yet implemented — ran out of runway this tick): don't change the browser or the launch hook for the whole suite. Scope Chrome + prepareAudit to just the two lighthouse specs — e.g. pull admin-lighthouse-audit.cy.js/form-preview-lighthouse-audit.cy.js out of split-specs.sh's sharded set and run them via their own npm script/CI step with --browser chrome, leaving e2e:githubrun (and the other 4 shards) on Electron.

Stylelint red on this PR is unrelated and pre-existing — css/formidableforms.css isn't generated on master either (that job shows skipped there); it's the same gap tracked in #6684, not something this diff caused.

Picking this back up next run to implement the split.

Crabcyborg and others added 25 commits September 18, 2026 11:01
Rebuild the primary blue ramp on an accessible 500
…abelledby

A11y: shared toggle no longer emits dangling aria-labelledby (aria_id_unique)
grey-400 measures 2.58:1 on the white admin body. The footer text is 12px,
so it is normal text and answers to 4.5:1; 500 is the lightest stop on the
ramp that clears it, at 4.97:1. The social icons take the same step.

The social links also had no hover feedback: their resting rule and
.frm_wrap a:hover are both (0,2,1) and this file loads later, so the grey
won. The added pseudo-class settles it at 600, and :focus-visible rides
along so the affordance is not mouse-only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Lift the admin footer text off the grey it could not be read against
Five reported contrast failures, four colours, one shared cause behind three
of them: li.frm_noallow.button, .frm_noallow set opacity 0.5 globally.
Opacity fades the text and what sits behind it toward the same backdrop, so
the ratio between them collapses whatever the colours are — a #065f46 badge
on white measures 7.68:1 at full strength and 3.36:1 at 0.65. The reported
#858992, #888c94 and #89dbb5 are grey-900 and success-500 seen through it,
not values anything declares. The state is carried by colour now.

The NEW pill set only a background and took its text colour from whatever it
landed in, so white on success-500 measured 2.62:1 before any dimming. Both
halves are declared here now, on 800.

The category count badge moves 400 to 500, the lightest stop clearing 4.5:1
on the #f9fafb it sits on. --medium-grey goes 0.65 to 0.70, which lifts
#73787c to #696d72 and 4.46:1 to 5.21:1 while staying translucent for the
surfaces its callers use.

The footer text named in the same report was already fixed in #3377.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The token is inlined into each web component's variables block, six times in
the bundle, and the build that carried the change into frm_admin.css left
that file at the old alpha. Two shipped artifacts disagreed on one token.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both read at 2.58:1 on white and both are text, so 4.5:1 applies. They share
one rule in the dashboard stylesheet, which is where this is fixed rather
than on either element.

That rule is (0,3,1) and outranks .frm_wrap a:hover, so Dismiss had no hover
feedback; the pair added here matches its selector and carries the extra
pseudo-class.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
crate-ci/typos's default branch is now main; master no longer exists
upstream (verified via repos/crate-ci/typos/branches).
Fix low-contrast grey text across the admin
Replace forced clicks on hover-only revealed elements (row-actions,
field action icons, template card buttons) with real CSS-reveal waits
instead of {force: true}, and make duplicateForm.cy.js's teardown run
in afterEach so a mid-test failure doesn't leak a stray "Test Form"
into later specs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Inline eslint-disable-next-line at each flagged site (jQuery.inArray
and pre-existing regex usages), per triage decision to suppress
rather than refactor. Underlying jQuery/regex code unchanged.

One site listed in the issue as js/formidable.js:2549 is actually
js/src/admin/admin.js:2549 (re-verified all 9 line numbers against a
standalone eslint + eslint-plugin-sonarjs run, not the issue's
snapshot numbers).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…rjs-errors

Suppress 9 pre-existing ESLint/DeepSource sonarjs errors
…_logic_row_js_from_lite

Add JS hook to phase out new logic row JS from lite
The Styles edit page renders two <form> elements (style settings, live
preview) and the Import/Export page renders two more (Import, Export) -
all four shared no accessible name, tripping the IBM Equal Access
aria_landmark_name_unique rule. Give each a distinct aria-label.

Closes Strategy11/formidable-pro#6695
Both new tests asserted the same thing (distinct, non-empty aria-label
per <form>) with duplicated regex/assertion logic.
…ist view

render_style_page() reads $_GET (form/style_id) to pick 'edit' vs 'list';
a leftover value from another test would silently render the list view's
single form instead, making this fail for the wrong reason.
The click-handler registration for .frm_add_logic_row ran the filter check
at builder setup time, before Formidable Pro's scripts (which register the
filter) are guaranteed to have run. Bind the handler unconditionally and
check the filter inside the callback instead, so it's evaluated at click
time after all scripts have loaded.

Fixes #3392
Crabcyborg and others added 30 commits September 28, 2026 15:07
…orm_builder

Lazy init drag and drop in form builder until a field is in viewport
…s_for_ajax_loaded_fields_until_in_viewport

Defer adding loading spinners for AJAX loaded fields until in viewport
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ocus-fallback

Fall back to first-error focus when the error summary isn't rendered
…-hidden

Make Formidable logo SVG aria-hidden (svg_graphics_labelled)
A labeled event only checked label presence, not which label was just
added, so any unrelated label add on a PR already carrying 'deploy' or
'beta deploy' re-triggered a real FTP deploy.

Fixes #3453

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…press-qadeploy

CI: don't re-run qadeploy jobs on unrelated label adds
…_test

Try to fix form templates e2e test
…n_newer_versions

Try to support new versions of PHPUnit
…-valid

A11y: valid widget roles for Styles page tabbable elements
…aria

A11y: give a Hidden-position field's input an accessible name
Registers cypress-audit's lighthouse task/prepareAudit alongside the
existing axe/html-validate plugins and adds one spec per page
(admin dashboard, front-end form preview), matching the existing
admin-a11y.cy.js / form-preview-a11y.cy.js split. Thresholds start
permissive - the lighthouse task logs raw category scores to the CI
log so real thresholds can be set from a measured baseline.
form-preview-lighthouse-audit.cy.js hit the same admin-ajax preview
URL as form-preview-a11y.cy.js but without cy.login() first, so
WordPress returned 403 and cy.visit() failed before the audit ran.

Separately, cy.lighthouse() silently skips ("Electron is not
supported") under Cypress's default Electron browser - Lighthouse
needs Chrome DevTools Protocol access, which Electron doesn't expose.
That's why the dashboard spec "passed" in 4s with no score output:
the audit never actually ran. Run the suite under --browser chrome
(present on the ubuntu-latest runner) instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run analysis run e2e tests Run the Cypress end-to-end suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants