feat(@angular-devkit/build-angular): Switch to karma-coverage by kyliau · Pull Request #17901 · angular/angular-cli

kyliau · 2020-06-10T17:57:12Z

This commit switches coverage tooling from karma-coverage-istanbul-reporter
to karma-coverage since it's better supported.

Fixes #17757

kyliau · 2020-07-01T17:14:07Z

Because the coverage test is run asynchronously in a "runaway" promise, I had to do two things here to make the tests work:

Call setTimeout to wait for coverage files to be written
Pass Karma results to the callback because the callback is immediately invoked even though the test hasn't actually completed. Passing the Karma results by reference allows us to retrieve the final exit code modified by karma-coverage if the coverage doesn't meet the threshold.

I'm not very satisfied with both approaches, but after discussing with @filipesilva I think this is the best compromise.
I'm totally happy to revise this PR if there are better suggestions.

alan-agius4 · 2020-07-02T12:03:43Z

+    expect(result.success).toBeTrue();
+    const karmaResults: TestResults = result.karmaResults as never;
+    // exitCode is non-zero because coverage failed
+    expect(karmaResults.exitCode).toBe(1);


With this change, it means that currently with the CLI, there is no way to exit the process with a non zero error code when code coverage threshold is not met.

alan-agius4

This looks great.

Left a couple of comments.

Also, currently code-coverage in the console we display a summary, however it seems that it doesn't work with the new plugin. Can you take a look?

TOTAL: 3 SUCCESS
TOTAL: 3 SUCCESS
TOTAL: 3 SUCCESS

=============================== Coverage summary ===============================
Statements   : 88.89% ( 8/9 )
Branches     : 100% ( 0/0 )
Functions    : 100% ( 0/0 )
Lines        : 85.71% ( 6/7 )
================================================================================

kyliau · 2020-07-29T23:30:45Z

Update: even though karma-coverage 2.0.3 was released to address the issue of onRunComplete() return a promise, more changes are needed upstream before this could be implemented on CLI side without hacky solutions. This is currently blocked on:

fix(reporter): set exit code when thresholds aren't met karma-runner/karma-coverage#420
feat(server): allow 'exit' listeners to set exit code karma-runner/karma#3541

Since karma-coverage 2.0.3 no longer modifies the karma results in-place, the solution implemented in this PR is no longer applicable.

alan-agius4 · 2020-07-31T21:25:17Z

@kyliau, any chance to get in touch with one of the Karma Googler’s to prioritise the above fixes?

googlebot · 2020-09-04T08:14:33Z

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

alan-agius4 · 2020-09-04T11:50:50Z

@googlebot I consent

googlebot · 2020-09-04T11:50:56Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

alan-agius4

LGTM, just one small NIT.

This commit switches coverage tooling from karma-coverage-istanbul-reporter to karma-coverage since it's better supported. Closes angular#17757

angular-automatic-lock-bot · 2020-10-09T08:11:16Z

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}

googlebot added the cla: yes label Jun 10, 2020

kyliau force-pushed the karma-coverage branch from 995b979 to caa9811 Compare June 10, 2020 18:36

SchnWalter mentioned this pull request Jun 10, 2020

karma-coverage-istanbul-reporter author requires payment for support and issue reporting #17757

Closed

1 task

alan-agius4 reviewed Jun 10, 2020

View reviewed changes

Comment thread docs/documentation/stories/code-coverage.md

alan-agius4 reviewed Jun 10, 2020

View reviewed changes

Comment thread packages/angular_devkit/build_angular/src/angular-cli-files/plugins/karma.ts Outdated

kyliau force-pushed the karma-coverage branch 2 times, most recently from 7a20487 to 6de7228 Compare June 10, 2020 23:04

kyliau force-pushed the karma-coverage branch 8 times, most recently from feb9f02 to a7fef26 Compare July 1, 2020 04:28

kyliau changed the title ~~feat(build-angular): Switch to karma-coverage~~ feat(@angular-devkit/build-angular): Switch to karma-coverage Jul 1, 2020

kyliau force-pushed the karma-coverage branch from a7fef26 to a33b3a5 Compare July 1, 2020 04:52

kyliau marked this pull request as ready for review July 1, 2020 17:11

kyliau commented Jul 1, 2020

View reviewed changes

Comment thread packages/angular_devkit/build_angular/src/karma/code-coverage_spec.ts Outdated

kyliau added the target: major This PR is targeted for the next major release label Jul 1, 2020

kyliau commented Jul 1, 2020

View reviewed changes

Comment thread packages/angular_devkit/build_angular/src/karma/code-coverage_spec.ts

alan-agius4 reviewed Jul 2, 2020

View reviewed changes

Comment thread packages/angular_devkit/build_angular/src/karma/code-coverage_spec.ts Outdated

alan-agius4 reviewed Jul 2, 2020

View reviewed changes

Comment thread packages/angular_devkit/build_angular/src/angular-cli-files/plugins/karma.ts Outdated

alan-agius4 reviewed Jul 2, 2020

View reviewed changes

Comment thread packages/angular_devkit/build_ng_packagr/test/ng-packaged/projects/lib/karma.conf.js

alan-agius4 requested changes Jul 2, 2020

View reviewed changes

kyliau force-pushed the karma-coverage branch 3 times, most recently from 6077037 to a33b3a5 Compare July 20, 2020 20:40

kyliau added the state: blocked on upstream label Jul 29, 2020

kyliau force-pushed the karma-coverage branch 7 times, most recently from f69bef7 to da462ea Compare September 4, 2020 02:21

alan-agius4 removed the state: blocked on upstream label Sep 4, 2020

googlebot added cla: no and removed cla: yes labels Sep 4, 2020

alan-agius4 reviewed Sep 4, 2020

View reviewed changes

Comment thread packages/angular_devkit/build_angular/src/angular-cli-files/plugins/karma.ts Outdated

googlebot added cla: yes and removed cla: no labels Sep 4, 2020

alan-agius4 approved these changes Sep 4, 2020

View reviewed changes

kyliau force-pushed the karma-coverage branch 2 times, most recently from 09ccaf4 to 4d89628 Compare September 5, 2020 05:54

feat(@angular-devkit/build-angular): Switch to karma-coverage

b162679

This commit switches coverage tooling from karma-coverage-istanbul-reporter to karma-coverage since it's better supported. Closes angular#17757

alan-agius4 force-pushed the karma-coverage branch from 4d89628 to b162679 Compare September 6, 2020 09:33

alan-agius4 approved these changes Sep 6, 2020

View reviewed changes

alan-agius4 added the action: merge The PR is ready for merge by the caretaker label Sep 6, 2020

alan-agius4 merged commit 8995e49 into angular:master Sep 6, 2020

kyliau deleted the karma-coverage branch September 8, 2020 17:16

angular-automatic-lock-bot Bot locked and limited conversation to collaborators Oct 9, 2020

Conversation

kyliau commented Jun 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kyliau commented Jul 1, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alan-agius4 Jul 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alan-agius4 left a comment

Choose a reason for hiding this comment

Uh oh!

kyliau commented Jul 29, 2020

Uh oh!

alan-agius4 commented Jul 31, 2020

Uh oh!

googlebot commented Sep 4, 2020

Uh oh!

Uh oh!

alan-agius4 commented Sep 4, 2020

Uh oh!

googlebot commented Sep 4, 2020

Uh oh!

alan-agius4 left a comment

Choose a reason for hiding this comment

Uh oh!

angular-automatic-lock-bot Bot commented Oct 9, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

kyliau commented Jun 10, 2020 •

edited

Loading

alan-agius4 Jul 2, 2020 •

edited

Loading