Sitelet https://github.com/msgwing/ZeroSMTP/pull/300
Skip to content

test: add automated test suite for zerosmtp-check - #300

Merged
msgwing merged 2 commits into
msgwing:mainfrom
Mohitingale13:test/add-zerosmtp-check-tests
Aug 26, 2026
Merged

msgwing merged 2 commits into
msgwing:mainfrom
Mohitingale13:test/add-zerosmtp-check-tests

Conversation

@Mohitingale13

Copy link
Copy Markdown
Contributor

What does this PR change?

Resolves #293 by adding an automated regression test suite for packages/zerosmtp-check using Node.js's built-in node:test runner.

Changes & Coverage

  • Added a dependency-free test suite using node:test and node:assert/strict.
  • Exported explain and explainJson from index.js for direct testing while keeping internal helpers private.
  • Added coverage for all 17 entries in ERRORS, including scope disambiguation for shared enhanced codes.
  • Added device panel code regression tests covering:
    • exact 1102
    • 0x1102
    • similar values such as 11020, 01102, and 1102a
  • Added tests for unknown and empty input, including explainJson.
  • Added CLI integration tests for recognized and unrecognized errors.
  • Added "test": "node --test test/" to package.json.
  • Added the test suite to the zerosmtp-check CI job.
  • Preserved the existing files list so tests are excluded from the published package.

Verification

  • 25/25 tests passing locally.
  • git diff --check passes.
  • npm pack --dry-run confirms the test directory is excluded from the package.
  • Intentionally mutating the historical \b regression causes the panel-code tests to fail, confirming the regression is covered.

@github-actions github-actions Bot added the ci label Aug 26, 2026
@msgwing

msgwing commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Merged. This is exactly the shape the issue asked for, and one part of it
deserves calling out.

Intentionally mutating the historical � regression causes the panel-code
tests to fail, confirming the regression is covered.

That is the part most people skip, and it is the part that matters. A guard
nobody has watched fail is not a guard — it is a file that makes everyone feel
safer. You broke it deliberately and confirmed the failure, so we now know the
test is load-bearing rather than decorative.

The boundary cases are the right ones. 1102 must match, 11020, 01102,
91102 and 1102a must not — and that set is precisely what stopped working
when � became a literal backspace character. The lookup silently found
nothing, nothing threw, and nobody noticed. Your test would have failed in the
same minute the bug was introduced.

Two more things done right that I want on the record, because they show you read
the surroundings rather than just the ticket:

  • files in package.json left alone, so the test directory stays out of the
    published package — confirmed with npm pack --dry-run. Somebody shipping
    their tests to every user is a small thing that would have been our fault
    forever.
  • The CI step placed inside the existing zerosmtp-check job rather than as a
    new workflow. This repository has been bitten before by a step added above the
    dependency it needs; yours is in the right order.

export on explain and explainJson is the minimal change that makes them
testable, with no behaviour touched. 24 checks green, 25 tests passing.

This is your sixth merged contribution here and the first one in code. If you
want another, say what kind of thing interests you and I will find the honest
version of it rather than inventing a task to have one available.

@msgwing
msgwing merged commit 539a5c9 into msgwing:main Aug 26, 2026
24 checks passed
@Mohitingale13

Mohitingale13 commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor Author

Thank you — I really appreciate that, especially the feedback about the mutation test. I deliberately wanted to verify that the regression test was actually load-bearing rather than just adding coverage for the sake of coverage.

I'd definitely be interested in another code task. I'm particularly interested in backend engineering, testing/reliability, APIs, automation, and AI/ML-related work, so anything in those areas would be great.

Also, if I may ask something slightly broader: I'm currently looking for a software engineering opportunity, including remote opportunities. I've really enjoyed contributing here and working through real issues in an existing codebase. If you happen to know of any opportunity where I could contribute professionally, I'd be very grateful to hear about it.

Either way, I'm happy to take on another task here. @msgwing

@msgwing

msgwing commented Aug 30, 2026

Copy link
Copy Markdown
Owner

Glad the mutation-testing framing landed — that instinct (does this test actually break when the behaviour breaks, not just add a line to a coverage report) is exactly the right one, and it's rare enough that it's worth saying so directly.

Backend/testing/reliability/automation is a good match for something that shipped today: packages/zerosmtp-mcp, a small Node MCP server (zero dependencies) with a check_relay_reachable tool that opens a real TCP+TLS connection and reports what happened. It has 9 tests, but one real path has zero coverage: what happens when the certificate itself is rejected (expired, self-signed, wrong host). The code distinguishes that from an ordinary connection failure and reports it as a finding rather than a timeout — and nobody has written a test that actually forces a bad certificate and checks the message. If you want it: packages/zerosmtp-mcp/index.js's certowe array names the exact error codes to trigger, and packages/zerosmtp-mcp/test/protocol.test.js has the pattern for talking to the server over stdio the way a real client does. Happy to open it as a proper issue if you want to take it.

On the broader question — I want to answer it straight rather than politely dodge it. This isn't a company with roles to offer; it's a small open-source project, and I have no hiring pipeline or contacts to point you to, so I won't pretend otherwise. What I can say honestly: five merged PRs here, including a test suite you designed with real judgment about what's worth testing, is a genuine, public, citable body of work — and that's worth something on its own terms, even though it's not the same as a job lead. Sorry I can't do more on that front, and thank you for asking anyway.

@Mohitingale13

Copy link
Copy Markdown
Contributor Author

Thanks for the honest answer — I really appreciate you being straightforward about it.

And yes, I'd definitely like to take on the zerosmtp-mcp certificate-validation test. That sounds like a good fit for the kind of backend/testing work I'm trying to get deeper into. @msgwing

@msgwing

msgwing commented Aug 30, 2026

Copy link
Copy Markdown
Owner

Thanks for the offer! Someone else actually beat you to that specific one (the certificate-validation test in zerosmtp-mcp) by less than an hour - their PR is already up for review.

We do have a few other open ones if you're interested: TrueNAS (#288), Zabbix (#289), or PRTG (#290) integration. Let us know which one appeals to you.

@Mohitingale13

Copy link
Copy Markdown
Contributor Author

Thanks! @msgwing I've now wrapped up the documentation side, so I'd like to focus on a coding task next.

If there are any open issues involving backend/Node.js, testing, reliability, automation, or other implementation work that you think would be a good fit, I'd be happy to take one on. I'd especially like to work on something where I can make an actual code change rather than another device/integration entry... for my own practice I'd say

msgwing added a commit that referenced this pull request Aug 30, 2026
…shing (#386)

Found today while building the VS Code extension: npm still serves
1.2.1 (published 2026-08-22 17:58), but #300 (2026-08-26) exported
explain()/explainJson() as part of the public API and added a 25-test
suite - four days of real functionality nobody running `npx
zerosmtp-check` from npm actually has yet. The VS Code extension had to
depend on this package via file:../zerosmtp-check specifically because
of this gap.

Version bump only - .github/workflows/publish-npm.yml is manual by its
own design ("publishing is the owner's decision", a version is
permanent once published). Filing a do-akceptacji issue with the exact
dispatch instructions rather than triggering it myself.
msgwing added a commit that referenced this pull request Aug 30, 2026
#388)

* chore(zerosmtp-check): bump to 1.3.0 - explain/explainJson need publishing

Found today while building the VS Code extension: npm still serves
1.2.1 (published 2026-08-22 17:58), but #300 (2026-08-26) exported
explain()/explainJson() as part of the public API and added a 25-test
suite - four days of real functionality nobody running `npx
zerosmtp-check` from npm actually has yet. The VS Code extension had to
depend on this package via file:../zerosmtp-check specifically because
of this gap.

Version bump only - .github/workflows/publish-npm.yml is manual by its
own design ("publishing is the owner's decision", a version is
permanent once published). Filing a do-akceptacji issue with the exact
dispatch instructions rather than triggering it myself.

* chore(vscode): use the published zerosmtp-check@1.3.0 instead of file:

Gap closed today (#386, #387): zerosmtp-check@1.3.0 is now on npm with
explain()/explainJson() actually exported. Switched the dependency from
file:../zerosmtp-check to ^1.3.0 and verified end to end - reinstalled
from a clean node_modules, confirmed the resolved package.json really
is 1.3.0, ran the extension's own test suite against it (9/9), and
ran the exact dynamic import() the extension code uses
(zerosmtp-check/index.js) directly to confirm explain() is callable
from the real published package, not just present by version number.
msgwing added a commit that referenced this pull request Aug 30, 2026
* chore(zerosmtp-check): bump to 1.3.0 - explain/explainJson need publishing

Found today while building the VS Code extension: npm still serves
1.2.1 (published 2026-08-22 17:58), but #300 (2026-08-26) exported
explain()/explainJson() as part of the public API and added a 25-test
suite - four days of real functionality nobody running `npx
zerosmtp-check` from npm actually has yet. The VS Code extension had to
depend on this package via file:../zerosmtp-check specifically because
of this gap.

Version bump only - .github/workflows/publish-npm.yml is manual by its
own design ("publishing is the owner's decision", a version is
permanent once published). Filing a do-akceptacji issue with the exact
dispatch instructions rather than triggering it myself.

* chore(vscode): use the published zerosmtp-check@1.3.0 instead of file:

Gap closed today (#386, #387): zerosmtp-check@1.3.0 is now on npm with
explain()/explainJson() actually exported. Switched the dependency from
file:../zerosmtp-check to ^1.3.0 and verified end to end - reinstalled
from a clean node_modules, confirmed the resolved package.json really
is 1.3.0, ran the extension's own test suite against it (9/9), and
ran the exact dynamic import() the extension code uses
(zerosmtp-check/index.js) directly to confirm explain() is callable
from the real published package, not just present by version number.

* ci: build Jekyll on every PR/push, not only on deploy to main

Found during review of #366: pages-deploy.yml only builds Jekyll on
push to main, so a broken Liquid template (unclosed {% if %}/{% elsif
%}, a bad include) in a PR passes every other check here and only
breaks once already on production. That PR was verified by hand
(grepping the if/elsif/endif chain) rather than by a gate - which
worked once but doesn't scale to every future change in
docs/_layouts/ or docs/_config.yml. Closes #367.

New job runs the same build pages-deploy.yml runs (same Ruby version,
same --config flags, same fetch-depth: 0 for jekyll-last-modified-at),
minus the deploy steps. No Ruby available locally to test this
directly - the real test is this PR's own CI run.
msgwing added a commit that referenced this pull request Sep 1, 2026
`czeka-czlowiek` i `zalegle-zewnetrzne` pytaja GitHuba o `state: 'open'`.
Scalony pull request jest zamkniety, wiec praca przyjeta i nieodnotowana
byla dla calego nadzoru niewidzialna z definicji - a to jedyne miejsce,
w ktorym konczy zycie wklad kontrybutora.

Zmierzone na wszystkich 14 scalonych wnioskach od ludzi z zewnatrz:
piec nie ma ani jednego naszego slowa (#238 i #245 od dziewieciu dni),
jeden ma ostatnie zdanie ich (#300), retencja 2 z 5, @slegarraga milczy
od 26 dni. Zlecenie tego zadania wymienialo jeden zalegly wklad.

Bramka nie liczy komentarzy, tylko sprawdza ich autora, i to zmienia dwa
wyniki: #362 (@lesbass) ma komentarz, ale napisal go inny kontrybutor,
wiec jest dlugiem; #83-#85 (@slegarraga) nie maja komentarza od nas,
tylko recenzje z trescia, wiec dlugiem nie sa.

Cisza kontrybutora liczona od JEGO ostatniej czynnosci, nie od naszego
scalenia - inaczej wlasne klikniecie byloby dowodem, ze on wciaz z nami
jest. Na @slegarraga roznica wynosi piec dni.

Czego to zadanie celowo nie robi: nie pisze podziekowan (automatyczne
"dziekujemy" mowi czlowiekowi wprost, ze po drugiej stronie nie bylo
nikogo), nie zaczepia nikogo, kto ucichl (prog 21 dni jest wybrany, nie
wyliczony - n=2 odstepy od jednej osoby - wiec uspienie nigdy nie zaklada
zgloszenia samo), nie przypisuje wlasciciela.

Logika w tools/contributor-care.js, testowana na prawdziwych danych tych
14 wnioskow, importuje regule "kto napisal ostatni" z unanswered-external.js
zamiast trzymac jej druga kopie. Sprawdzona przez zepsucie: odwrocenie
progu dojrzalosci wywraca 4 z 16 testow, odwrocenie testu autorstwa 8 z 16.
msgwing added a commit that referenced this pull request Sep 1, 2026
`czeka-czlowiek` i `zalegle-zewnetrzne` pytaja GitHuba o `state: 'open'`.
Scalony pull request jest zamkniety, wiec praca przyjeta i nieodnotowana
byla dla calego nadzoru niewidzialna z definicji - a to jedyne miejsce,
w ktorym konczy zycie wklad kontrybutora.

Zmierzone na wszystkich 14 scalonych wnioskach od ludzi z zewnatrz:
piec nie ma ani jednego naszego slowa (#238 i #245 od dziewieciu dni),
jeden ma ostatnie zdanie ich (#300), retencja 2 z 5, @slegarraga milczy
od 26 dni. Zlecenie tego zadania wymienialo jeden zalegly wklad.

Bramka nie liczy komentarzy, tylko sprawdza ich autora, i to zmienia dwa
wyniki: #362 (@lesbass) ma komentarz, ale napisal go inny kontrybutor,
wiec jest dlugiem; #83-#85 (@slegarraga) nie maja komentarza od nas,
tylko recenzje z trescia, wiec dlugiem nie sa.

Cisza kontrybutora liczona od JEGO ostatniej czynnosci, nie od naszego
scalenia - inaczej wlasne klikniecie byloby dowodem, ze on wciaz z nami
jest. Na @slegarraga roznica wynosi piec dni.

Czego to zadanie celowo nie robi: nie pisze podziekowan (automatyczne
"dziekujemy" mowi czlowiekowi wprost, ze po drugiej stronie nie bylo
nikogo), nie zaczepia nikogo, kto ucichl (prog 21 dni jest wybrany, nie
wyliczony - n=2 odstepy od jednej osoby - wiec uspienie nigdy nie zaklada
zgloszenia samo), nie przypisuje wlasciciela.

Logika w tools/contributor-care.js, testowana na prawdziwych danych tych
14 wnioskow, importuje regule "kto napisal ostatni" z unanswered-external.js
zamiast trzymac jej druga kopie. Sprawdzona przez zepsucie: odwrocenie
progu dojrzalosci wywraca 4 z 16 testow, odwrocenie testu autorstwa 8 z 16.
@msgwing

msgwing commented Sep 1, 2026

Copy link
Copy Markdown
Owner

@Mohitingale13 — you asked this on 30 August and got two days of silence. That is the second time you have asked for coding work and been left waiting, and it is the single worst thing this project can do, because you are the only person who has ever come back and asked for more.

So: the cause, then the work.

The cause, fixed rather than apologised for. Every check we had asked GitHub for open threads. A merged pull request is closed, so accepted work — and any conversation left hanging on it — was invisible to all of our tooling by construction. That is also why #238, #245, #368 and #369 sat merged and unacknowledged. #412 adds the gate that sees closed threads, tested against all fourteen external contributions this repository has ever merged.

The work — #408, and specifically the half of it that is code.

tools/build-cli-errors.py carries a dictionary KODY_URZADZEN of printer panel codes, with a comment promising that every entry is already published on a docs page and is therefore "not a new claim". After #390 merged, that promise is false: 027-779 is in the CLI and on no page anywhere.

We have tools/check-facts.py, which exists precisely to stop one surface contradicting another — and it does not look at that dictionary at all (grep -c KODY_URZADZEN tools/check-facts.py → 0). So the comment is doing the job of a gate, which is the failure mode this project has the most scar tissue about: a rule written in prose instead of enforced.

What would help most is the enforcement, not the page: extend check-facts.py so every key in KODY_URZADZEN must appear in docs/, and make it fail on 027-779 today. Two things worth knowing before you start, both learned here the hard way:

  1. Break it on purpose and watch it refuse. A check whose failing path never runs is decoration — one gate here shipped green and then crashed with UnboundLocalError the first time it was actually needed. Feed it a bad entry, see the refusal, then fix the data.
  2. A failed grep is not proof of absence. Confirm your search sees a known-present case (1102 resolves to six files) before trusting that it sees nothing for 027-779.

If you would rather stay in Node: packages/zerosmtp-mcp has exactly two test files and packages/zerosmtp-check has one, against a CLI whose error-matching logic has already broken silently once — a stray backspace character inside a regex, which no test existed to catch. Uncovered branches there are real reliability work and I would take a pull request against any of them.

Either is yours if you want it; say which and I will put stan:w-pracy on it so nobody doubles up. And thank you for #300 itself — the test suite you asked for and then wrote is what makes the paragraph above possible to say.

@Mohitingale13

Copy link
Copy Markdown
Contributor Author

I'll take #408, specifically the check-facts.py enforcement work.

I'll focus on making the CLI panel-code dictionary and the generated documentation an enforced invariant, including regression coverage for the missing 027-779 entry and a deliberate negative test to verify the gate actually fails when the invariant is broken. @msgwing

@msgwing

msgwing commented Sep 7, 2026

Copy link
Copy Markdown
Owner

@Mohitingale13 — 11 days without a word from us on this one. That is on us.

What this pull request turned into, verified just now rather than remembered:

You wrote the first tests zerosmtp-check ever had. It had none. Today that file runs 28 tests across 5 suites, all passing — the suite grew on the foundation you laid, and every one of those additions was possible because the harness already existed.

The consequence you could not have known about when you wrote it: that package is now published on npm and is what npx zerosmtp-check runs on strangers' machines. Its matching logic had already broken silently once before — a backspace character that crept into a regular expression and was noticed by nobody. Your tests are the reason that cannot happen quietly again. A published version is permanent; the tests are what stand between a bad one and the registry.

Eight of your pull requests are merged into this project — more than anyone else outside it. That is worth saying plainly rather than leaving in the commit log.

You asked on 2026-08-30 for coding work rather than data entry, and there is an answer waiting in that thread. If it is not the right shape, say so and we will find one that is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a real test suite for zerosmtp-check (no network needed)

2 participants