Summary
Three faults in realtime/integration/proxy/connection_resume.md, all in the fixtures and
assertions rather than in what the tests set out to prove.
RTN15h1 asserts statusCode == 401 alongside code == 40171, and says in its own note that it
follows ably-js. ably-js pairs 40171 with statusCode: 403, and so does ably-python. One
character.
RTN14h's proxy rule writes "connectionKey": "__PASSTHROUGH__" twice, expecting uts-proxy to
substitute the key the server issued. uts-proxy v0.3.0 has no such sentinel and passes the literal
through, so the client resumes with ?resume=__PASSTHROUGH__ and the sandbox rejects it. This one
is a choice between two repositories rather than a text fix.
RTN19a filters the proxy event log on e.type == "ws_frame_to_server" and e.message.action == "MESSAGE". The proxy emits neither field that way, and uts/docs/proxy.md — the suite's own
documentation of the log — already spells out the correct form. The same file gets it right nine
lines earlier, in the poll_until block of the same test. Same class as #545.
Line references are against d9a04ca, which is still main; paths are relative to uts/.
1. realtime/integration/proxy/connection_resume.md:500-506 — RTN15h1 asserts 401 where both SDKs report 403
Test 9, realtime/proxy/RTN15h1/token-error-nonrenewable-failed-0 (:412-518). The assertions
(:500-506):
# Error reason reflects the token error
# NOTE: ably-js reports error code 40171 ("Token not renewable") rather than the injected
# 40142, because the SDK detects it has no means to renew (no key, no authCallback, no
# authUrl) and substitutes a more specific error code before transitioning to FAILED.
ASSERT client.connection.errorReason IS NOT null
ASSERT client.connection.errorReason.code == 40171
ASSERT client.connection.errorReason.statusCode == 401
The note names ably-js as the source of truth for the substituted 40171, and 40171 is right. The
statusCode beside it is not what ably-js attaches. ably/ably-js,
src/common/lib/client/auth.ts:615-631 (main @ 35269c8):
} else {
const msg =
'Need a new token, but authOptions does not include any way to request one (no authUrl, authCallback, or key)';
Logger.logAction(
this.logger,
Logger.LOG_ERROR,
'Auth()',
'library initialized with a token literal without any way to renew the token when it expires (no authUrl, authCallback, or key). See https://help.ably.io/error/40171 for help',
);
throw new ErrorInfo({
message: msg,
code: 40171,
statusCode: 403,
remediation:
'Initialise the client with one of ClientOptions.{ key, authUrl, authCallback } so the SDK can refresh tokens.',
});
}
ably-python raises the same pairing from the same condition, ably/rest/auth.py:198-200:
msg = "Need a new token but auth_options does not include a way to request one"
log.exception(msg)
raise AblyAuthException(msg, 403, 40171)
Measured, running the section verbatim end to end through uts-proxy against the sandbox — a
real token from requestToken(), no key and no authCallback, the proxy injecting DISCONNECTED
with 40142/401 after one second and closing:
connection state: FAILED
errorReason.code: 40171
errorReason.statusCode: 403
40142/401 is the error the proxy injects; 403 is what the SDK substitutes along with 40171. The
401 in the assertion looks like the injected status code carried down into the substituted error
by hand. The derived test asserts 403 and passes; the site carries a # UTS SPEC ERROR: comment
naming this issue's finding.
2. realtime/integration/proxy/connection_resume.md:792 and :794 — RTN14h asks uts-proxy for a substitution it does not make
Test 22, realtime/proxy/RTN14h/resume-after-ttl-expiry-0 (:762-895). Its first rule replaces
the first CONNECTED to shorten connectionStateTtl and pin a known connectionId (:786-808):
"match": { "type": "ws_frame_to_client", "action": "CONNECTED", "count": 1 },
"action": {
"type": "replace",
"message": {
"action": 4,
"connectionId": "proxy-ttl-test-id",
"connectionKey": "__PASSTHROUGH__",
"connectionDetails": {
"connectionKey": "__PASSTHROUGH__",
...
"connectionStateTtl": 2000,
"maxIdleInterval": 15000
}
}
},
"times": 1,
"comment": "RTN14h: Replace 1st CONNECTED to set short connectionStateTtl (2s) and known connectionId"
The intent is clear — keep the real connection key while overriding the TTL — but replace is
documented as a whole-frame swap with no templating. ably/uts-proxy docs/API.md at the pinned
v0.3.0 lists the frame-manipulation actions as suppress, delay, inject_to_client,
inject_to_client_and_close, replace ("Replace the frame with a different one") and
suppress_onwards. No sentinel, placeholder or field-merge mode appears anywhere in the document.
The literal 15 characters go to the client.
Measured through uts-proxy v0.3.0 against the sandbox:
client.connection.key after CONNECTED: __PASSTHROUGH__
reconnect ws_connect queryParams: {'resume': '__PASSTHROUGH__', ...}
sandbox response: {'code': 80018, 'message': 'invalid connection key: __PASSTHROUGH__'}
__PASSTHROUGH__ appears nowhere else in uts/ — this is the only fixture that asks for it.
Our derived test keeps the rule verbatim rather than working around it, because the test is gated
on an unrelated ably-python defect (the server's connectionStateTtl is parsed and never used, so
SUSPENDED never arrives) that fires long before the connection key matters. So this one is not
blocking us today, but it will block anyone whose SDK honours the TTL.
The resolution is one of two things, and we do not have a view on which:
- uts-proxy grows the substitution — a documented sentinel, or a merge mode for
replace that
overrides named fields and leaves the rest of the real frame intact. That is the more useful of
the two, since every replace rule in the suite that wants to change one field currently has to
fabricate the other twelve.
- the specification stops asking for it — drop both
connectionKey lines and let the fixture
live with whatever the replacement frame carries, or state a concrete key and accept that the
resume is rejected server-side, which for RTN14h is arguably fine: the assertion is that the
resume param is present, not that it succeeds (:882-885).
Either way the rule as written cannot do what its comment says.
3. realtime/integration/proxy/connection_resume.md:1007-1010 — RTN19a reads event-log fields the proxy does not emit
Test 23, realtime/proxy/RTN19a/unacked-resent-on-resume-0 (:899-1027). The assertions
(:1006-1014):
# RTN19a: The MESSAGE frame was sent on both transports (original + resend)
message_frames = log.filter(e =>
e.type == "ws_frame_to_server" AND
e.message.action == "MESSAGE"
)
ASSERT message_frames.length >= 2
# RTN19a2: On successful resume, the resent message has the same msgSerial
ASSERT message_frames[0].message.msgSerial == message_frames[1].message.msgSerial
Neither field exists in that shape. ws_frame_to_server is a rule match type, not an event
type: the proxy's event log has one frame type, ws_frame, with the direction in a separate
direction field, and message.action as the protocol integer rather than the name. That filter
matches nothing, so message_frames is empty and message_frames[0] is a subscript out of range.
uts/docs/proxy.md — which this file links at :9 — documents it (docs/proxy.md:164-174):
{
"events": [
{ "type": "ws_connect", "url": "ws://...", "queryParams": { "key": "..." } },
{ "type": "ws_frame", "direction": "server_to_client", "message": { "action": 4, ... } },
{ "type": "ws_disconnect", "initiator": "proxy", "closeCode": 1006 },
{ "type": "http_request", "method": "GET", "path": "/channels/test/messages" },
{ "type": "http_response", "status": 200 }
]
}
and gives this exact recipe under "Common Log Assertions" (docs/proxy.md:185-188):
# Verify a specific frame was sent
frames = log.filter(e => e.type == "ws_frame" AND e.direction == "client_to_server")
attach_frames = frames.filter(f => f.message.action == 10) # ATTACH
ASSERT attach_frames.length == 1
ably/uts-proxy's docs/API.md at v0.3.0 agrees, under Event Log Format: type is one of
ws_connect, ws_frame, ws_disconnect, http_request, http_response, action, and
direction is client_to_server or server_to_client.
The same test already gets this right nine lines earlier, in its Test Steps (:969-977):
poll_until(
condition: () => {
log = session.get_log()
message_sent = log has ws_frame client_to_server action==15
ack_suppressed = log has ws_frame server_to_client action==1 with ruleMatched
return message_sent AND ack_suppressed
},
timeout: 10s
)
ws_frame, client_to_server, integer action — correct. So this is a single stale block, not a
file-wide misunderstanding. The other log reads in this file (e.type == "ws_connect", at :116,
:259, :396, :618, :749, :876, :1002, :1148, :1273) are all fine, as are the
ws_frame_to_client / ws_frame_to_server match types in the rules at :307, :786, :921
and :1192 — those are the documented rule API and should not be touched.
Same class as #545, which records connection_recovery_test.md's RTN16f/RTN16j reading
e.type == "ws_frame" AND e.direction == "client_to_server" against the realtime unit tier's
MockWebSocket, whose contract has neither field. The two files drift in opposite directions from
their respective harnesses; the common cause is event-log field names being written from memory.
Our derived test reads the real field names through a frames(log, direction, action) helper
defined at the top of the file, and asserts exactly what the specification asserts — both the
>= 2 count and the msgSerial equality. It passes.
Suggested fix
For RTN15h1. Replace :501-506 with:
# NOTE: ably-js reports error code 40171 ("Token not renewable") with statusCode 403,
# rather than the injected 40142/401, because the SDK detects it has no means to renew
# (no key, no authCallback, no authUrl) and substitutes its own error before
# transitioning to FAILED.
ASSERT client.connection.errorReason IS NOT null
ASSERT client.connection.errorReason.code == 40171
ASSERT client.connection.errorReason.statusCode == 403
For RTN14h. Nothing to paste — see the two options in section 2. If the specification is the
side that moves, deleting :792 and :794 is the whole change, since :872's
ASSERT client.connection.id != "proxy-ttl-test-id" and :885's resume IS NOT null do not
depend on the key's value. If uts-proxy is the side that moves, the rule can stay exactly as
written and this section needs only a line in docs/proxy.md recording the sentinel.
For RTN19a. Replace :1007-1010 with the form docs/proxy.md documents:
# RTN19a: The MESSAGE frame was sent on both transports (original + resend)
message_frames = log.filter(e =>
e.type == "ws_frame" AND
e.direction == "client_to_server" AND
e.message.action == 15 # MESSAGE
)
:1011 and :1014 then work unchanged.
Found while deriving uts/realtime/integration for ably-python.
Summary
Three faults in
realtime/integration/proxy/connection_resume.md, all in the fixtures andassertions rather than in what the tests set out to prove.
RTN15h1 asserts
statusCode == 401alongsidecode == 40171, and says in its own note that itfollows ably-js. ably-js pairs 40171 with
statusCode: 403, and so does ably-python. Onecharacter.
RTN14h's proxy rule writes
"connectionKey": "__PASSTHROUGH__"twice, expecting uts-proxy tosubstitute the key the server issued. uts-proxy v0.3.0 has no such sentinel and passes the literal
through, so the client resumes with
?resume=__PASSTHROUGH__and the sandbox rejects it. This oneis a choice between two repositories rather than a text fix.
RTN19a filters the proxy event log on
e.type == "ws_frame_to_server"ande.message.action == "MESSAGE". The proxy emits neither field that way, anduts/docs/proxy.md— the suite's owndocumentation of the log — already spells out the correct form. The same file gets it right nine
lines earlier, in the
poll_untilblock of the same test. Same class as #545.Line references are against
d9a04ca, which is stillmain; paths are relative touts/.1.
realtime/integration/proxy/connection_resume.md:500-506— RTN15h1 asserts 401 where both SDKs report 403Test 9,
realtime/proxy/RTN15h1/token-error-nonrenewable-failed-0(:412-518). The assertions(
:500-506):The note names ably-js as the source of truth for the substituted 40171, and 40171 is right. The
statusCodebeside it is not what ably-js attaches.ably/ably-js,src/common/lib/client/auth.ts:615-631(main@35269c8):ably-python raises the same pairing from the same condition,
ably/rest/auth.py:198-200:Measured, running the section verbatim end to end through uts-proxy against the sandbox — a
real token from
requestToken(), no key and no authCallback, the proxy injecting DISCONNECTEDwith 40142/401 after one second and closing:
40142/401 is the error the proxy injects; 403 is what the SDK substitutes along with 40171. The
401 in the assertion looks like the injected status code carried down into the substituted error
by hand. The derived test asserts 403 and passes; the site carries a
# UTS SPEC ERROR:commentnaming this issue's finding.
2.
realtime/integration/proxy/connection_resume.md:792and:794— RTN14h asks uts-proxy for a substitution it does not makeTest 22,
realtime/proxy/RTN14h/resume-after-ttl-expiry-0(:762-895). Its first rule replacesthe first CONNECTED to shorten
connectionStateTtland pin a knownconnectionId(:786-808):The intent is clear — keep the real connection key while overriding the TTL — but
replaceisdocumented as a whole-frame swap with no templating.
ably/uts-proxydocs/API.mdat the pinnedv0.3.0lists the frame-manipulation actions assuppress,delay,inject_to_client,inject_to_client_and_close,replace("Replace the frame with a different one") andsuppress_onwards. No sentinel, placeholder or field-merge mode appears anywhere in the document.The literal 15 characters go to the client.
Measured through uts-proxy v0.3.0 against the sandbox:
__PASSTHROUGH__appears nowhere else inuts/— this is the only fixture that asks for it.Our derived test keeps the rule verbatim rather than working around it, because the test is gated
on an unrelated ably-python defect (the server's
connectionStateTtlis parsed and never used, soSUSPENDED never arrives) that fires long before the connection key matters. So this one is not
blocking us today, but it will block anyone whose SDK honours the TTL.
The resolution is one of two things, and we do not have a view on which:
replacethatoverrides named fields and leaves the rest of the real frame intact. That is the more useful of
the two, since every
replacerule in the suite that wants to change one field currently has tofabricate the other twelve.
connectionKeylines and let the fixturelive with whatever the replacement frame carries, or state a concrete key and accept that the
resume is rejected server-side, which for RTN14h is arguably fine: the assertion is that the
resumeparam is present, not that it succeeds (:882-885).Either way the rule as written cannot do what its comment says.
3.
realtime/integration/proxy/connection_resume.md:1007-1010— RTN19a reads event-log fields the proxy does not emitTest 23,
realtime/proxy/RTN19a/unacked-resent-on-resume-0(:899-1027). The assertions(
:1006-1014):Neither field exists in that shape.
ws_frame_to_serveris a rule match type, not an eventtype: the proxy's event log has one frame type,
ws_frame, with the direction in a separatedirectionfield, andmessage.actionas the protocol integer rather than the name. That filtermatches nothing, so
message_framesis empty andmessage_frames[0]is a subscript out of range.uts/docs/proxy.md— which this file links at:9— documents it (docs/proxy.md:164-174):{ "events": [ { "type": "ws_connect", "url": "ws://...", "queryParams": { "key": "..." } }, { "type": "ws_frame", "direction": "server_to_client", "message": { "action": 4, ... } }, { "type": "ws_disconnect", "initiator": "proxy", "closeCode": 1006 }, { "type": "http_request", "method": "GET", "path": "/channels/test/messages" }, { "type": "http_response", "status": 200 } ] }and gives this exact recipe under "Common Log Assertions" (
docs/proxy.md:185-188):ably/uts-proxy'sdocs/API.mdatv0.3.0agrees, under Event Log Format:typeis one ofws_connect,ws_frame,ws_disconnect,http_request,http_response,action, anddirectionisclient_to_serverorserver_to_client.The same test already gets this right nine lines earlier, in its Test Steps (
:969-977):ws_frame,client_to_server, integer action — correct. So this is a single stale block, not afile-wide misunderstanding. The other log reads in this file (
e.type == "ws_connect", at:116,:259,:396,:618,:749,:876,:1002,:1148,:1273) are all fine, as are thews_frame_to_client/ws_frame_to_servermatch types in the rules at:307,:786,:921and
:1192— those are the documented rule API and should not be touched.Same class as #545, which records
connection_recovery_test.md's RTN16f/RTN16j readinge.type == "ws_frame" AND e.direction == "client_to_server"against the realtime unit tier'sMockWebSocket, whose contract has neither field. The two files drift in opposite directions fromtheir respective harnesses; the common cause is event-log field names being written from memory.
Our derived test reads the real field names through a
frames(log, direction, action)helperdefined at the top of the file, and asserts exactly what the specification asserts — both the
>= 2count and themsgSerialequality. It passes.Suggested fix
For RTN15h1. Replace
:501-506with:For RTN14h. Nothing to paste — see the two options in section 2. If the specification is the
side that moves, deleting
:792and:794is the whole change, since:872'sASSERT client.connection.id != "proxy-ttl-test-id"and:885'sresume IS NOT nulldo notdepend on the key's value. If uts-proxy is the side that moves, the rule can stay exactly as
written and this section needs only a line in
docs/proxy.mdrecording the sentinel.For RTN19a. Replace
:1007-1010with the formdocs/proxy.mddocuments::1011and:1014then work unchanged.Found while deriving
uts/realtime/integrationfor ably-python.