Sitelet https://github.com/NodeOps-app/treg2/commit/f7db3593cb103965947d0130cd22616d3f38f422
Skip to content

Commit f7db359

Browse files
authored
docs(ops): move hosted deployment details private (superdesigndev#470)
* docs(ops): move hosted deployment details private * docs: define paired treg.to checkout
1 parent 46c5510 commit f7db359

24 files changed

Lines changed: 348 additions & 1189 deletions

File tree

‎.agents/skills/tools-registry-context/MAP.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ Regenerate via `scripts/build-map.py`.
1717
| `README.md` | foundation/charter.md |
1818
| `alembic.ini` | architecture/data-model.md |
1919
| `assets/brand/og-card.html` | interface/seo.md |
20+
| `deploy/render.example.yaml` | ops/deploy.md |
2021
| `docs/CLAUDE-CONNECTOR-SUBMISSION.md` | architecture/mcp-oauth.md |
2122
| `dsh/cordis.patch.yml` | interface/skill.md |
2223
| `dsh/index.js` | interface/skill.md |
@@ -27,7 +28,6 @@ Regenerate via `scripts/build-map.py`.
2728
| `plugins/minimax/.minimax-plugin/plugin.json` | interface/skill.md |
2829
| `plugins/treg/.cursor-plugin/plugin.json` | interface/skill.md |
2930
| `pyproject.toml` | architecture/import-boundaries.md, ops/deploy.md |
30-
| `render.yaml` | ops/deploy.md |
3131
| `scripts/backfill_call_archive_links.py` | architecture/archive.md |
3232
| `scripts/build_plugin.py` | interface/skill.md |
3333
| `scripts/catalog_drift.py` | architecture/catalog.md |
@@ -467,5 +467,5 @@ Regenerate via `scripts/build-map.py`.
467467
| `interface/shell.md` | `shell.py`, `cli.py` |
468468
| `interface/skill.md` | `skill.md`, `web.py`, `mcp_install.py`, `build_plugin.py`, `plugin.json`, `marketplace.json`, `plugin.json`, `plugin.json`, `package.json`, `cordis.patch.yml`, `index.js`, `plugin.json`, `minimax_plugin.py` |
469469
| `ops/capacity.md` | `__init__.py`, `collectors.py`, `policy.py`, `sweep.py`, `view.py`, `routes.py`, `signatures.py`, `verify.py`, `marks.py`, `test_capacity_protect.py`, `limiter.py`, `overflow_spend.py`, `routes_view.py`, `overflow.py`, `0007_overflow_spend.py`, `test_capacity_overflow.py`, `test_capacity_overflow_spend.py`, `0008_org_platform_overflow_disabled.py`, `test_capacity_smoothing.py`, `overflow_seed.json`, `__init__.py`, `orthogonal.py`, `monid.py`, `catalogs.py`, `0006_overflow_route.py`, `test_capacity_overflow_routes.py`, `test_influencersclub_overflow.py`, `worker.py`, `provider_balances.py`, `0005_capacity_policy_snapshot.py`, `test_capacity_know.py`, `test_capacity_collectors.py` |
470-
| `ops/deploy.md` | `pyproject.toml`, `__main__.py`, `maintenance.py`, `env.py`, `worker.py`, `selfhost.sh`, `config.py`, `db.py`, `email.py`, `audit.py`, `dev-local.sh`, `render.yaml` |
470+
| `ops/deploy.md` | `pyproject.toml`, `__main__.py`, `maintenance.py`, `env.py`, `worker.py`, `selfhost.sh`, `config.py`, `db.py`, `email.py`, `audit.py`, `dev-local.sh`, `render.example.yaml` |
471471
| `reference/glossary.md` | `2026-06-30-jason-tools-registry.md` |

‎AGENTS.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,23 @@ catalog of external endpoints plus its own team's tools without ever holding an
77
load-bearing mechanic is a proxy that makes the caller's **real upstream request**, injects the
88
credential server-side and relays the answer verbatim. We never model an upstream API.
99

10+
## Paired treg.to checkout
11+
12+
For work on the hosted treg.to service, clone the public `treg` repository and private
13+
`treg-internal` repository as siblings with those exact directory names. When `../treg-internal`
14+
exists, treat both repositories as one operational workspace:
15+
16+
- `treg` owns public product code, portable behavior and self-hosting documentation.
17+
- `treg-internal` owns live production configuration, operational runbooks, incident evidence and
18+
private admin tools.
19+
- Read both repositories before changing production behavior, but never copy credentials, live
20+
environment exports, customer data or raw logs between them.
21+
- Commit and open PRs separately. State the merge order whenever one PR links to or depends on the
22+
other.
23+
24+
Do not clone `treg-internal` inside this repository and do not make it a Git submodule. Start agents
25+
from the repository that owns the task; the sibling path supplies the other half of treg.to context.
26+
1027
## Non-negotiables
1128

1229
Everything else in this file is guidance; these are the contract, and they win over any other passage.

‎README.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -295,7 +295,8 @@ uv run python -m treg keygen # print a fresh Fernet key for TREG_SECRET_KEY
295295
> database drivers, and encryption. `pip install tools-registry` alone gives just the `treg` command for
296296
> talking to an existing registry.
297297
298-
The team instance is hosted on **Render** (web service + Postgres) at `treg.to`.
298+
The official hosted service is available at `treg.to`. Its production topology and live settings are
299+
maintained in the private [operator runbook](https://github.com/superdesigndev/treg-internal/blob/main/docs/production/deploy.md).
299300

300301
## Configuration
301302

@@ -312,7 +313,7 @@ Environment variables (prefix `TREG_`, read from `.env`):
312313
| `TREG_GOOGLE_CLIENT_ID` / `_SECRET` | *(empty)* | Google OAuth sign-in (redirect `<public_url>/auth/google/callback`); empty hides the button |
313314
| `TREG_INSTAGRAM_CLIENT_ID` / `_SECRET` | *(empty)* | Instagram App ID and secret for direct Instagram Login (redirect `<public_url>/oauth/callback`) |
314315
| `TREG_META_CLIENT_ID` / `_SECRET` | *(empty)* | Meta app credentials for Facebook Pages, Meta Ads, and optional Instagram `page-tools` |
315-
| `TREG_OAUTH_REVIEW_PENDING` | `instagram-login,page-messages` | Registry review keys awaiting production access. Remove `page-messages` after Page messaging approval; set empty after direct Instagram approval. |
316+
| `TREG_OAUTH_REVIEW_PENDING` | `instagram-login,page-messages` | Comma-separated registry review keys whose capabilities must remain gated; hosted review state is maintained privately. |
316317
| `TREG_RESEND_API_KEY` / `TREG_EMAIL_FROM` | *(empty)* | transactional email via Resend (OTP codes + invites); From must be a Resend-verified sender |
317318
| `TREG_BLOCKED_EMAIL_DOMAINS` | *(empty)* | comma-separated email domains refused at every sign-up/sign-in door and at team creation (subdomains included, case-insensitive). Empty blocks nothing — no list ships in the code |
318319
| `TREG_ADMIN_TOKEN` | *(empty)* | cross-tenant **super-admin** bearer; authorizes every `/admin/*` endpoint. Empty disables the env path (only `is_superadmin` users reach `/admin`). Keep it long + secret. |

‎SECURITY.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -73,8 +73,8 @@ Honesty is part of the model. Two items are deliberately deferred:
7373

7474
1. **Server runs do not yet have filesystem/network isolation.** The resource limits above cap denial of
7575
service, but a full jail (a locked-down user + egress allow-list, like the local sandbox) requires a
76-
container deployment and is planned. On the reference deployment there is no on-disk secret file to
77-
read (the encryption key is an environment variable), and only allow-listed CLIs may run.
76+
container deployment and is planned. In the supported hosted configuration the encryption key is
77+
an environment variable rather than an on-disk secret file, and only allow-listed CLIs may run.
7878
2. **The CLI-login handshake is in-process.** The short-lived pairing state for `treg login` lives in the
7979
server process (it self-heals on retry and carries no rate-limit value). Running more than one server
8080
instance requires sticky routing for that one flow, or moving it to shared storage.

‎deploy/render.example.yaml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Generic Render Blueprint for a self-hosted registry.
2+
#
3+
# Copy this file into your own deployment repository and adjust names, region,
4+
# plans and environment variables. The treg.to production topology and settings
5+
# are intentionally maintained in the private treg-internal repository:
6+
# https://github.com/superdesigndev/treg-internal/blob/main/docs/production/deploy.md
7+
8+
databases:
9+
- name: registry-db
10+
databaseName: registry
11+
region: oregon
12+
plan: basic-256mb
13+
14+
services:
15+
- type: web
16+
name: registry
17+
runtime: python
18+
region: oregon
19+
plan: starter
20+
branch: main
21+
buildCommand: pip install ".[server]"
22+
preDeployCommand: python -m treg upgrade
23+
startCommand: python -m treg
24+
healthCheckPath: /meta
25+
envVars:
26+
- key: TREG_DATABASE_URL
27+
fromDatabase:
28+
name: registry-db
29+
property: connectionString
30+
- key: TREG_PUBLIC_URL
31+
value: https://registry.example.com
32+
- key: TREG_EMAIL_DEV_MODE
33+
value: "false"
34+
- key: TREG_SECRET_KEY
35+
sync: false
36+
- key: TREG_SESSION_SECRET
37+
sync: false
38+
- key: TREG_GITHUB_CLIENT_ID
39+
sync: false
40+
- key: TREG_GITHUB_CLIENT_SECRET
41+
sync: false
42+
- key: TREG_RESEND_API_KEY
43+
sync: false
44+
- key: TREG_EMAIL_FROM
45+
value: registry <no-reply@registry.example.com>

‎docs/context/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,7 +55,7 @@ covers (frontmatter `sources:`). Regenerate this index with
5555

5656
| Fragment | Status | Covers |
5757
|---|---|---|
58-
| [Provider capacity — knowing what treg's own vendor accounts have left](ops/capacity.md) | shipped (steps B–F built; production rollout = the shadow week, then TREG_OVERFLOW_MODE=on) | __init__.py, collectors.py, policy.py, sweep.py, … |
58+
| [Provider capacity — knowing what treg's own vendor accounts have left](ops/capacity.md) | shipped | __init__.py, collectors.py, policy.py, sweep.py, … |
5959
| [Running & deploying the server](ops/deploy.md) | shipped | pyproject.toml, __main__.py, maintenance.py, env.py, … |
6060

6161
## Reference

‎docs/context/architecture/archive.md‎

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -203,7 +203,8 @@ team already received. Failure evidence (`error_*`) is untouched and still admin
203203
Rows older than the migration have no link and cannot get an exact one (the key needs the query
204204
and body the audit row never kept); `scripts/backfill_call_archive_links.py` links them best-
205205
effort — same endpoint, same byte size, fetch within ±10 s, unambiguous in both directions —
206-
dry run by default, `--apply` to write, `--render` for the prod allowlist dance.
206+
dry run by default and requires `--apply` to write. Deployment-specific execution belongs in the
207+
operator runbook.
207208

208209
## Result admission
209210

@@ -377,7 +378,8 @@ would have it stored — judge `cache` per provider with that in mind.
377378
`TREG_ARCHIVE_MODE` (config `archive_mode`, default `off`) → `archive.mode()`:
378379
`off` | `shadow` (record + learn, serve nothing — phase 0) | `serve` (shadow + answer eligible
379380
fresh hits — phase 1+). Any unrecognized value degrades to `off`: a typo must disable, never
380-
enable. Rollback in production is a dashboard env edit, no deploy.
381+
enable. Rollback is an environment-setting change through the deployment's normal configuration
382+
process.
381383

382384
## Conservative comparison and controlled serving (2026-09-08)
383385

@@ -550,19 +552,18 @@ loop-bound-semaphore pattern (four until 2026-09-07; every slot is paid per uvic
550552
again per rolling-deploy instance, and a recording is one INSERT of a body already in memory).
551553
Before it, a burst could put up to 512 concurrent short sessions in front of the API's 15-slot pool
552554
(SToneX's pool-pressure report); those writes now land on the BACKGROUND pool instead
553-
(`ops/deploy.md` § Three pools), so the semaphore is the inner bound rather than the only one.
555+
(`ops/deploy.md` § Database pools), so the semaphore is the inner bound rather than the only one.
554556
Queued recordings wait inside their fire-and-forget task, so the caller is unaffected; the 30s
555557
bound covers wait+write, so a stuck queue still sheds rather than wedges. Throttled, not shed: the
556558
burst test proves all 12 concurrent recordings land while peak DB concurrency stays ≤2.
557559

558-
**Memory bound (2026-09-07 OOM fix).** Each pending task holds its `body` bytes in a closure — up to
559-
`_MAX_PENDING` (512) tasks × `archive_max_body_bytes` (2 MB) = 1 GB worst case. After #363 reduced
560-
concurrent writes from 4 to 2, backlog built faster under heavy traffic and the 2026-09-07T00:43:06Z
561-
OOM killed production at 4 GB. `_MAX_PENDING_BYTES` (256 MiB) caps body bytes in DB
560+
**Memory bound.** Each pending task holds its `body` bytes in a closure, so task count alone is not a
561+
sufficient memory bound. `_MAX_PENDING_BYTES` (256 MiB) caps body bytes in DB
562562
pending work: `record()` sheds when EITHER the task count OR the bytes threshold is exceeded. The
563563
done callback releases bytes when a task completes, keeping the budget accurate. The independent
564564
R2 queue adds 128 MiB by default, for a combined 384 MiB body budget before SDK, compression
565-
and terminal-evidence overhead.
565+
and terminal-evidence overhead. Production incident evidence and deployment sizing are maintained in
566+
the private operator runbook.
566567

567568
The semaphore is process-local; the recorder also supports deployment with multiple processes. An exact in-process key
568569
lock is acquired before the semaphore, so duplicate recordings queue without consuming both

‎docs/context/architecture/auth-secrets.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -314,8 +314,8 @@ module symbols:
314314
this. No commit changed, no test failed (nothing in the suite makes a live call), and the two failure
315315
modes read differently: a version that **never existed** returns an HTML 404, a **sunset** one returns
316316
a JSON 400 `UNSUPPORTED_VERSION`. `POST /health/run` would surface it on the day it breaks — it probes
317-
every credential through the same versioned `probe_path` — but nothing schedules it; `render.yaml`
318-
carries only Render's own `healthCheckPath: /meta`. Bump the version in all four places together:
317+
every credential through the same versioned `probe_path`, but nothing schedules it by default.
318+
Operators may add a health worker to their own deployment. Bump the version in all four places together:
319319
`oauth_providers.GOOGLE_ADS`, `catalog/google-ads.yaml`, `catalog/google-ads.extended.yaml`, and
320320
`scripts/catalog_ingest.py:GADS_VERSION`.
321321

‎docs/context/architecture/contactout.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,9 @@ related:
3131
`token` header and verifies via free `GET /v1/stats`, requiring both HTTP success and
3232
`status_code: 200`. A zero allowance does not invalidate a credential. The existing credential
3333
ladder makes an org's tool/key win over the platform key and bypass treg billing and capacity checks.
34-
`Settings.platform_key_contactout` reads `TREG_PLATFORM_KEY_CONTACTOUT`; `render.yaml` declares
35-
the server slot and forwards it to the worker. The existing platform provider allow-list still
36-
controls serving. No credential is committed or copied into a platform Secret row.
34+
`Settings.platform_key_contactout` reads `TREG_PLATFORM_KEY_CONTACTOUT`. A deployment supplies the
35+
secret to the server and capacity worker. The existing platform provider allow-list still controls
36+
serving. No credential is committed or copied into a platform Secret row.
3737

3838
## Surface and selectors
3939

‎docs/context/architecture/data-model.md‎

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -183,17 +183,16 @@ uses this metadata, never the encrypted token's shape.
183183
Poll rows remain available by call reference and in admin diagnostics, but `/calls` excludes
184184
them before pagination. No migration or historical reclassification is required.
185185

186-
**Its indexes are the platform's throughput.** It is the largest table (2.94M rows / 1.68 GB on
187-
prod 2026-09-06) and every question asked of it is "… since <time>", so a `created_at` that no
186+
**Its indexes are the platform's throughput.** It is the largest table and every time-window
187+
question needs a compatible `created_at` index. Without one,
188188
index carried meant the planner chose an index for the other column and filtered the date in
189-
memory - reading an endpoint's or an org's WHOLE history to answer a 30-day one. Revision 0020
190-
adds `(endpoint_id, created_at)` for the catalog observation refresh (`domain/catalog/stats.py`,
191-
which had read 1.60 BILLION tuples across 570k scans) and `(org_id, created_at)` for the
192-
per-member daily counts (`routers/orgs.py`, 295M across 70k); 0016 already pairs
189+
memory, reading an endpoint's or an org's whole history to answer a bounded one. Revision 0020
190+
adds `(endpoint_id, created_at)` for the catalog observation refresh and `(org_id, created_at)`
191+
for the per-member daily counts; 0016 already pairs
193192
`(endpoint_id, id)` for the newest-N feed and 0012 a partial index on `cached`. The cost of
194193
getting this wrong is not a slow page: all three connection pools share one Postgres, so a scan
195194
here queues every other query and the API pool empties into `503 treg_saturated` - see
196-
[deploy](../ops/deploy.md) § Three pools. The table has no retention sweep yet, so it only grows.
195+
[deploy](../ops/deploy.md) § Database pools. The table has no retention sweep yet, so it only grows.
197196

198197
**`LedgerEntry` is the other one, and it was the larger.** It is append-only and never pruned
199198
(4.38M rows / 2.3 GB on prod 2026-09-06, ~400k rows a day), and `ledger.spent_today` - the
@@ -337,7 +336,7 @@ The API builds a single-binding tool from flat fields via `_flat_binding()`; inj
337336
Three async SQLAlchemy engines against one database, declared by `POOL_SPECS` and exposed as
338337
`session_maker` (api), `admin_session_maker` (`/admin/*`) and `background_session_maker` (audit,
339338
archive writes, the ads worker) - a bulkhead, so no class of work can exhaust another's slots; sizes,
340-
statement timeouts and the reasoning are in [deploy](../ops/deploy.md) § Three pools. On SQLite all
339+
statement timeouts and the reasoning are in [deploy](../ops/deploy.md) § Database pools. On SQLite all
341340
three alias one engine. The post-relay bookkeeping steps of `/call/` use `session_maker`; the request
342341
session is committed before the relay so none of them ever waits on it, see
343342
[proxy-model](proxy-model.md) § Connection discipline. The public

0 commit comments

Comments
 (0)