# Relay Extension + Python Receiver — Complete Documentation

Real-time capture & dispatch system for Amazon Relay loadboard. Two components: Chrome MV3 extension + Python receiver (FastAPI + WebSocket).

**Date:** 2026-05-25 · [Romanian version](DOCS.md)

---

## 1. Overall architecture

```
┌─────────────────────────────────────────────────────────────────┐
│                          BROWSER                                │
│                                                                 │
│  Amazon Relay tab (relay.amazon.com / mock relay-api.myvio.eu)  │
│  ┌──────────────────┐  ┌──────────────────┐  ┌──────────────┐ │
│  │  PAGE world      │  │  ISOLATED world  │  │  SERVICE     │ │
│  │                  │  │                  │  │  WORKER      │ │
│  │  injected.js     │←→│ content_script   │←→│ background   │ │
│  │  (fetch/XHR      │  │ (bridge IPC)     │  │ .js (WS conn │ │
│  │   monkey-patch,  │  │                  │  │  + polling)  │ │
│  │   UI inject,     │  │                  │  │              │ │
│  │   book click)    │  │                  │  │              │ │
│  └──────────────────┘  └──────────────────┘  └──────┬───────┘ │
└──────────────────────────────────────────────────────┼─────────┘
                                                       │ WebSocket
                                                       ↓
┌─────────────────────────────────────────────────────────────────┐
│                  PYTHON RECEIVER (FastAPI)                      │
│                  HOST: 192.168.0.31:9001 (typical)              │
│                                                                 │
│  ws://...:9001/ws/stream                                        │
│    ├─ EXT_CLIENTS   (extensions pushing captures)               │
│    └─ DISPATCHERS   (browser pages receiving broadcasts)        │
│                                                                 │
│  HTTP /admin/* (stats, insights, raw-ring, reports, ...)        │
│                                                                 │
│  Memory:                                                        │
│    POOL{}            — active tours (deduplicated)              │
│    LAST_SEEN{}       — TTL per tour (grace period cleanup)      │
│    POLL_HISTORY      — last 200 captures metadata               │
│    LATENCIES         — last 100 ext→recv ms                     │
│    LATENCIES_OUT     — last 100 recv→UI ms (browser reports)    │
│    RAW_RING          — last N raw JSON bodies (configurable)    │
│    reports_db (SQLite) — book history, snapshots, events        │
└──────────────────────┬──────────────────────────────────────────┘
                       │ WS broadcast
                       ↓
                ┌─────────────────────────────────────┐
                │  BOOKER-ADMIN UI (relay_tours,      │
                │  relay_ws_sources, mock_tuning)     │
                │  Multiple dispatcher connections    │
                └─────────────────────────────────────┘
```

---

## 2. Chrome MV3 Extension

### 2.1 File structure

| File | World | Role |
|---|---|---|
| `manifest.json` | — | Extension declarations (perms, content_scripts, SW) |
| `background.js` | Service Worker | Persistent WS connection, polling scheduler, message router |
| `content_script.js` | ISOLATED | Bridge between page world and service worker (access to `chrome.*` API) |
| `injected.js` | PAGE | Monkey-patch fetch/XHR, capture body+headers, UI inject, book click |
| `popup.html` + `popup.js` | popup | User settings (rate, jitter, target host, pull mode, etc) |
| `book_unlock.json` | filesystem | **Safety gate**: if missing/false → all book attempts blocked |
| `assets/audio/*.mp3` | — | Sounds for new/price/booked alerts |

### 2.2 Capture flow (passive)

1. **Amazon page JS** calls `fetch('/api/loadboard/search', {...})`
2. **`injected.js` window.fetch override** intercepts:
   - Pre-cache: clone request → save body + headers (including `x-csrf-token`)
   - Then `await origFetch.apply(this, args)` → real request goes to Amazon
   - Measures `fetch_ms = performance.now() - t0` (local, no telemetry)
   - Clone response → `send()`
3. **`send()`** dispatches `window.postMessage({__relay_capture: true, body, ts, fetch_ms, ...}, "*")`
4. **`content_script.js`** listens to `window.message` → forwards to background via `chrome.runtime.sendMessage`
5. **`background.js`** sends payload over WebSocket to receiver (or queues if WS offline)

### 2.3 Polling flow (active)

1. **`background.js` timer** (configurable from popup) sends `trigger_active_fetch` to each eligible tab
2. **`content_script.js`** forwards via `postMessage({__relay_active: true, pull_via_ui_click, ...})`
3. **`injected.js`** handler:
   - **`pull_via_ui_click=true` (DEFAULT, Rocket Relay style)**: finds Refresh button via `#utility-bar .refresh-and-chat-box button` + SVG path filter `M20.128 2`, then SIMPLE `btn.click()` (NOT MouseEvent dispatchEvent) — Amazon React responds correctly. **NO fallback to direct fetch.**
   - **`pull_via_ui_click=false`**: direct doFetch to cached URL
4. Response arrives via passive flow (mock/Amazon performs fetch in response to click → normal capture)

### 2.4 Book flow (DOM click style)

All 10 gates must pass, otherwise book is blocked:

1. `book_unlock.json` exists + `enabled: true`
2. Popup toggle "Book ENABLED" = ON
3. `BOOK_HARDCODED_CONFIG.enabled = true` (in code)
4. Tour ID locked (`__relay_book_in_progress`)
5. Card exists in DOM (verified at click time)
6. URL fetch verify (URL contains EXACT tour_id from lock)
7. `dom_click_mode = true`
8. `fetch_url_verify = true`
9. **LEVEL 4 global fetch block**: any `/api/loadboard/{tid}/{v}/option/{oid}/majorVersion/{mv}` URL not passing gates → fake response with `errorCode: work_opportunity_not_available` (native Amazon code, no suspicious custom string)
10. **LEVEL 4 global XHR block**: same for XMLHttpRequest

### 2.5 Popup configuration

| Setting | Default | Effect |
|---|---|---|
| `enabled` | true | Master toggle — OFF = nothing is sent to receiver |
| `active_mode` | false | ON = auto polling at `poll_rate` |
| `poll_rate` | 2.0s | Base interval between polls |
| `poll_jitter` | 0 | ± random for anti-bot rate signature |
| `pull_via_ui_click` | **true** | Click on Refresh button (native telemetry) — NO fetch fallback |
| `poll_only_when_active` | false | Polls only active + focused tab |
| `target_host` | auto | Tab filter (relay.amazon.es/de/it/fr/uk/com, relay-api.myvio.eu) |
| `override_result_size` | 0 | Force `resultSize` in body (0 = leave natural) |
| `auto_paginate` | true | Aggregate pages via nextItemToken up to 20 pages |
| `alerts_enabled` | false | Move-to-top + CSS highlight on new tours |
| `alerts_audio_on` | true | Sounds (new.mp3, price.mp3, successbook.mp3) |
| `simulation_mode` | false | Dry-run book: highlight 15s + sound, NO real click |
| `fast_book_button_enabled` | false | Injects "🚀 Fast Book" button on each card |
| `book_user_enabled` | false | Popup toggle for authorized book |

---

## 3. Python Receiver

### 3.1 File structure

| File | Role |
|---|---|
| `receiver.py` | Main FastAPI app — WS + HTTP endpoints |
| `config.py` | Env vars + .env loader (no external deps) |
| `book_authorization.py` | Gate file for live book (lazy import) |
| `book_unlock.json` | Unlock status per receiver (separate from extension) |
| `reports_db.py` | Local SQLite for book history, snapshots, event log |
| `captures/*.json` | Optional disk dump (if `SAVE_RAW_ENABLED=true`) |

### 3.2 HTTP endpoints

| Endpoint | Returns | Used for |
|---|---|---|
| `GET /` | HTML dashboard | Quick view in browser |
| `GET /admin/stats` | tenant, target, pool_size, ws_clients, ext_clients, uptime, counters, **memory** (RSS, ring bytes) | Stats overview |
| `GET /admin/insights` | recent_polls (last 30), avg_latency_ms, avg_latency_out_ms, **recent_latency_out_ms**, polls_per_minute, **consecutive_fails**, **fail_auto_stop_threshold**, latest_pool | UI charts |
| `GET /admin/raw-ring/list` | Metadata for all ring items (no bodies) | UI Raw tab list |
| `GET /admin/raw-ring/zip?limit=N` | ZIP with last N JSONs + `_manifest.json` | Bulk export download |
| `GET /admin/raw-ring/{idx}` | Full body for one item | UI preview |
| `GET /admin/last-response` | Last captured response | Quick inspection |
| `GET /admin/reports/book-history` | SQLite DB — book history |  |
| `GET /admin/reports/snapshots` | SQLite DB — hourly snapshots |  |
| `GET /admin/reports/events` | SQLite DB — event log (info/warn/error) |  |

### 3.3 WebSocket endpoints

| Path | Roles |
|---|---|
| `WS /ws/stream` | Multiplex — ext (push captures) or dispatcher (recv broadcasts) after first `identify` message |

**Messages ext→receiver:**
- `loadboard_response` — full capture (url, status, body, ts, fetch_ms, source)
- `ext_status` — status reported by extension (poll rate, active flag, errors)
- `book_result` — book result for audit

**Messages receiver→dispatcher:**
- `add` / `update` — new tour or version bump
- `gone` — expired tour (TTL grace period)
- `ext_status` — broadcast to dispatchers (UI)

**Messages dispatcher→receiver:**
- `identify` — role declaration
- `latency_report` — UI reports latency `Date.now() - msg.__server_ts`
- `ext_command` — remote control (pause/resume polling, change rate)

### 3.4 Environment variables (config.py)

| Env Var | Default | Effect |
|---|---|---|
| `RECEIVER_PORT` | 9001 | HTTP+WS port |
| `RECEIVER_HOST` | 0.0.0.0 | Bind address |
| `GRACE_PERIOD_S` | 1.5 | How long unseen tour stays in POOL before TTL cleanup |
| `POOL_MAX_SIZE` | 0 | LRU cap on POOL (0 = unlimited) |
| `EXT_MAX_MSG_BYTES` | 10MB | WS message size limit from extension |
| `BOOK_UNLOCK_FILE` | `./book_unlock.json` | Path to gate file |
| `BOOK_AUDIT_ENABLED` | true | Log book attempts to SQLite |
| `SAVE_RAW_ENABLED` | false | Disk dump captures to `./captures/` |
| `SAVE_RAW_DIR` | `./captures` | Folder for dumps |
| `SAVE_RAW_MAX_FILES` | 5000 | FIFO rotation |
| **`RAW_RING_SIZE`** | **50** | Raw JSON ring buffer in RAM (1000 = ~140MB) |
| **`FAIL_AUTO_STOP_THRESHOLD`** | **0** | Auto-stop receiver after N consecutive fails (0 = OFF) |

### 3.5 Safety: book_unlock.json

```json
{
  "enabled": false,
  "token": "",
  "note": "Change enabled: true + reload to allow book"
}
```

If file **missing** OR `enabled !== true` → all book paths blocked (redundant gate with extension one). Optional token = additional proof.

---

## 4. What Amazon can detect — honest analysis

### 4.1 What Amazon SEES (their telemetry)

| Detectable | How | Mitigation |
|---|---|---|
| **HTTP requests to `/api/loadboard/search`** | Standard server-side logging | Unavoidable — user makes the request via UI anyway. Rate matters: 0.5s × 5 tabs = ~10 req/sec validated empirically on B_PLUS |
| **Normal headers** (cookie, csrf, UA) | Server log | Pass-through, unchanged by extension |
| **Native click events on buttons** (Reservar, Confirm, Refresh) | Pendo, csa.ContentInteraction, optimus | TRUSTED click events emitted by browser when user/extension does `btn.click()` — telemetry receives the event. GOOD for us: shows normal user activity. |
| **`window.fetch` wrapped** | Theoretically detectable via `fetch.toString()` | In practice, prod anti-bot doesn't do this. Rocket Relay runs for years without issues. |
| **Page stays open for long time** | Time-on-page metrics | Polling only on active + focused tab (`poll_only_when_active`) reduces signature |

### 4.2 What Amazon DOES NOT SEE

| Invisible | Why |
|---|---|
| **Internal postMessage messages** (`__relay_capture: true`) | Only if they had `window.addEventListener('message', ...)` with exact filtering on flag — they don't in prod |
| **`performance.now()` local measurements** | Stdlib function, doesn't generate network |
| **Body captured by extension** | Stays on device + WebSocket to your LAN |
| **WebSocket to receiver** (`ws://192.168.0.31:9001/`) | Private LAN, inaccessible from amazon.com origin |
| **DOM modifications** (move-to-top, added CSS classes) | Only if they have active MutationObserver — in prod they don't observe all DOM |
| **Programmatic clicks `btn.click()`** vs trusted clicks | Native event listeners fire identically. Only if React onClick checks `event.isTrusted` (rare). Rocket Relay confirms `btn.click()` works on real Amazon. |
| **Injected Fast Book button** | Extra element in DOM, but no API calls reporting "extension detected" |

### 4.3 Comparison with Rocket Relay (reverse-engineering audit)

The audit at `/var/www/booker-admin/audit/kcnegnbacaohnhpmeckfakhlnahliklf/8.91_0/` confirms:
- RR uses same pattern: `document.querySelector('.css-q7ppch')?.click()` for refresh
- RR doesn't emit any extra event to Amazon
- RR has been running since 2021 on thousands of users without public bans

**Conclusion:** our extension's risk profile = identical to Rocket Relay. The only significant "tell" would be aggressive polling RATE (over 2-3 req/sec sustained per account).

### 4.4 Bot signatures we avoid

- ❌ Click with coords `(0, 0)` → we use `realisticClick()` with real coords from `getBoundingClientRect` ± random jitter (for book buttons)
- ❌ Constant robotic polling → `poll_jitter` configurable for interval randomization
- ❌ Polling on inactive tab → `poll_only_when_active`
- ❌ Requests with stripped headers → we preserve ALL headers from cached request (cookie, csrf, accept-language, etc)

---

## 5. End-to-end setup

### 5.1 Extension setup

1. **Load unpacked** from `chrome://extensions` → "Load unpacked" → select `chrome-extension` folder
2. **`book_unlock.json`** next to manifest:
   ```json
   {"enabled": true, "token": "some-secret"}
   ```
   Restart extension (chrome://extensions → 🔄) after modification.
3. **Popup**: set `Receiver URL` = `ws://192.168.0.31:9001/ws/stream`
4. Check `Active mode` + `Pull via UI click` (default ON)
5. Save

### 5.2 Receiver setup

```bash
cd python-receiver/app
python3 -m venv venv && source venv/bin/activate
pip install fastapi uvicorn  # + optional: psutil

# .env (next to receiver.py)
RECEIVER_PORT=9001
RAW_RING_SIZE=1000
FAIL_AUTO_STOP_THRESHOLD=20
GRACE_PERIOD_S=1.5

# book_unlock.json
{"enabled": true, "token": ""}

# Start
python3 receiver.py
```

On **Windows**:
- Cross-platform RSS lookup via ctypes doesn't require psutil
- Use `pythonw.exe receiver.py` to run in background without console

### 5.3 Booker-admin UI setup

```sql
INSERT INTO relay_ws_sources (user_id, name, url, active)
VALUES (1, 'LocalReceiver', 'ws://192.168.0.31:9001/ws/stream', 1);
```

Then on `/relay_ws_sources` → click stats icon → modal with:
- 6 live charts (status pie, ext→recv latency, body size, polls/min, fetch ms, recv→UI ms)
- "Last captures" tab + "Raw JSONs in RAM" tab
- "Download ZIP" button for last 50/100/500/1000

---

## 6. Pros / Cons by design

### ✅ Pros

| Aspect | Detail |
|---|---|
| **PAGE/ISOLATED/SW decoupling** | Strictly respects MV3 model, no `chrome.*` access from PAGE world |
| **Cross-tab orchestration** | Background SW has global view — book tab selection based on who has the tour in DOM |
| **CSRF auto-capture** | Cached from real fetch, never hardcoded |
| **Multi-host support** | `target_host: auto` covers es/de/it/fr/uk/com + mock myvio.eu |
| **Pull via UI click default** | Natural telemetry, indistinguishable from real user |
| **10-gate safety chain for book** | Defense in depth — book_unlock.json + popup + hardcoded + DOM verify + URL verify + LEVEL 4 fetch/XHR block |
| **Receiver = single source of truth** | Multiple dispatchers can consume simultaneously (UI, agent, etc) |
| **Reports DB** | Local SQLite for retroactive audit (book history, snapshots, events) |
| **Raw ring in RAM + ZIP export** | Live debug without disk overhead, downloadable anytime |
| **Auto-stop on consecutive fails** | Circuit breaker when Amazon blocks us |
| **Multi-platform memory monitoring** | Linux /proc, Windows ctypes, macOS resource — no dependencies |

### ⚠️ Cons / Trade-offs

| Aspect | Why it's a trade-off | Mitigation |
|---|---|---|
| **MV3 service worker can sleep** | Chrome stops SW after 30s idle | `chrome.alarms` keepalive at 20s + `rescheduleActiveFetch()` on each wake |
| **Fetch wrapper detectable** via `fetch.toString()` | Anti-bot can theoretically check | In practice doesn't happen (Rocket Relay confirms) |
| **No fallback to direct fetch** when pull_via_ui_click ON | If click fails, refresh doesn't happen | Fail loud — log to console, user sees immediately. Trade-off: zero risk of bot signature from fetch without telemetry |
| **Receiver is single point of failure** | If it crashes → all dispatchers blind | Queue in background.js (200 messages) + reconnect with backoff |
| **RAW_RING is volatile** | Lost on restart | Enable `SAVE_RAW_ENABLED` for disk persistence + FIFO rotation |
| **CSRF expires on mock container restart** | `VALID_SESSIONS` in memory | F5 page for new session; for prod persistence, need SQLite/Redis |
| **Fixed polling rate per receiver** | All tabs share same rate | Per-tab rate not supported currently |
| **Mixed content blocking** | Browser HTTPS can't fetch HTTP receiver | Server-side PHP proxy (already implemented in relay_ws_sources_api.php) |
| **WebSocket scaling** | One receiver = one Python process | For multi-tenant scaling need sticky session / sharding |
| **No backpressure** | If receiver is overloaded, ext keeps sending | Acceptable for 1-5 ext clients; for 100+ need rate limiting |

### 🔍 Compared to alternatives

| Strategy | Pro | Con |
|---|---|---|
| **Direct fetch from extension** (no UI click) | Faster (~50ms) | Bot signature: no Pendo telemetry, missing csa.ContentInteraction events |
| **Pull via UI click** (CURRENT) | Natural telemetry, indistinguishable from user | ~100-200ms overhead per refresh (extra round-trip through React lifecycle) |
| **Headless puppeteer external** | Cross-browser, scriptable | Detectable via `navigator.webdriver`, headless flag |
| **MITM proxy (mitmproxy / Burp)** | Total request/response access | Manual configured, not portable |

---

## 7. Operations

### 7.1 Deploy

**Extension** (Chrome dev mode):
1. Modify files locally
2. `chrome://extensions` → 🔄 Reload "Relay Tour Catcher"
3. **F5 on open tabs** (content scripts don't re-inject on extension reload!)

**Receiver** (Windows 192.168.0.31):
```bash
# Stop current
# (taskkill /F /IM python.exe or Ctrl+C in console)
# Copy new files
scp receiver.py user@192.168.0.31:/path/to/app/
# Restart
python receiver.py
```

**Booker-admin UI**:
- Edit `templates/relay_ws_sources/*.html` and `plib/relay/relay_ws_sources_api/*.php` locally
- Clear Smarty cache: `rm /var/www/booker-admin/templates_c/*relay_ws_sources*`
- F5 page

### 7.2 Debug

| Issue | Check |
|---|---|
| Extension not sending to receiver | Service worker logs (`chrome://extensions` → Relay → "service worker") |
| Polling doesn't start | In popup: `Active mode` ON? Save pressed? `pull_via_ui_click` ON? |
| "Receiver process —" | Deploy latest receiver.py (ctypes Windows fallback) |
| "stale" in health | `LAST_POLL_AT` updates only on status=200; check `consecutive_fails` |
| Mixed content errors | Browser HTTPS can't `http://192.168.0.31`. Use PHP proxy (`q=stats&id=X`) |
| Corrupt ZIP | Check PHP `ob_end_clean()` + headers `Content-Encoding: identity` |
| 422 on `/admin/raw-ring/zip` | FastAPI route order — `/zip` must be BEFORE `/{idx}` |

### 7.3 Monitoring

- **`/admin/stats`** — pool size, ext_clients, uptime, RAM RSS
- **`/admin/insights`** — recent polls, latency avg + time series, consecutive_fails
- **Reports DB** — `/admin/reports/events?level=error` for historical issues
- **UI charts in stats modal** — live 1s refresh visualization

---

## 8. Known limitations

1. **Auto-paginate max 20 pages** — if user requests `resultSize=50` and there are 2000 tours, stops at 1000 (20 × 50)
2. **POLL_HISTORY in-memory** — lost on receiver restart
3. **RAW_RING in-memory** — same
4. **SQLite Reports DB** — single-writer; for concurrent receivers need PostgreSQL
5. **Single book click fallback** — DOM click; direct fetch disabled by design (risky)
6. **Dynamic CSRF generation** at mock — token expires on container restart

---

## 9. Glossary

- **Tour / WorkOpportunity (WO)** — an Amazon Relay transport job
- **Pool** — set of active tours in receiver memory
- **Grace period** — TTL after which an unseen tour is removed from pool
- **Dispatcher** — any WS client consuming broadcasts (UI, agent, etc)
- **Capture** — an intercepted `/api/loadboard/search` response
- **Passive** — capture from page's own Amazon fetch
- **Active** — capture from extension's scheduled trigger
- **Round complete** — response with `nextItemToken=0` (last page of a sequence)
- **Fast Book** — extension-injected button on each tour card for one-click book
- **Simulation mode** — dry-run: highlight + sound, no real Reservar click

---

## 10. Backups

All major changes have backups in `/var/www/booker-admin/backups/`:
- `relay_stats_20260525_020909/` — before adding charts + raw ring + ZIP export

For full rollback: copy from backup in place.