> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cfosilvia.com/llms.txt
> Use this file to discover all available pages before exploring further.

# DR PDF OVER LINQ DESIGN

# DR Report PDF over iMessage (via Linq) — Design Doc

> Status: **Phase 2 in progress.** Phase 1 shipped in [PR #3923](https://github.com/CFO-Silvia/Silvia-Web/pull/3923) / cherry-picked to main in [PR #3924](https://github.com/CFO-Silvia/Silvia-Web/pull/3924) — pre-signed S3 URL helper + `silvia_sms_logs.user_context` observability types + `LINQ_DR_PDF_ATTACH_CLERK_IDS` allow-list, all shipping OFF (scaffolding, no send-site imports yet). Phase 2 wires the gate into `app/api/sms/research-complete/route.ts` behind the same flag.
>
> **Doc correction applied in Phase 2 (§6a rewrite):** the original §6a proposed a per-line ZDR fetch via `GET /v3/phone_numbers/:id` + a 15-min in-memory cache + webhook invalidation. That mechanism was wrong — Linq confirmed there is no per-line ZDR endpoint; the sole source of truth is the `zero_retention: boolean` field on each webhook event (see the extended docstring on `NormalizedInboundMessage.zeroRetention` in `lib/messaging/types.ts`). Phase 2 reads that observation off `silvia_sms_logs.user_context.zero_retention` from the user's most recent inbound Linq event, which is per-line-correct (the outbound lands on the same physical line as the recent inbound) and requires zero extra API calls.

## Goal

When a Deep Research run completes and we fire the completion iMessage bubble sequence via Linq, **attach the actual PDF report** to the message so the user gets the full research delivered in-thread — not just a link.

## Why this is worth doing

* **UX**: today the DR completion is a text bubble + URL bubble (OG preview card). User has to tap through, wait for the web app to load, sign in if signed out, wait for the report to render. With the PDF attached, the report is one tap away in Files / Preview.
* **iMessage-native pattern**: this is exactly what a human financial advisor sending a report over iMessage does. Full parity with the "feels like texting a person" pitch we made when moving to Linq.
* **Offline / low-signal**: the PDF stays in the user's Files even without internet. Web link doesn't.
* **Reduces load on `/research`**: fewer sessions cold-open the web app when the user just wanted to skim the report.

## Prior art in the codebase

* **Linq media parts**: `SendMessageInput.mediaUrls?: string[]` at `lib/messaging/types.ts:138` already exists and is threaded through the Linq provider's `buildParts()` at `lib/messaging/providers/linq.ts:115-129`. Plumbing is done — no adapter changes required.
* **PDF storage**: new DR sessions store `report.pdf` in S3 at `s3://prod-deep-research/{env}/{clerk_id}/{date}/{session_id}/data/report.pdf`. The `GET /api/report/download` route already knows how to resolve those paths.
* **Client-side PDF generator**: `lib/research/markdown-to-pdf.ts` renders legacy sessions' inline markdown to PDF in the browser via `jsPDF`. Not usable server-side without a Node-compatible refactor or a headless-browser wrapper.

## What Linq supports (per [Linq's sending-messages docs](https://docs.linqapp.com/guides/messaging/sending-messages/))

* **Up to 100 parts per message**, **up to 40 public-URL media parts** per message.
* Media URL must be **fetchable by Linq's servers** (they download the file themselves and re-deliver it). Auth-gated URLs (Clerk / signed cookies) will NOT work.
* **iMessage / RCS**: PDFs render as inline attachments with a tap-to-open link into Files / Preview. Handles up to \~100 MB.
* **SMS fallback (MMS)**: carriers commonly cap at 500 KB–1 MB, many outright reject PDFs, some strip attachments silently. Not a reliable channel for PDFs.

## Constraints & open questions

### 1. Authentication for Linq's fetch

`/api/report/download` today gates on Clerk auth (`auth()` guard, returns 401 for unauth). Linq's servers won't send Clerk cookies. Options:

| Option | Pros | Cons |
| - | - | - |
| **A. Pre-signed S3 URL** with short TTL (e.g. 24h) | S3-native, no new endpoint, encodes access control in the URL itself | Exposes bucket path structure to Linq (minor); we sign 1x per Lambda call so retry must produce a deterministic URL |
| **B. New public-token endpoint** `/api/research/pdf/[session]/[short-lived-token]` | Full control over access rules (revocation, per-session tokens, IP allow-list for Linq's crawlers) | New endpoint + token store; more moving parts |

**Recommendation: A (pre-signed S3 URL)** for the first cut. Simpler, uses existing infra, and the URL rot after TTL is a feature (matches ZDR posture — Linq should download once and rely on their own retention).

#### 1a. TTL vs. retry window (CodeRabbit PR #3902)

The pre-signed URL's TTL must be **strictly greater than the maximum end-to-end retry window** for a single DR completion, otherwise a Lambda retry after cron reclaim can present Linq with an expired URL and the PDF silently drops.

Current retry math:

| Stage | Bound |
| - | - |
| Lambda `dr_sms_notifications` claim lease | 2 min (see the 503 reclaim path in `app/api/sms/research-complete/route.ts`) |
| Cron reclaim cadence | 5 min |
| Max sane retry attempts before we give up | 6 (matches the DR-complete route's `attempts` cap) |
| **End-to-end worst case** | **\~42 min** (6 attempts × 7 min per cycle, wall-clock) |

Two viable ways to make the URL outlive that window:

* **A1 (recommended)**: Sign with a fixed **24 h TTL**, using a deterministic timestamp anchor (e.g. `sessionId` or the DR run's `started_at` timestamp) so retries reproduce the exact same URL string. Idempotency stays intact (see §5), and 24 h ≫ 42 min gives ample headroom for cron backfill anomalies.
* **A2**: Sign **once** on the first attempt (not per-attempt) with a TTL that must still exceed the 42-min retry window — proposed floor **60 min**, safer choice **2 h** for cron-backfill anomalies — and persist the exact URL string in `dr_sms_notifications.attachment_url` so subsequent retries reuse the persisted value verbatim. The naming "per-attempt signing" from the previous revision was misleading (CodeRabbit PR #3902 flagged the ambiguity + the too-short 30-min example): the sign call runs once at attempt #1 and the retries just read the column. Adds a schema column + one write, so A1 is preferred unless we later need per-URL revocation windows shorter than a day.

Do NOT sign with `Date.now()` — non-deterministic across retries, breaks §5's idempotency contract and leaks a fresh URL per retry (defeats "Linq downloads once").

### 2. iMessage-only (RCS is NOT sufficient)

MMS fallback with a multi-MB PDF is going to fail on most carriers. Even RCS is not safe here — [Linq's protocol-selection docs](https://docs.linqapp.com/guides/messaging/sending-messages/) call out that `preferred_service: 'RCS'` falls back to SMS when RCS isn't available for the recipient, and there's no "RCS-only" mode. So the gate must be **iMessage-capable, right now** — not "iMessage or RCS", and not "iMessage at some point in the past".

Options considered:

| Option | Pros | Cons |
| - | - | - |
| **A. Attach unconditionally** | Simple | Any non-iMessage delivery drops the attachment or breaks it (MMS strips PDFs, RCS-then-SMS drops them) |
| **B. Historical-channel heuristic** — check `silvia_sms_logs.user_context.service` for this user's last N inbound events; only attach if ≥80% were `iMessage` | Uses data we already have, no extra Linq round-trip | Stale — user changed carriers / lost iCloud / ported number; the heuristic will still say "iMessage" and Linq will fall back to SMS/MMS |
| **C. Live iMessage capability check** — call [`POST /v3/capability/check_imessage`](https://docs.linqapp.com/channel/imessage/guides/chats/capability-checks/) BEFORE the send; if positive, send with `preferred_service: 'iMessage'`. If check returns negative OR the capability endpoint errors, skip PDF and send URL-only. | Definitive at send-time; `preferred_service: 'iMessage'` fails the send (rather than silently falling back) if the recipient becomes non-iMessage between check + send | Extra Linq API round-trip per DR completion (\~100–500/day, well within our rate budget); introduces `preferred_service` as a new pattern on this route (radars use `forceManagedLine` today, but the two are orthogonal) |
| **D. Dual-bubble** — URL-only + PDF-attached, sender decides per-recipient | Universal | Doubles API calls, and the PDF bubble still needs Option C's gate to decide whether to fire at all |

**Recommendation: C (live capability check + `preferred_service: 'iMessage'`)** — CodeRabbit PR #3902 correctly flagged Option B (the historical heuristic) as unsafe because Linq's runtime protocol selection can diverge from any inbound record we've stored.

The B heuristic remains useful as a **pre-filter to skip the capability API call** for users we're confident aren't on iMessage (e.g. we've never seen an `iMessage` inbound from them). This trims the API-call volume without weakening the gate:

```
if (historicalChannel !== 'iMessage')          → skip PDF (URL-only)
else:
  cap = await linq.checkImessage(recipient)
  if (cap.eligible)  → attach PDF, preferred_service: 'iMessage'
  else               → skip PDF (URL-only)
```

Any capability-check error (network, 5xx, timeout) falls into the `URL-only` branch — same safe-by-default posture as §6's ZDR gate.

### 3. Legacy sessions (no server PDF)

Legacy DR sessions store markdown in DynamoDB and render the PDF in the browser only. Options:

| Option | Pros | Cons |
| - | - | - |
| **A. Skip PDF for legacy sessions** (URL bubble only, unchanged) | No new dependencies | Legacy users see two-tier UX |
| **B. Server-side markdown → PDF** using a Node-compatible library (e.g. `puppeteer` headless, `@react-pdf/renderer`, or refactor `markdown-to-pdf.ts` to run in Node) | Full coverage | New heavy dependency (Puppeteer alone is 200MB+ Chromium; @react-pdf is 2MB+ but requires rewriting the renderer); Vercel serverless has a 250MB unzipped code limit — Puppeteer needs `chrome-aws-lambda` or the Sparticuz variant |
| **C. Trigger the existing client-side generator on the web app and cache the result to S3** (out-of-band job) | Reuses existing renderer | Requires a headed browser somewhere (existing `markdown-to-pdf.ts` uses browser DOM); adds a job / worker |

**Recommendation: A (skip legacy)** for the first cut. Legacy sessions are a shrinking corpus; new sessions get PDF-in-thread; legacy sessions keep the existing URL bubble. Revisit if legacy usage stays high.

### 4. Bubble sequence

Today's Linq DR completion is:

1. **Bubble 1**: `"Your deep research is ready."` (text, self-contained)
2. **Bubble 2**: `https://www.cfosilvia.com/research?session=<id>&utm_source=sms` (bare URL for OG unfurl)

Proposed with PDF:

1. **Bubble 1 (unchanged)**: `"Your deep research is ready."` — plus `mediaUrls: [pdfUrl]` so the PDF renders as an attachment inside bubble 1 (Linq docs allow text + media in a single message as long as they're separate parts and consecutive text parts are absent — this is fine).
2. **Bubble 2 (unchanged)**: bare URL for OG unfurl.

**Why not a 3rd bubble for the PDF?** Linq docs say media parts and text parts can share a message (as separate parts). Attaching to bubble 1 is one fewer Linq API call.

**Why not put the PDF on bubble 2?** Linq docs say "Link parts must be the only part in a message — they render as a rich preview". Bubble 2 today is a text part containing a URL (which iMessage detects and unfurls) — but if we change it to a `link` part with a media sibling, we lose the rich preview per Linq's rule.

### 5. Idempotency

The DR completion is at-least-once (Lambda retries). Bubble 1 already deduped by `idempotencyKey`. The `mediaUrls` array must be **deterministic** across retries so the same idempotency key produces the same message content. Pre-signed URLs are deterministic if we sign with a **fixed timestamp** (e.g. run start time, or `sessionId`-derived) rather than `Date.now()`.

### 6. ZDR / retention

* **ZDR-enabled Linq lines**: Linq downloads the PDF, delivers once, discards. Aligns with our ZDR posture.
* **Non-ZDR lines**: Linq retains a copy per their own retention policy. If we haven't enabled ZDR on the delivery line yet, this exposes an internal PDF in Linq's storage.

#### 6a. Recent-inbound-observation ZDR gate

ZDR is a **per-line** property in Linq's model — see [Linq's ZDR docs](https://docs.linqapp.com/channel/imessage/guides/platform/zero-day-retention/). But there is **no phone-numbers endpoint that returns a line's current ZDR setting**: Linq's own docs (verified in code — see the `NormalizedInboundMessage.zeroRetention` docstring at `lib/messaging/types.ts:92-121`) state "There is no separate endpoint to query a phone number's retention setting directly — treat this field as the source of truth, per event". The **sole authoritative signal** is the `zero_retention: boolean` field Linq stamps on every affected webhook event.

We already persist that field on every inbound Linq event to `silvia_sms_logs.user_context.zero_retention` via `app/api/messages/incoming/linq/route.ts:301`. So the ZDR gate reads the **user's most recent inbound Linq observation** from Supabase — no additional Linq API call, and per-line-correct because Silvia's reply lands on the exact same physical Linq line as the received inbound (Linq's reply-to threading anchors the outbound to that line).

**Context: Linq confirmed account-wide ZDR is enabled for us.** So in practice every observation for our account should be `zero_retention: true`. This gate is a defense-in-depth check that catches the case where Linq flips ZDR off (whether by their action or by a per-line configuration drift) without our infra noticing — the observation stamp gives ops a real-time audit signal.

**Decision matrix** for the ZDR gate step (subordinate to the earlier feature-flag / legacy-session / capability-check gates — hitting any of those already routes to URL-only):

| Recent inbound observation | Action | Reasoning |
| - | - | - |
| Most recent obs is `zero_retention: true`, ≤ 30 days old | **Attach PDF** ✅ | Confirms ZDR is active on the physical line we're replying from |
| Most recent obs is `zero_retention: false` | URL-only, stamp `zdr_not_enabled` | Definitive: Linq flipped ZDR off on this line; do NOT put a financial-report PDF onto their storage |
| Most recent obs is `zero_retention: null` (Linq didn't send the field on that event) | Attach PDF, log `zdr_unknown` in user\_context | Field absence is not the same as `false` — happens on pre-ZDR-rollout events and never on current events. With account-wide ZDR on, this is safe. |
| No inbound event for this user in the last 30 days | Attach PDF, log `zdr_unobserved` | User's first outbound; account-wide ZDR is on so this is safe. The next inbound will stamp a real observation. |
| DB read error | URL-only, stamp `zdr_not_enabled` (safe-default) | Fail closed on infrastructure trouble |

**Staleness horizon**: 30 days is a soft ceiling on how old an observation is still "trusted". Rationale: (a) Linq's `phone_number.status_updated` webhook fires on the account-level ZDR flag toggle so a flip would produce a fresh event, (b) DR-completion isn't triggered by users who haven't texted Silvia in a month, so in practice we're always reading a very fresh observation, (c) 30 days > any credible operational window for a stale-observation false positive.

**Non-goal**: proactively enabling ZDR on lines, or checking every line in the account before every send. Both are addressed by Linq's platform-level ZDR being enabled account-wide.

**Why the original doc was wrong**: this section originally described a `GET /v3/phone_numbers/:id` fetch + 15-minute cache + `phone_number.status_updated` invalidation. That mechanism assumed a Linq endpoint that doesn't exist. The rewrite (Phase 2 build) uses the mechanism Linq's docs actually specify — see the code-comment cross-references above.

### 7. Cost

* **S3 egress**: negligible — Linq downloads once per delivery, \~2-5 MB per PDF, \~\$0.00025 per delivery at current S3 pricing.
* **Linq pricing**: per-line, not per-MB. No incremental cost.
* **Vercel**: no impact (pre-signing an S3 URL is a local CPU op, no network round-trip).

### 8. Observability

New `silvia_sms_logs.user_context` fields for Linq DR completion rows:

* `pdf_attached: boolean` — was a PDF media part included on bubble 1?
* `pdf_gate_reason: 'attached' | 'legacy_session_no_pdf' | 'channel_not_imessage' | 'zdr_not_enabled' | 'signing_failed' | 'feature_disabled' | 'no_line_binding'` — for auditing the gate decision.
* `pdf_zdr_observation` and `pdf_capability_check` — sub-observations backing the gate decision; see the field docstrings on `DrPdfAttachmentContext` in `lib/messaging/dr-pdf-attachment.ts` for value semantics.

Slack alert if `pdf_gate_reason === 'signing_failed'` starts trending (indicates AWS creds / bucket policy drift). A rising `no_line_binding` share indicates users are hitting the Twilio→Linq mid-run migration path more often than expected and warrants revisiting the DR-notification seeding to backfill Linq threading anchors.

## Rollout plan

1. **Phase 1** — shipped in [PR #3923](https://github.com/CFO-Silvia/Silvia-Web/pull/3923) (staging) and [PR #3924](https://github.com/CFO-Silvia/Silvia-Web/pull/3924) (main cherry-pick):
   * `lib/messaging/dr-pdf-attachment.ts` exports `presignDrReportPdf` (deterministic 24 h pre-signed URL via caller-supplied `signingDate`, closes §1a A1 + §5), `isDrPdfAttachEnabled` (comma-separated Clerk-ID allow-list on `LINQ_DR_PDF_ATTACH_CLERK_IDS`, default OFF), and the shared `DrPdfGateReason` / `DrPdfAttachmentContext` types for the §8 observability fields.
   * Unit tests in `lib/messaging/__tests__/dr-pdf-attachment.test.ts` cover the determinism-across-retries invariant, the legacy-session fallback, the ownership + anchor-validity guards (future-date rejection, TTL boundary as `>=`), the flag parsing rules, and the redaction posture on signing-failure logs.
   * Nothing is imported by a send-site yet — this ships as scaffolding so Phase 2 lands as a focused wiring diff.
2. **Phase 2** — shipping in this branch (`feat/dr-pdf-linq-phase2-send-wiring`):
   * `lib/linq/api-client.ts`: `checkImessageCapability` wrapper around `POST /v3/capability/check_imessage` (§4.2 Option C). Never throws — returns `{ eligible, error? }`. Lives on the shared Linq API client alongside the other `/v3/*` wrappers rather than a standalone provider file; an earlier revision of this doc referenced `lib/messaging/providers/linq-capability.ts`, which was never created (CodeRabbit PR #3925 flagged the drift).
   * `lib/messaging/dr-pdf-gate.ts`: end-to-end gate orchestrator. Runs the feature-flag check → legacy-session check → recent-inbound ZDR observation (from `silvia_sms_logs.user_context.zero_retention` per §6a) → live iMessage capability check (via `checkImessageCapability` above, with a 5-minute in-memory cache) → S3 pre-sign, and returns one of the §8 `DrPdfGateReason` codes with the pre-signed URL when `reason === 'attached'`. Does NOT implement §4.2 B's historical-channel pre-filter — an earlier revision of this doc claimed it did; Phase 2 skips it as an unnecessary optimization at pilot scale and defers it to a future phase if rate-limit signals warrant.
   * `lib/messaging/outbound.ts`: new `SendSilviaReplyInput` fields — `firstChunkMediaUrls?: string[]` (attached only to the first chunk's provider `sendMessage` call), `firstChunkUserContextExtra?: Record<string, unknown>` (merged into the first chunk's `silvia_sms_logs.user_context` row, on both success and failure sends so the audit trail survives a bad provider response), and `preferredService?: 'imessage' | 'rcs' | 'sms'` (forwarded on EVERY chunk — the wire pin must apply to the whole reply so a multi-chunk send can't have chunk 1 pinned to iMessage while chunk 2 open-cascades to SMS). Contradictory combinations (`channel: 'imessage'` + `preferredService: 'sms' | 'rcs'`) throw at input validation before any content decoration — flagged on CodeRabbit PR #3925 as a compliance risk (`channel: 'imessage'` strips the TCPA footer, but the explicit protocol pin would then deliver over carrier SMS without it).
   * `app/api/sms/research-complete/route.ts`: bubble 1 now evaluates the gate, attaches `firstChunkMediaUrls: [pdfUrl]` when `attached`, and stamps every §8 observability field on `silvia_sms_logs.user_context`. Bubble 2 (URL) and the URL-inline fallback are unchanged. Feature is default-OFF via `isDrPdfAttachEnabled(clerkId)`. Route ALSO short-circuits the gate call to `pdf_gate_reason: 'no_line_binding'` (URL-only) when `notification.source_provider_message_id` / `parent_provider_message_id` are missing — without a thread anchor, Linq's managed line selection can pick a different physical line than the one our capability + ZDR observations were made against (per Linq docs on `from_selection.reason = 'reused_active_chat'`; CodeRabbit PR #3925 flagged this).
   * New integration test coverage for each `DrPdfGateReason` code + the happy path (PDF attached with `preferredService: 'imessage'`).
3. **Phase 3**: Roll the allow-list flag to a wider pilot cohort. Monitor `pdf_gate_reason` distribution + Linq delivery-rate telemetry for 1 week before removing the flag entirely.
4. **Phase 4** (deferred, may be unnecessary): Legacy-session server-side PDF renderer. Revisit if the `legacy_session_no_pdf` count in `user_context` stays high after Phase 3 is stable.

## Explicit non-goals

* **Twilio users**: they stay on the current inline-URL body. Attaching PDFs via MMS is a delivery-quality liability.
* **Radar SMS**: does not get PDF attachment (radars are short signals, not reports). URL-only is correct.
* **Any inbound PDF handling**: covered by the sibling PR that persists inbound Linq images to the vault. PDF inbound handling is a separate follow-up.

## References

* [Linq — Sending Messages](https://docs.linqapp.com/guides/messaging/sending-messages/)
* [Linq — Zero-Day Retention](https://docs.linqapp.com/channel/imessage/guides/platform/zero-day-retention/)
* Codebase: `lib/messaging/providers/linq.ts` (media parts implementation)
* Codebase: `lib/research/markdown-to-pdf.ts` (client-side PDF generator, legacy sessions)
* Codebase: `app/api/report/download/route.ts` (existing PDF resolver, Clerk-gated)
* Codebase: `app/api/sms/research-complete/route.ts` (target route for PDF attachment)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.