​

SMS Implementation Summary

​

Overview

​

Architecture

​

Routes

​

1. /api/sms/incoming (POST)

​

2. /api/sms/delivery-status (POST)

​

3. /api/sms/opt-in (GET/POST)

​

4. Advanced Opt-Out (Built into /api/sms/incoming)

​

How It Works

User sends SMS/MMS to Silvia's number (+1 831 215 4002)
         ↓
Twilio receives message and triggers webhook
         ↓
POST /api/sms/incoming
         ↓
1. Parse Twilio form data (From, Body, MessageSid, NumMedia, MediaUrl0, etc.)
2. Check for images (NumMedia > 0)
3. Normalize phone number (e.g., +16082159565 → 16082159565)
4. Look up user in all_users table by phone_number
         ↓
If user not found:
  - Log attempt in silvia_sms_logs
  - Send: "Hi! I'm Silvia. I don't recognize this number. Sign up at cfosilvia.com"
  - Return
         ↓
If user found:
  - Check sms_opt_in status:
    * If false: Send "To chat with Silvia via SMS, enable SMS at cfosilvia.com/settings"
    * Return
  - Log incoming message in silvia_sms_logs (with media URLs if present)
  - Download images from Twilio (if any):
    * Authenticate with Twilio credentials
    * Download image to buffer
    * Convert to base64
  - Retrieve last 20 SMS messages for conversation context
  - Build conversation history (oldest to newest)
  - Generate AI response using generateText (same as email):
    * Import tools via createChatTools()
    * Use SMS-specific system prompt (concise, no markdown, image analysis)
    * Build multimodal content (text + images if present)
    * Call generateText with stopWhen: stepCountIs(10)
    * Claude tries first, falls back to GPT-5 if error
    * Extract text from result.response.messages
         ↓
AI processes request:
  - Analyzes images (if present) using Claude's vision
  - Uses ALL the same tools as webapp (getFinancialMetrics, etc.)
  - Executes tool calls (e.g., fetch net worth data)
  - Generates concise, SMS-friendly response
  - Returns final text
         ↓
Clean up response for SMS:
  - Remove markdown formatting (**, *, #)
  - Remove links, code blocks
  - Trim whitespace
         ↓
Send response via Twilio:
  - Split long messages (>1600 chars) into multiple SMS
  - Send each segment via Twilio API
  - Log outbound message in silvia_sms_logs
         ↓
Return success to Twilio

​

Key Technical Details

​

AI SDK Version

• Required: AI SDK v5.0.60+ (updated from 5.0.15)
• Why: Fixed critical bug with Claude + stopWhen + tool calls
• Before update: Claude would fail with “messages.1.content.0.tool_use.input: Input should be a valid dictionary”
• After update: Claude works perfectly with multi-step tool execution

​

Implementation Pattern

1. Use generateText (not streamText)
2. Use stopWhen: stepCountIs(10) for multi-step execution
3. Extract response from result.response.messages[last]
4. Fallback to GPT-5 if Claude fails

​

Why Not Use /api/chat?

​

MMS (Image) Support

• 📄 Receipts - “What’s in this receipt?” → Claude extracts merchant, amount, items
• 📊 Charts/Graphs - “Analyze this chart” → Claude describes trends and insights
• 📋 Statements - “What’s my balance?” → Claude reads account info
• 📈 Stock Charts - “Is this a good buy?” → Claude analyzes price action
• 💳 Credit Card Bills - “How much do I owe?” → Claude extracts balance

1. Twilio receives MMS with image(s)
2. Twilio sends NumMedia, MediaUrl0, MediaContentType0 in webhook
3. Our webhook downloads image from Twilio (requires auth)
4. Convert image to base64
5. Pass to Claude as multimodal content (text + image)
6. Claude analyzes using vision capabilities
7. Return concise SMS response

• Only processes images (skips videos/other media)
• Supports multiple images per message
• Uses Claude Sonnet 4.5’s vision capabilities
• Falls back to GPT-5 if Claude fails (GPT-5 also has vision)
• Image data stored in user_context JSONB column in logs

​

Files Created

​

1. lib/twilio.ts

• sendSMS(to, body) - Send a single SMS via Twilio API
• sendLongSMS(to, body) - Split and send long messages (>1600 chars)
• normalizePhoneNumber(phone) - Convert phone to database format (digits only, with country code)
  • Example: +1 (608) 215-9565 → 16082159565
  • Adds 1 prefix for 10-digit US numbers
• splitMessage(text, maxLength) - Split long text into SMS-sized chunks
• verifyTwilioSignature(signature, url, params) - Verify webhook authenticity (optional)

​

2. app/api/sms/incoming/route.ts

• Receives POST requests from Twilio with form data
• Parses: From, Body, MessageSid, To, NumMedia, MediaUrl0, etc.
• Detects and downloads images from Twilio (MMS support)
• Normalizes phone number using same logic as voice webhook
• Looks up user in all_users table by phone_number
• Handles unknown users with friendly signup message
• Uses generateText directly with multimodal content (text + images)
• Includes SMS-specific system prompt with image analysis instructions
• Collects response and cleans markdown
• Sends response via Twilio (splits if needed)
• Logs all interactions in silvia_sms_logs (with media URLs)
• Error handling with user-friendly fallback messages

• maxDuration = 60 seconds (Vercel timeout)
• Uses Supabase service role key (bypasses RLS)
• Stores fromNumber for error handling
• Cleans response: removes **, *, #, links, code blocks

​

3. app/api/sms/delivery-status/route.ts

• Receives POST requests from Twilio with delivery updates
• Parses: MessageSid, MessageStatus, ErrorCode, ErrorMessage
• Updates silvia_sms_logs table with delivery status
• Tracks: queued, sending, sent, delivered, undelivered, failed
• Logs delivery errors for debugging

​

4. sql-queries/create-sms-logs-table.sql

CREATE TABLE silvia_sms_logs (
  id UUID PRIMARY KEY,
  clerk_id TEXT,  -- Can be null for unknown numbers
  phone_number TEXT NOT NULL,
  direction TEXT NOT NULL,  -- 'inbound' or 'outbound'
  message_content TEXT NOT NULL,
  twilio_message_sid TEXT,
  related_message_sid TEXT,  -- Links responses to original messages
  response_sent BOOLEAN DEFAULT false,
  error_message TEXT,
  delivery_status TEXT,  -- Twilio delivery status
  delivery_updated_at TIMESTAMP,
  delivery_error_code TEXT,
  delivery_error_message TEXT,
  user_context JSONB,  -- Snapshot of user data
  created_at TIMESTAMP DEFAULT NOW()
)

• clerk_id (for user queries)
• phone_number (for lookup)
• created_at (for time-based queries)
• twilio_message_sid (for delivery status updates)

​

5. sql-queries/add-delivery-status-columns.sql

​

Files Modified

​

app/api/chat/route.ts

1. Accept clerkId from request body (line 71)
   const { messages, conversationId, sessionId, responseFormat, clerkId: clerkIdFromBody } = await req.json()
   
2. Support internal authentication (lines 124-130)
   // Support both Clerk auth and internal clerkId (for SMS/voice webhooks)
   const { userId: clerkIdFromAuth } = await auth()
   const clerkId = clerkIdFromBody || clerkIdFromAuth
   
   if (!clerkId) {
     return Response.json({ error: 'Unauthorized' }, { status: 401 })
   }
   
3. Add SMS-specific system prompt (lines 245-265)
   if (isSMS) {
     systemPromptWithMemory = `${SYSTEM_PROMPT}
   
   ## SMS RESPONSE MODE
   
   You are currently responding via SMS text message. Follow these critical guidelines:
   
   1. **BE CONCISE**: Keep responses under 300 characters when possible.
   2. **NO MARKDOWN**: Don't use markdown formatting.
   3. **NO EMOJIS**: Avoid emojis.
   4. **PRIORITIZE KEY INFO**: Lead with the most important information.
   5. **SUGGEST APP FOR DETAILS**: For complex queries, suggest checking the app.
   6. **USE TOOLS SPARINGLY**: Prefer quick, direct responses.
   7. **NO CHARTS/TABLES**: Don't mention visual elements.
   
   Keep it short, clear, and actionable.`
   }
   
4. Conditional message processing (lines 337-351)
   • For SMS: Uses manual message normalization (simple structure)
   • For webapp: Uses AI SDK’s convertToModelMessages (original behavior, unchanged)
   • This ensures SMS works without affecting webapp functionality at all

​

Environment Variables Required

# Twilio (already in .env)
TWILIO_ACCOUNT_SID=AC...
TWILIO_AUTH_TOKEN=...
TWILIO_PHONE_NUMBER=+18312154002

# App URL (for internal API calls)
NEXT_PUBLIC_APP_URL=https://cfosilvia.com  # Production
# or
NEXT_PUBLIC_APP_URL=https://xxx.ngrok-free.app  # Local testing with ngrok

# Supabase (already configured)
NEXT_PUBLIC_SUPABASE_URL=...
SUPABASE_SERVICE_ROLE_KEY=...

​

Opt-In/Opt-Out System (Two-Layer Compliance)

​

Why Two Layers?

​

Layer 1: Twilio’s Automatic Opt-Out

• Twilio automatically handles keywords: STOP, START, HELP, UNSTOP, QUIT, CANCEL, etc.
• When user texts STOP:
  1. Twilio adds their number to a carrier-level blocklist
  2. Twilio sends automatic confirmation: “You have successfully been unsubscribed…”
  3. All future messages to that number fail with Error 21610
  4. Our webhook never sees this message
• When user texts START or UNSTOP:
  1. Twilio removes them from blocklist
  2. Twilio sends automatic confirmation: “You have successfully been re-subscribed…”
  3. Messages can be sent again

• ❌ Can’t customize STOP/START behavior for toll-free numbers
• ❌ Can’t prevent Twilio from blocking messages after STOP
• ❌ Our webhook doesn’t receive STOP/START messages (unless we enable Advanced Opt-Out)

​

Layer 2: Our Application’s Opt-In System

ALTER TABLE all_users ADD COLUMN sms_opt_in BOOLEAN DEFAULT false;

1. User signs up → sms_opt_in = false (default)
2. User goes to Settings → SMS Notifications → Checks box
3. /api/sms/opt-in called → Updates sms_opt_in = true
4. Welcome SMS sent: “Hi! SMS is now enabled. Text me anytime for instant insights…”
5. User can now text Silvia → /api/sms/incoming checks sms_opt_in = true

// In /api/sms/incoming/route.ts
if (!userData.sms_opt_in) {
  await sendLongSMS(
    from,
    "To chat with Silvia via SMS, please enable SMS notifications at cfosilvia.com/settings"
  )
  return
}

​

Syncing Both Layers (Advanced Opt-Out)

1. Enable Advanced Opt-Out in Twilio Console → Messaging Services
2. ✅ No separate webhook needed! Twilio sends OptOutType to your existing incoming webhook
3. When user texts STOP:
   • Twilio handles blocking + sends confirmation
   • Twilio calls /api/sms/incoming with extra parameter: OptOutType=STOP
   • We detect it and update: sms_opt_in = false
4. When user texts START:
   • Twilio unblocks + sends confirmation
   • Twilio calls /api/sms/incoming with OptOutType=START
   • We detect it and update: sms_opt_in = true

​

Opt-In/Opt-Out Flow Diagram

┌─────────────────────────────────────────────────────────────────┐
│                    USER WANTS TO ENABLE SMS                     │
└─────────────────────────────────────────────────────────────────┘
                              ↓
                    Settings → Enable SMS ☑
                              ↓
               POST /api/sms/opt-in {optIn: true}
                              ↓
                   Database: sms_opt_in = true
                              ↓
                 Welcome SMS sent to user's phone
                              ↓
                    ✅ User can now text Silvia


┌─────────────────────────────────────────────────────────────────┐
│               USER WANTS TO DISABLE SMS (Method 1)              │
└─────────────────────────────────────────────────────────────────┘
                              ↓
                    Settings → Disable SMS ☐
                              ↓
              POST /api/sms/opt-in {optIn: false}
                              ↓
                   Database: sms_opt_in = false
                              ↓
                ❌ User can't text Silvia anymore


┌─────────────────────────────────────────────────────────────────┐
│               USER WANTS TO DISABLE SMS (Method 2)              │
└─────────────────────────────────────────────────────────────────┘
                              ↓
                       User texts: "STOP"
                              ↓
              Twilio intercepts + blocks at carrier level
                              ↓
          Twilio sends confirmation: "You have been unsubscribed..."
                              ↓
    Twilio calls: POST /api/sms/incoming {OptOutType: STOP}
                              ↓
         We detect OptOutType and update database
                              ↓
                   Database: sms_opt_in = false
                              ↓
            ✅ Both layers synced, user fully opted out


┌─────────────────────────────────────────────────────────────────┐
│                   USER WANTS TO RE-ENABLE SMS                   │
└─────────────────────────────────────────────────────────────────┘
                              ↓
                     User texts: "START"
                              ↓
               Twilio removes carrier-level block
                              ↓
         Twilio sends confirmation: "You have been re-subscribed..."
                              ↓
    Twilio calls: POST /api/sms/incoming {OptOutType: START}
                              ↓
                   Database: sms_opt_in = true
                              ↓
            ✅ Both layers synced, user can text again

​

Legal Compliance (TCPA)

1. ✅ Express written consent - Checkbox in settings with clear language
2. ✅ Clear disclosure - “By checking this box, you agree to receive recurring messages…”
3. ✅ Opt-out mechanism - Reply STOP + settings toggle
4. ✅ Terms link - /sms-terms page with full T&Cs
5. ✅ Rate disclosure - “Msg&Data rates may apply”
6. ✅ Frequency disclosure - “Message frequency varies”
7. ✅ Help keyword - Reply HELP (handled by Twilio)

By checking this box, you agree to receive recurring messages from Silvia.
Message & data rates may apply. Reply STOP to cancel, HELP for help.
View our Privacy Policy and SMS Terms.

​

Twilio Configuration

​

Phone Number Setup

1. Go to Twilio Console → Phone Numbers → Manage → Active Numbers
2. Click on your number: +1 (831) 215-4002
3. Under “Messaging Configuration”:
   • A message comes in: Webhook
   • URL: https://www.cfosilvia.com/api/sms/incoming
   • HTTP Method: POST
4. Under “Messaging Service”: Select “Silvia” (your messaging service)
5. Under “Status Callback URL” (optional):
   • URL: https://www.cfosilvia.com/api/sms/delivery-status
   • HTTP Method: POST

​

Advanced Opt-Out Setup

1. Go to Twilio Console → Messaging → Services → Silvia
2. Navigate to Advanced Opt-Out Management
3. Enable Advanced Opt-Out ✅
4. ✅ No webhook configuration needed!
   • Twilio automatically sends OptOutType parameter to your existing webhook
   • The parameter is added to requests sent to: https://www.cfosilvia.com/api/sms/incoming
5. When users text STOP/START/HELP, your incoming webhook will receive the OptOutType parameter

​

A2P 10DLC Registration (Required for Production)

• Status: Completed ✅
• Required for: Sending SMS to US numbers from long codes
• Timeline: 1-3 business days for approval
• Approval: Campaign approved and active

​

Testing

​

Local Testing with ngrok

1. Start dev server: npm run dev
2. Start ngrok: ngrok http 3000
3. Update .env: NEXT_PUBLIC_APP_URL=https://xxx.ngrok-free.app
4. Restart dev server
5. Update Twilio webhook to ngrok URL
6. Send SMS to test

​

Production Testing

1. Deploy to Vercel
2. Add environment variables in Vercel dashboard
3. Update Twilio webhook to production URL
4. Complete A2P 10DLC registration (or use toll-free number)
5. Send SMS to test

​

Test Commands

# Test webhook directly (requires ngrok)
curl -X POST https://xxx.ngrok-free.app/api/sms/incoming \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -H "ngrok-skip-browser-warning: true" \
  -d "From=+16082159565" \
  -d "Body=What is my net worth?" \
  -d "MessageSid=SM_test_123" \
  -d "To=+18312154002"

​

Key Features

​

User Lookup Logic

1. Extract phone from Twilio’s From field: +1 (608) 215-9565
2. Normalize: Remove all non-digits → 16082159565
3. Add country code if missing: 6082159565 → 16082159565
4. Query database: SELECT * FROM all_users WHERE phone_number = '16082159565'
5. If found: Process SMS with user context
6. If not found: Send signup message

​

Example Conversations

​

Cost Estimates

​

Per Message

• Twilio SMS: $0.0079 per segment (US)
• Claude API: ~$0.001 per response (varies by length)
• Total: ~$0.009 per message, ~$0.018 per conversation

​

Monthly Estimates

• 100 conversations/day: ~$54/month
• 500 conversations/day: ~$270/month
• 1000 conversations/day: ~$540/month

​

Additional Costs

• Phone number: $1.15/month (long code) or$2/month (toll-free)
• A2P 10DLC registration: One-time $4-15 fee

​

Database Queries

​

View recent SMS conversations

SELECT 
  clerk_id,
  phone_number,
  direction,
  message_content,
  delivery_status,
  created_at
FROM silvia_sms_logs
ORDER BY created_at DESC
LIMIT 50;

​

Check delivery failures

SELECT 
  phone_number,
  message_content,
  delivery_status,
  delivery_error_code,
  delivery_error_message,
  created_at
FROM silvia_sms_logs
WHERE delivery_status IN ('failed', 'undelivered')
ORDER BY created_at DESC;

​

User SMS activity

SELECT 
  clerk_id,
  COUNT(*) as total_messages,
  COUNT(CASE WHEN direction = 'inbound' THEN 1 END) as inbound,
  COUNT(CASE WHEN direction = 'outbound' THEN 1 END) as outbound,
  MAX(created_at) as last_message
FROM silvia_sms_logs
WHERE clerk_id IS NOT NULL
GROUP BY clerk_id
ORDER BY total_messages DESC;

​

Monitoring & Debugging

​

Check Twilio logs

1. Go to Twilio Console → Monitor → Logs → Messaging
2. Filter by your phone number
3. Check for delivery failures or errors

​

Check Supabase logs

SELECT * FROM silvia_sms_logs 
WHERE error_message IS NOT NULL 
ORDER BY created_at DESC;

​

Check Vercel logs

1. Go to Vercel Dashboard → Your project → Logs
2. Filter by /api/sms/incoming
3. Look for errors or slow requests

​

Known Limitations

1. A2P 10DLC required for production - Messages won’t deliver until registration is complete
2. 160 character SMS limit - Long messages are split (costs more)
3. No rich media in basic SMS - Would need MMS for images/charts
4. Rate limits - Twilio has rate limits per number
5. Delivery delays - SMS can take 1-10 seconds to deliver

​

Next Steps

​

Immediate

• Complete SMS implementation
• Test locally with ngrok
• Complete A2P 10DLC registration in Twilio
• Deploy to production
• Test with real phone

​

Future Enhancements

• Add rate limiting (max 10 SMS/hour per user)
• Add proactive SMS notifications (e.g., “Your portfolio is down 5% today”)
• Add SMS analytics dashboard
• Add sentiment analysis for user messages

​

Architecture Benefits

​

Why This Approach Works

1. Code Reuse: SMS uses the same /api/chat endpoint as the web app
   • No duplicate AI logic
   • Same tools and capabilities
   • Consistent personality and responses
2. Minimal Changes: Only added responseFormat flag and clerkId parameter
   • Existing chat system unchanged
   • Easy to maintain
   • Simple to extend
3. Consistent User Experience: Same Silvia across all channels
   • Web chat
   • Voice calls (ElevenLabs)
   • SMS messages
   • Future: Email, WhatsApp, etc.
4. Scalable: Built on serverless architecture
   • Auto-scales with demand
   • No infrastructure management
   • Pay only for what you use

​

Troubleshooting

​

SMS not received

• Check A2P 10DLC registration status
• Verify Twilio webhook is configured correctly
• Check Twilio logs for delivery failures
• Ensure phone number is in E.164 format in database

​

Webhook errors

• Check Vercel logs for errors
• Verify environment variables are set
• Test webhook URL is accessible
• Check ngrok is running (for local testing)

​

User not found

• Verify phone number format in database matches normalized format
• Check all_users table has phone_number column

​

Current Status

​

✅ Completed

• SMS webhook endpoint (/api/sms/incoming)
• Delivery status webhook (/api/sms/delivery-status)
• User lookup by phone number
• AI response generation with full tool access
• Multi-step tool execution (same as email)
• SMS-specific system prompt (concise, no markdown)
• Message splitting for long responses
• Comprehensive logging (silvia_sms_logs table)
• AI SDK updated to v5.0.60+ (fixed Claude bug)
• Claude + GPT-5 fallback working perfectly

​

⏳ Pending

• A2P 10DLC Registration - Required for SMS delivery in the US
  • Brand registration
  • Campaign registration
  • Estimated time: 1-5 business days for approval
  • Once approved, SMS will be delivered to users

​

🧪 Testing

• Local testing with ngrok: ✅ Working
• Tool execution: ✅ Working (getFinancialMetrics tested)
• Response generation: ✅ Working (concise, SMS-friendly)
• Message sending: ✅ Working (queued in Twilio)
• Message delivery: ⏳ Blocked until A2P registration complete

​

Next Steps

1. Complete A2P 10DLC Registration in Twilio
   • Fill out brand information
   • Create campaign
   • Wait for approval (1-5 business days)
2. Test with Real Users
   • Once A2P approved, test with multiple users
   • Verify message delivery
   • Test different queries (net worth, spending, holdings, etc.)
3. Monitor and Optimize
   • Monitor SMS logs in Supabase
   • Track delivery status
   • Optimize response conciseness if needed

• Ensure user has phone number set in their profile

​

Long response times

• Check Claude API latency
• Verify tool calls aren’t timing out
• Consider reducing system prompt length for SMS
• Check database query performance

​

Success Metrics

• ✅ Webhook returns {"success":true}
• ✅ User receives SMS response
• ✅ Messages logged in silvia_sms_logs table
• ✅ Response is concise and SMS-friendly
• ✅ Unknown users get signup message
• ✅ Delivery status updates received (if configured)