diff --git a/.Agent Context/COMMS_INTEGRATION_HANDOFF.md b/.Agent Context/COMMS_INTEGRATION_HANDOFF.md new file mode 100644 index 00000000..f8234f39 --- /dev/null +++ b/.Agent Context/COMMS_INTEGRATION_HANDOFF.md @@ -0,0 +1,265 @@ +# Velocity Comms Integration — Handoff Document + +## 1. Architecture Recommendation + +### Goal +Add a native **Conversations** module to Project Velocity so brokers/agents can manage WhatsApp (and future SMS/call) threads without leaving the WebOS. + +### Design Philosophy +- **Native Velocity UI**: Dark glass panels, compact density, blue accent, no iframe embeds. +- **Provider-agnostic backend**: Abstract `CommsProvider` class with adapter pattern. +- **CRM-first**: Every thread attempts to link to `crm_people` by `primary_phone`. Unresolved numbers are surfaced for manual linking. +- **Mock-first development**: The module renders fully without real credentials. + +### Provider Comparison + +| Provider | Best For | Velocity Fit | 72-Hour Viability | +|----------|----------|--------------|-------------------| +| **Chatwoot** | Full support suite (email, SMS, WA) | Too heavy to embed; good UX reference | Low — would require stripping UI | +| **WAHA** | Lightweight WhatsApp Web gateway | Good adapter candidate | High — simple REST, easy webhooks | +| **Evolution API** | Modern WA gateway with groups, status, typing | Best adapter candidate | **High** — active community, clean webhooks | +| **Meta Cloud API** | Official WABA; template-based outbound | Required for production scale at large builders | Medium — needs Meta Business verification | + +**Recommended 72-hour route:** +1. **Day 1**: Merge schema + backend routes + mock provider. Frontend compiles with mock data. +2. **Day 2**: Connect Evolution API or WAHA in a staging environment. Test inbound webhook → thread creation. +3. **Day 3**: CRM linking, settings UI, call-log upload placeholder, and smoke tests. + +For production, plan a **dual-provider** setup: +- **Evolution/WAHA** for quick conversational messaging (no Meta approval needed). +- **Meta Cloud API** for template-based broadcast/re-engagement once Business Manager is verified. + +--- + +## 2. Exact Files Created + +``` +app/src/types/commsTypes.ts +app/src/lib/commsApi.ts +app/src/components/modules/Comms.tsx +backend/db/schema_comms.sql +backend/services/comms_provider.py +backend/services/comms_waha_provider.py +backend/services/comms_evolution_provider.py +backend/services/comms_ingest.py +backend/api/routes_comms.py +COMMS_INTEGRATION_HANDOFF.md +``` + +--- + +## 3. Patch Instructions for Existing Files + +### A. `app/src/types/index.ts` +Add `'comms'` to the `ModuleId` union: +```typescript +export type ModuleId = 'dashboard' | 'oracle' | 'sentinel' | 'inventory' | 'settings' | 'catalyst' | 'admin' | 'crm' | 'comms'; +``` + +### B. `app/src/App.tsx` +1. Import the new component: +```typescript +import { Comms } from '@/components/modules/Comms'; +``` +2. Insert the route into `MODULE_ROUTES` **just before** `settings`: +```typescript +{ id: 'comms', path: '/comms', title: 'Conversations', component: Comms }, +``` + +### C. `app/src/components/layout/Sidebar.tsx` +1. Import a new icon: +```typescript +import { MessageCircle } from 'lucide-react'; +``` +2. Add to `NAV_ICONS`: +```typescript +const NAV_ICONS: Record = { + '/dashboard': LayoutGrid, + '/oracle': MessageSquarePlus, + '/sentinel': ScanFace, + '/inventory': Building2, + '/catalyst': Megaphone, + '/comms': MessageCircle, // ← NEW + '/settings': Sliders, + '/admin': Shield, + '/crm': Users, +}; +``` + +### D. `backend/main.py` +1. Import the router near the other imports: +```python +from backend.api.routes_comms import router as comms_router +``` +2. Include it after the other routers: +```python +app.include_router(comms_router, prefix="/api/comms", tags=["Comms"]) +``` + +--- + +## 4. Environment Variables + +Add these to your `.env` or systemd environment: + +```bash +# Provider selection: mock | waha | evolution | meta_cloud +COMMS_PROVIDER=mock + +# Provider connectivity +COMMS_PROVIDER_BASE_URL= +COMMS_PROVIDER_API_KEY= +COMMS_INSTANCE_ID=default + +# Webhook security +COMMS_WEBHOOK_SECRET= + +# Phone normalization +COMMS_DEFAULT_COUNTRY_CODE=91 + +# Media storage +COMMS_MEDIA_STORAGE_DIR=/opt/dlami/nvme/assets/comms + +# Transcription (none | openai | local) +COMMS_TRANSCRIPTION_PROVIDER=none +``` + +**No secrets are hardcoded in source.** + +--- + +## 5. Database Migration + +Run the SQL file against your Postgres database: + +```bash +psql $DATABASE_URL -f backend/db/schema_comms.sql +``` + +Tables created: +- `comms_threads` — conversation headers with CRM link +- `comms_messages` — individual messages (inbound/outbound/system) +- `comms_call_logs` — call records with optional transcript +- `comms_settings` — key-value config store + +--- + +## 6. API Routes + +| Method | Path | Purpose | +|--------|------|---------| +| GET | `/api/comms/threads` | List threads (search, status, pagination) | +| GET | `/api/comms/threads/{id}` | Get single thread with CRM enrichment | +| GET | `/api/comms/threads/{id}/messages` | Chronological messages | +| POST | `/api/comms/threads/{id}/messages` | Send outbound message via provider | +| POST | `/api/comms/threads/{id}/link-person` | Link thread to `crm_people.id` | +| POST | `/api/comms/threads/{id}/notes` | Add system note | +| POST | `/api/comms/threads/{id}/tasks` | Add system task | +| POST | `/api/comms/webhooks/{provider}` | Public webhook endpoint | +| GET | `/api/comms/settings` | Get comms configuration | +| PATCH | `/api/comms/settings` | Update configuration | +| POST | `/api/comms/provider/test` | Test provider connectivity | +| POST | `/api/comms/recordings/transcribe` | Queue transcription job | + +--- + +## 7. Frontend Route Changes + +- New sidebar item: **Conversations** (icon: `MessageCircle`) +- Position: directly above **Settings** +- Route: `/comms` +- Component: `Comms.tsx` with three-pane layout (Inbox | Chat | CRM Rail) + +--- + +## 8. Settings Changes + +A new **Communications** subsection should be added inside your existing Settings module (or as a standalone card). Fields: + +| Field | Type | Description | +|-------|------|-------------| +| Provider | select | mock / waha / evolution / meta_cloud | +| Provider Base URL | text | e.g. `http://localhost:3000` | +| API Key | password | masked after save | +| Instance ID | text | WA/Evolution session name | +| Phone Number ID | text | Meta Cloud API only | +| Webhook Callback URL | text | Auto-populated or custom | +| Webhook Secret | password | Sets `webhook_secret_set` flag | +| Default Assignment User | select | User dropdown from `/api/auth/users` | +| Auto-link by Phone | toggle | Match `crm_people.primary_phone` automatically | +| Create CRM Interaction on Inbound | toggle | Write to `intel_interactions` if table exists | +| Default Country Code | text | e.g. `91` for India | +| Transcription Provider | select | none / openai / local | +| Connection Test | button | Calls `POST /api/comms/provider/test` | + +--- + +## 9. Smoke Test Steps + +1. **DB**: Run `schema_comms.sql`. Verify tables exist. +2. **Backend**: Start FastAPI. Confirm `/health` returns `db_pool: connected`. +3. **Backend**: `curl -X POST http://localhost:8000/api/comms/provider/test` → should return mock success. +4. **Frontend**: Load Velocity. Sidebar should show **Conversations**. +5. **Frontend**: Click Conversations. Mock mode should render 3 threads and messages. +6. **Frontend**: Send a message in mock thread. Optimistic update → mock delivery checkmark. +7. **Backend**: Post sample webhook: + ```bash + curl -X POST http://localhost:8000/api/comms/webhooks/evolution -H "Content-Type: application/json" -d '{"event":"messages.upsert","instance":"default","data":{"key":{"remoteJid":"919876543210@s.whatsapp.net","fromMe":false,"id":"test-1"},"message":{"conversation":"Hello from webhook"},"messageTimestamp":1710000000}}' + ``` +8. **Backend**: Verify thread + message inserted. Check `comms_threads` for new row. +9. **Frontend**: Refresh inbox. New thread should appear. +10. **CRM Link**: Click "Link to Contact" (or call `POST /api/comms/threads/{id}/link-person`) and verify `person_id` is set. + +--- + +## 10. Known Limitations + +- **Call recording via WhatsApp API**: Neither WAHA nor Evolution supports native WhatsApp call recording. Call logs are designed for **external telephony intake** (manual upload or webhook from a PBX/VoIP system). Recording file + transcript workflow is scaffolded but needs a real transcription provider (OpenAI Whisper, AWS Transcribe, or faster-whisper) wired in. +- **Media downloads**: `get_media()` is stubbed for WAHA/Evolution. Production needs signed URL handling or local file download. +- **Meta Cloud API adapter**: Not yet implemented. Add `comms_meta_provider.py` when Meta Business verification is complete. +- **Template messages**: Only placeholder methods exist. Template approval flow (Meta) or local template storage must be built for outbound campaigns. +- **Webhook auth**: Currently accepts any payload. Add HMAC/signature verification per provider before production. +- **Rate limiting**: Not implemented. Add FastAPI rate-limit middleware on `/api/comms/webhooks/{provider}`. +- **phonenumbers library**: `comms_ingest.py` gracefully degrades to regex if `phonenumbers` is not installed. Install it for robust E.164 normalization: + ```bash + pip install phonenumbers + ``` + +--- + +## 11. What Still Needs Real Credentials + +| Item | What You Need | +|------|---------------| +| **Evolution API** | A running Evolution instance (Docker), API key, and a paired WhatsApp number. | +| **WAHA** | A running WAHA container, session QR-scan, and API key. | +| **Meta Cloud API** | Meta Business Manager, verified business, WhatsApp Business Account, permanent access token, phone number ID. | +| **Transcription** | OpenAI API key (for Whisper) or local faster-whisper model path. | +| **CRM enrichment** | Ensure `crm_people` table exists with `primary_phone` indexed. | + +--- + +## 12. What to Verify Before Production + +- [ ] Webhook endpoint is exposed via HTTPS (ngrok/cloudflare tunnel for local dev). +- [ ] `COMMS_WEBHOOK_SECRET` is set and signature verification is enabled in `routes_comms.py`. +- [ ] Database has indexes on `comms_threads(phone_e164)` and `comms_messages(thread_id, created_at)`. +- [ ] `crm_people.primary_phone` is normalized to E.164 before comms matching. +- [ ] Media storage directory exists and is writable (`COMMS_MEDIA_STORAGE_DIR`). +- [ ] Outbound message queue / retry logic is added (currently synchronous). +- [ ] GDPR/opt-out handling is implemented if targeting EU markets. +- [ ] Backup strategy for `comms_messages` (contains legal conversation records). + +--- + +## 13. Next Iteration Ideas + +- **Bulk broadcast**: Template-based outbound to filtered CRM segments. +- **AI reply suggestions**: Integrate Oracle / local LLM to draft replies based on CRM context. +- **Voice notes**: Upload `.ogg` audio, transcribe, store transcript as message. +- **Read receipts**: Poll provider for delivery/read status and update `comms_messages`. +- **Assignment rules**: Round-robin or load-based auto-assignment to agents. + +--- + +*Document generated for Project Velocity v1.1 — Comms Module Integration* diff --git a/app/src/App.tsx b/app/src/App.tsx index 0711ed61..8ebb87b4 100644 --- a/app/src/App.tsx +++ b/app/src/App.tsx @@ -11,6 +11,7 @@ import { Inventory } from '@/components/modules/Inventory'; import { Settings } from '@/components/modules/Settings'; import { Catalyst } from '@/components/modules/Catalyst'; import { CRM } from '@/components/modules/CRM'; +import { Comms } from '@/components/modules/Comms'; import { NotificationCenter } from '@/components/layout/NotificationCenter'; import { useCrmBootstrap } from '@/hooks/useCrmBootstrap'; import type { ModuleId } from '@/types'; @@ -53,6 +54,7 @@ export const MODULE_ROUTES: Array<{ { id: 'inventory', path: '/inventory', title: 'Inventory', component: Inventory }, { id: 'catalyst', path: '/catalyst', title: 'The Catalyst', component: Catalyst }, { id: 'crm', path: '/crm', title: 'CRM', component: CRM }, + { id: 'comms', path: '/comms', title: 'Conversations', component: Comms }, { id: 'settings', path: '/settings', title: 'Settings', component: Settings }, { id: 'admin', path: '/admin', title: 'Admin', component: AdminPage, adminOnly: true }, ]; diff --git a/app/src/components/layout/Sidebar.tsx b/app/src/components/layout/Sidebar.tsx index 6a0cf001..2f069ce6 100644 --- a/app/src/components/layout/Sidebar.tsx +++ b/app/src/components/layout/Sidebar.tsx @@ -9,6 +9,7 @@ import { Megaphone, Shield, Users, + MessageCircle, type LucideIcon, } from 'lucide-react'; import { useStore } from '@/store/useStore'; @@ -21,6 +22,7 @@ const NAV_ICONS: Record = { '/sentinel': ScanFace, '/inventory': Building2, '/catalyst': Megaphone, + '/comms': MessageCircle, '/settings': Sliders, '/admin': Shield, '/crm': Users, diff --git a/app/src/components/modules/Comms.tsx b/app/src/components/modules/Comms.tsx new file mode 100644 index 00000000..da02378a --- /dev/null +++ b/app/src/components/modules/Comms.tsx @@ -0,0 +1,728 @@ +import { useEffect, useRef, useState } from 'react'; +import { motion, AnimatePresence } from 'framer-motion'; +import { + MessageCircle, + Search, + Send, + Phone, + PhoneCall, + Paperclip, + MoreVertical, + Link as LinkIcon, + User, + Settings, + Bot, + AlertCircle, + CheckCheck, + Clock, + ChevronLeft, + Inbox, + Voicemail, + Plus, + Hash, +} from 'lucide-react'; +import { useStore } from '@/store/useStore'; +import { + fetchCommsThreads, + fetchCommsMessages, + sendCommsMessage, + linkCommsThreadToPerson, + fetchCommsSettings, +} from '@/lib/commsApi'; +import type { + CommsThread, + CommsMessage, + CommsSettings, + SendMessagePayload, +} from '@/types/commsTypes'; + +/* ── Mock generators for demo / offline mode ─────────────────────────────── */ + +function generateMockThreads(): CommsThread[] { + const now = new Date().toISOString(); + return [ + { + threadId: 'mock-1', + phoneE164: '+919876543210', + displayName: 'Rahul Sharma', + channel: 'whatsapp', + status: 'open', + unreadCount: 2, + lastMessageAt: now, + lastMessagePreview: 'Is the 3BHK still available?', + provider: 'mock', + createdAt: now, + updatedAt: now, + crmPerson: { + id: 'p1', + fullName: 'Rahul Sharma', + primaryPhone: '+919876543210', + leadStatus: 'hot', + projectName: 'Atri Aqua', + }, + }, + { + threadId: 'mock-2', + phoneE164: '+919988776655', + displayName: 'Unknown Number', + channel: 'whatsapp', + status: 'open', + unreadCount: 1, + lastMessageAt: now, + lastMessagePreview: 'Send me the brochure please', + provider: 'mock', + createdAt: now, + updatedAt: now, + }, + { + threadId: 'mock-3', + phoneE164: '+911122334455', + displayName: 'Priya Patel', + channel: 'call', + status: 'resolved', + unreadCount: 0, + lastMessageAt: now, + provider: 'mock', + createdAt: now, + updatedAt: now, + crmPerson: { + id: 'p2', + fullName: 'Priya Patel', + primaryPhone: '+911122334455', + leadStatus: 'qualified', + projectName: 'Godrej Elevate', + }, + }, + ]; +} + +function generateMockMessages(threadId: string): CommsMessage[] { + const now = new Date(); + const t1 = new Date(now.getTime() - 1000 * 60 * 60 * 2).toISOString(); + const t2 = new Date(now.getTime() - 1000 * 60 * 30).toISOString(); + const t3 = new Date(now.getTime() - 1000 * 60 * 5).toISOString(); + + if (threadId === 'mock-1') { + return [ + { + messageId: 'm1', + threadId, + direction: 'inbound', + messageType: 'text', + body: 'Hi, I saw your listing for Atri Aqua. Is the 3BHK still available?', + deliveryStatus: 'read', + createdAt: t1, + provider: 'mock', + senderName: 'Rahul Sharma', + }, + { + messageId: 'm2', + threadId, + direction: 'outbound', + messageType: 'text', + body: 'Yes sir, absolutely. We have a premium corner unit on the 12th floor with marina view.', + deliveryStatus: 'read', + createdAt: t2, + provider: 'mock', + }, + { + messageId: 'm3', + threadId, + direction: 'inbound', + messageType: 'text', + body: 'What is the final price and can I schedule a visit this weekend?', + deliveryStatus: 'delivered', + createdAt: t3, + provider: 'mock', + senderName: 'Rahul Sharma', + }, + ]; + } + if (threadId === 'mock-2') { + return [ + { + messageId: 'm4', + threadId, + direction: 'inbound', + messageType: 'text', + body: 'Send me the brochure please', + deliveryStatus: 'delivered', + createdAt: t3, + provider: 'mock', + senderName: 'Unknown', + }, + ]; + } + return []; +} + +/* ── Component ───────────────────────────────────────────────────────────── */ + +export function Comms() { + useStore(); + const [threads, setThreads] = useState([]); + const [activeThreadId, setActiveThreadId] = useState(null); + const [messages, setMessages] = useState([]); + const [settings, setSettings] = useState(null); + const [loading, setLoading] = useState(true); + const [searchQuery, setSearchQuery] = useState(''); + const [composerText, setComposerText] = useState(''); + const [sending, setSending] = useState(false); + const [mockMode, setMockMode] = useState(false); + const [showCrmRail, setShowCrmRail] = useState(true); + const messagesEndRef = useRef(null); + + const activeThread = threads.find((t) => t.threadId === activeThreadId) || null; + + useEffect(() => { loadInitial(); }, []); + useEffect(() => { if (activeThreadId) loadMessages(activeThreadId); }, [activeThreadId]); + useEffect(() => { messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' }); }, [messages]); + + async function loadInitial() { + try { + setLoading(true); + const [threadsRes, settingsRes] = await Promise.all([ + fetchCommsThreads({ limit: 50 }), + fetchCommsSettings().catch(() => null), + ]); + setThreads(threadsRes.threads || []); + setSettings(settingsRes); + setMockMode(false); + } catch (e) { + console.warn('Comms backend unavailable, switching to mock mode', e); + setMockMode(true); + setThreads(generateMockThreads()); + setSettings({ + provider: 'mock', + webhookSecretSet: false, + autoLinkByPhone: false, + createCrmInteractionOnInbound: false, + defaultCountryCode: '91', + }); + } finally { + setLoading(false); + } + } + + async function loadMessages(threadId: string) { + try { + if (mockMode) { + setMessages(generateMockMessages(threadId)); + return; + } + const res = await fetchCommsMessages(threadId, { limit: 100 }); + setMessages(res.messages || []); + } catch (e) { + setMessages([]); + } + } + + async function handleSend(e?: React.FormEvent) { + e?.preventDefault(); + if (!composerText.trim() || !activeThreadId) return; + + const payload: SendMessagePayload = { + messageType: 'text', + body: composerText.trim(), + }; + + const optimistic: CommsMessage = { + messageId: `opt-${Date.now()}`, + threadId: activeThreadId, + direction: 'outbound', + messageType: 'text', + body: payload.body, + deliveryStatus: 'pending', + createdAt: new Date().toISOString(), + provider: activeThread?.provider || 'mock', + }; + setMessages((prev) => [...prev, optimistic]); + setComposerText(''); + setSending(true); + + try { + if (mockMode) { + await new Promise((r) => setTimeout(r, 800)); + setMessages((prev) => + prev.map((m) => + m.messageId === optimistic.messageId + ? { ...m, deliveryStatus: 'sent', messageId: `mock-${Date.now()}` } + : m + ) + ); + } else { + await sendCommsMessage(activeThreadId, payload); + await loadMessages(activeThreadId); + } + } catch (err) { + setMessages((prev) => + prev.map((m) => + m.messageId === optimistic.messageId ? { ...m, deliveryStatus: 'failed' } : m + ) + ); + } finally { + setSending(false); + } + } + + async function handleLinkPerson(personId: string) { + if (!activeThreadId || mockMode) return; + try { + await linkCommsThreadToPerson(activeThreadId, { personId }); + await loadInitial(); + if (activeThreadId) await loadMessages(activeThreadId); + } catch (e) { + console.error(e); + } + } + + const filteredThreads = threads.filter((t) => { + const q = searchQuery.toLowerCase(); + return ( + (t.displayName || '').toLowerCase().includes(q) || + t.phoneE164.includes(q) || + (t.lastMessagePreview || '').toLowerCase().includes(q) + ); + }); + + if (loading) { + return ( +
+
+
+ Loading conversations… +
+
+ ); + } + + /* ── Empty state: provider not configured ──────────────────────────────── */ + if (!mockMode && (!settings || settings.provider === 'mock')) { + return ( +
+
+
+ +
+

Conversations

+

+ Connect a WhatsApp provider to start receiving and sending messages. + Until then, preview the interface with mock data. +

+
+ + +
+
+
+ ); + } + + return ( +
+ {/* ════════════════════════════════════════════════════════════════════ + LEFT: Inbox + ════════════════════════════════════════════════════════════════════ */} + + + {/* ════════════════════════════════════════════════════════════════════ + CENTER: Conversation Timeline + ════════════════════════════════════════════════════════════════════ */} +
+ {activeThread ? ( + <> + {/* ── Client Identity Strip ── */} +
+
+
+ {activeThread.displayName?.[0]?.toUpperCase() || } +
+
+
+

+ {activeThread.displayName || activeThread.phoneE164} +

+ {activeThread.channel === 'whatsapp' && ( + + WhatsApp + + )} + {activeThread.channel === 'call' && ( + + Call + + )} +
+

{activeThread.phoneE164}

+
+
+
+ + + +
+
+ + {/* ── Messages ── */} +
+ + {messages.map((msg) => ( + +
+ {msg.messageType === 'text' &&

{msg.body}

} + {msg.mediaUrl && ( +
+ media +
+ )} +
+ + {new Date(msg.createdAt).toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' })} + + {msg.direction === 'outbound' && ( + <> + {msg.deliveryStatus === 'pending' && } + {msg.deliveryStatus === 'sent' && } + {msg.deliveryStatus === 'delivered' && } + {msg.deliveryStatus === 'read' && } + {msg.deliveryStatus === 'failed' && } + + )} +
+
+ + ))} + +
+
+ + {/* ── Composer ── */} +
+ {!activeThread.personId && ( +
+ + This number is not linked to a CRM contact. + +
+ )} +
+ +
+