feat: Whatsapp Integration

2026-04-28 13:41:14 +05:30
parent 7ee51543d9
commit 3623bacbac
15 changed files with 2549 additions and 3 deletions
--- a/Context/COMMS_INTEGRATION_HANDOFF.md
+++ b/Context/COMMS_INTEGRATION_HANDOFF.md
@@ -0,0 +1,265 @@
+# Velocity Comms Integration — Handoff Document
+
+## 1. Architecture Recommendation
+
+### Goal
+Add a native **Conversations** module to Project Velocity so brokers/agents can manage WhatsApp (and future SMS/call) threads without leaving the WebOS.
+
+### Design Philosophy
+- **Native Velocity UI**: Dark glass panels, compact density, blue accent, no iframe embeds.
+- **Provider-agnostic backend**: Abstract `CommsProvider` class with adapter pattern.
+- **CRM-first**: Every thread attempts to link to `crm_people` by `primary_phone`. Unresolved numbers are surfaced for manual linking.
+- **Mock-first development**: The module renders fully without real credentials.
+
+### Provider Comparison
+
+| Provider | Best For | Velocity Fit | 72-Hour Viability |
+|----------|----------|--------------|-------------------|
+| **Chatwoot** | Full support suite (email, SMS, WA) | Too heavy to embed; good UX reference | Low — would require stripping UI |
+| **WAHA** | Lightweight WhatsApp Web gateway | Good adapter candidate | High — simple REST, easy webhooks |
+| **Evolution API** | Modern WA gateway with groups, status, typing | Best adapter candidate | **High** — active community, clean webhooks |
+| **Meta Cloud API** | Official WABA; template-based outbound | Required for production scale at large builders | Medium — needs Meta Business verification |
+
+**Recommended 72-hour route:**
+1. **Day 1**: Merge schema + backend routes + mock provider. Frontend compiles with mock data.
+2. **Day 2**: Connect Evolution API or WAHA in a staging environment. Test inbound webhook → thread creation.
+3. **Day 3**: CRM linking, settings UI, call-log upload placeholder, and smoke tests.
+
+For production, plan a **dual-provider** setup:
+- **Evolution/WAHA** for quick conversational messaging (no Meta approval needed).
+- **Meta Cloud API** for template-based broadcast/re-engagement once Business Manager is verified.
+
+---
+
+## 2. Exact Files Created
+
+```
+app/src/types/commsTypes.ts
+app/src/lib/commsApi.ts
+app/src/components/modules/Comms.tsx
+backend/db/schema_comms.sql
+backend/services/comms_provider.py
+backend/services/comms_waha_provider.py
+backend/services/comms_evolution_provider.py
+backend/services/comms_ingest.py
+backend/api/routes_comms.py
+COMMS_INTEGRATION_HANDOFF.md
+```
+
+---
+
+## 3. Patch Instructions for Existing Files
+
+### A. `app/src/types/index.ts`
+Add `'comms'` to the `ModuleId` union:
+```typescript
+export type ModuleId = 'dashboard' | 'oracle' | 'sentinel' | 'inventory' | 'settings' | 'catalyst' | 'admin' | 'crm' | 'comms';
+```
+
+### B. `app/src/App.tsx`
+1. Import the new component:
+```typescript
+import { Comms } from '@/components/modules/Comms';
+```
+2. Insert the route into `MODULE_ROUTES` **just before** `settings`:
+```typescript
+{ id: 'comms', path: '/comms', title: 'Conversations', component: Comms },
+```
+
+### C. `app/src/components/layout/Sidebar.tsx`
+1. Import a new icon:
+```typescript
+import { MessageCircle } from 'lucide-react';
+```
+2. Add to `NAV_ICONS`:
+```typescript
+const NAV_ICONS: Record<string, LucideIcon> = {
+  '/dashboard': LayoutGrid,
+  '/oracle':    MessageSquarePlus,
+  '/sentinel':  ScanFace,
+  '/inventory': Building2,
+  '/catalyst':  Megaphone,
+  '/comms':     MessageCircle,   // ← NEW
+  '/settings':  Sliders,
+  '/admin':     Shield,
+  '/crm':       Users,
+};
+```
+
+### D. `backend/main.py`
+1. Import the router near the other imports:
+```python
+from backend.api.routes_comms import router as comms_router
+```
+2. Include it after the other routers:
+```python
+app.include_router(comms_router, prefix="/api/comms", tags=["Comms"])
+```
+
+---
+
+## 4. Environment Variables
+
+Add these to your `.env` or systemd environment:
+
+```bash
+# Provider selection: mock | waha | evolution | meta_cloud
+COMMS_PROVIDER=mock
+
+# Provider connectivity
+COMMS_PROVIDER_BASE_URL=
+COMMS_PROVIDER_API_KEY=
+COMMS_INSTANCE_ID=default
+
+# Webhook security
+COMMS_WEBHOOK_SECRET=
+
+# Phone normalization
+COMMS_DEFAULT_COUNTRY_CODE=91
+
+# Media storage
+COMMS_MEDIA_STORAGE_DIR=/opt/dlami/nvme/assets/comms
+
+# Transcription (none | openai | local)
+COMMS_TRANSCRIPTION_PROVIDER=none
+```
+
+**No secrets are hardcoded in source.**
+
+---
+
+## 5. Database Migration
+
+Run the SQL file against your Postgres database:
+
+```bash
+psql $DATABASE_URL -f backend/db/schema_comms.sql
+```
+
+Tables created:
+- `comms_threads` — conversation headers with CRM link
+- `comms_messages` — individual messages (inbound/outbound/system)
+- `comms_call_logs` — call records with optional transcript
+- `comms_settings` — key-value config store
+
+---
+
+## 6. API Routes
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| GET | `/api/comms/threads` | List threads (search, status, pagination) |
+| GET | `/api/comms/threads/{id}` | Get single thread with CRM enrichment |
+| GET | `/api/comms/threads/{id}/messages` | Chronological messages |
+| POST | `/api/comms/threads/{id}/messages` | Send outbound message via provider |
+| POST | `/api/comms/threads/{id}/link-person` | Link thread to `crm_people.id` |
+| POST | `/api/comms/threads/{id}/notes` | Add system note |
+| POST | `/api/comms/threads/{id}/tasks` | Add system task |
+| POST | `/api/comms/webhooks/{provider}` | Public webhook endpoint |
+| GET | `/api/comms/settings` | Get comms configuration |
+| PATCH | `/api/comms/settings` | Update configuration |
+| POST | `/api/comms/provider/test` | Test provider connectivity |
+| POST | `/api/comms/recordings/transcribe` | Queue transcription job |
+
+---
+
+## 7. Frontend Route Changes
+
+- New sidebar item: **Conversations** (icon: `MessageCircle`)
+- Position: directly above **Settings**
+- Route: `/comms`
+- Component: `Comms.tsx` with three-pane layout (Inbox | Chat | CRM Rail)
+
+---
+
+## 8. Settings Changes
+
+A new **Communications** subsection should be added inside your existing Settings module (or as a standalone card). Fields:
+
+| Field | Type | Description |
+|-------|------|-------------|
+| Provider | select | mock / waha / evolution / meta_cloud |
+| Provider Base URL | text | e.g. `http://localhost:3000` |
+| API Key | password | masked after save |
+| Instance ID | text | WA/Evolution session name |
+| Phone Number ID | text | Meta Cloud API only |
+| Webhook Callback URL | text | Auto-populated or custom |
+| Webhook Secret | password | Sets `webhook_secret_set` flag |
+| Default Assignment User | select | User dropdown from `/api/auth/users` |
+| Auto-link by Phone | toggle | Match `crm_people.primary_phone` automatically |
+| Create CRM Interaction on Inbound | toggle | Write to `intel_interactions` if table exists |
+| Default Country Code | text | e.g. `91` for India |
+| Transcription Provider | select | none / openai / local |
+| Connection Test | button | Calls `POST /api/comms/provider/test` |
+
+---
+
+## 9. Smoke Test Steps
+
+1. **DB**: Run `schema_comms.sql`. Verify tables exist.
+2. **Backend**: Start FastAPI. Confirm `/health` returns `db_pool: connected`.
+3. **Backend**: `curl -X POST http://localhost:8000/api/comms/provider/test` → should return mock success.
+4. **Frontend**: Load Velocity. Sidebar should show **Conversations**.
+5. **Frontend**: Click Conversations. Mock mode should render 3 threads and messages.
+6. **Frontend**: Send a message in mock thread. Optimistic update → mock delivery checkmark.
+7. **Backend**: Post sample webhook:
+   ```bash
+   curl -X POST http://localhost:8000/api/comms/webhooks/evolution      -H "Content-Type: application/json"      -d '{"event":"messages.upsert","instance":"default","data":{"key":{"remoteJid":"919876543210@s.whatsapp.net","fromMe":false,"id":"test-1"},"message":{"conversation":"Hello from webhook"},"messageTimestamp":1710000000}}'
+   ```
+8. **Backend**: Verify thread + message inserted. Check `comms_threads` for new row.
+9. **Frontend**: Refresh inbox. New thread should appear.
+10. **CRM Link**: Click "Link to Contact" (or call `POST /api/comms/threads/{id}/link-person`) and verify `person_id` is set.
+
+---
+
+## 10. Known Limitations
+
+- **Call recording via WhatsApp API**: Neither WAHA nor Evolution supports native WhatsApp call recording. Call logs are designed for **external telephony intake** (manual upload or webhook from a PBX/VoIP system). Recording file + transcript workflow is scaffolded but needs a real transcription provider (OpenAI Whisper, AWS Transcribe, or faster-whisper) wired in.
+- **Media downloads**: `get_media()` is stubbed for WAHA/Evolution. Production needs signed URL handling or local file download.
+- **Meta Cloud API adapter**: Not yet implemented. Add `comms_meta_provider.py` when Meta Business verification is complete.
+- **Template messages**: Only placeholder methods exist. Template approval flow (Meta) or local template storage must be built for outbound campaigns.
+- **Webhook auth**: Currently accepts any payload. Add HMAC/signature verification per provider before production.
+- **Rate limiting**: Not implemented. Add FastAPI rate-limit middleware on `/api/comms/webhooks/{provider}`.
+- **phonenumbers library**: `comms_ingest.py` gracefully degrades to regex if `phonenumbers` is not installed. Install it for robust E.164 normalization:
+  ```bash
+  pip install phonenumbers
+  ```
+
+---
+
+## 11. What Still Needs Real Credentials
+
+| Item | What You Need |
+|------|---------------|
+| **Evolution API** | A running Evolution instance (Docker), API key, and a paired WhatsApp number. |
+| **WAHA** | A running WAHA container, session QR-scan, and API key. |
+| **Meta Cloud API** | Meta Business Manager, verified business, WhatsApp Business Account, permanent access token, phone number ID. |
+| **Transcription** | OpenAI API key (for Whisper) or local faster-whisper model path. |
+| **CRM enrichment** | Ensure `crm_people` table exists with `primary_phone` indexed. |
+
+---
+
+## 12. What to Verify Before Production
+
+- [ ] Webhook endpoint is exposed via HTTPS (ngrok/cloudflare tunnel for local dev).
+- [ ] `COMMS_WEBHOOK_SECRET` is set and signature verification is enabled in `routes_comms.py`.
+- [ ] Database has indexes on `comms_threads(phone_e164)` and `comms_messages(thread_id, created_at)`.
+- [ ] `crm_people.primary_phone` is normalized to E.164 before comms matching.
+- [ ] Media storage directory exists and is writable (`COMMS_MEDIA_STORAGE_DIR`).
+- [ ] Outbound message queue / retry logic is added (currently synchronous).
+- [ ] GDPR/opt-out handling is implemented if targeting EU markets.
+- [ ] Backup strategy for `comms_messages` (contains legal conversation records).
+
+---
+
+## 13. Next Iteration Ideas
+
+- **Bulk broadcast**: Template-based outbound to filtered CRM segments.
+- **AI reply suggestions**: Integrate Oracle / local LLM to draft replies based on CRM context.
+- **Voice notes**: Upload `.ogg` audio, transcribe, store transcript as message.
+- **Read receipts**: Poll provider for delivery/read status and update `comms_messages`.
+- **Assignment rules**: Round-robin or load-based auto-assignment to agents.
+
+---
+
+*Document generated for Project Velocity v1.1 — Comms Module Integration*