feat: Complete code integration of modules (#18)

The complete code integration is done.

Co-authored-by: Sagnik <sagnik7896@gmail.com>
Reviewed-on: sagnik/Project_Velocity#18

.Agent Context/Issues/2026-04-12-sourik-root-integration-closure.md

@@ -0,0 +1,135 @@
+# Sourik Root Integration Closure
+
+## Summary
+
+This issue tracks the root-side integration of the useful Sourik subsystems into the current Project Velocity mainline without replacing the root FastAPI shell, root Sentinel ownership, or the existing Python-native Oracle v1 surface.
+
+The objective was to absorb the missing operational pieces into the root codebase so Sprint 1 truth is judged by current product reality rather than by stale planning artifacts.
+
+## Scope Closed In This Pass
+
+### CRM backend and sync
+
+- landed canonical root CRM routes for:
+  - `GET/POST/PUT/DELETE /api/leads`
+  - `GET /api/leads/{lead_id}`
+  - `GET /api/leads/demographics`
+  - `GET/POST /api/chat-logs`
+  - `GET /api/kanban/board`
+  - `PUT /api/kanban/move`
+  - `POST /api/leads/seed-synthetic`
+- added dedicated CRM websocket stream at `GET /ws/crm`
+- added root-side CRM event broadcasting for:
+  - lead create
+  - lead update
+  - lead delete
+  - kanban move
+  - chat log create
+  - synthetic seed runs
+  - Oracle writebacks
+
+### Oracle append and writeback contract
+
+- kept root Oracle v1 as the primary public Oracle surface
+- appended persona and workflow orchestration under the existing Oracle v1 flow
+- added real MCP execution endpoint:
+  - `POST /api/oracle/mcp/execute`
+- added Oracle action ledger and read/writeback routes:
+  - `GET /api/oracle/actions`
+  - `GET /api/oracle/actions/{action_id}`
+  - `POST /api/oracle/actions/writeback`
+- Oracle prompt submissions now create persisted action records tied to execution output
+- Oracle writebacks now support canonical lead updates for:
+  - score adjustments
+  - stage changes
+  - qualification changes
+  - metadata patching
+  - note append
+  - Oracle-generated message insertion into `chat_logs`
+
+### MCP and search
+
+- converted the MCP registry from a placeholder slot into an executable root service
+- external search now executes against:
+  - Brave Search if `BRAVE_API_KEY` is configured
+  - DuckDuckGo fallback otherwise
+- CRM and local property retrieval tools now execute against the root CRM schema through the root DB pool
+
+### Catalyst marketing backend parity
+
+- replaced hardcoded campaign summaries with unified ad-network service backed by root code
+- added root Meta plus Google Ads unified campaign listing
+- added unified insights endpoint with platform filtering
+- added budget update route for Meta plus Google
+- added bid strategy update route for Meta plus Google
+- added Google-aware campaign creation path so Catalyst campaign creation is no longer Meta-only
+
+Routes covered in this pass:
+
+- `GET /api/catalyst/campaigns`
+- `POST /api/catalyst/campaigns/create`
+- `GET /api/catalyst/insights/realtime`
+- `PUT /api/catalyst/budget`
+- `PUT /api/catalyst/bid-strategy`
+
+### Frontend carry-forward
+
+- preserved the existing root Catalyst shell
+- kept the vertically stacked `Marketing` sub-tab inside Catalyst
+- no second marketing app or second frontend API source of truth was introduced
+
+## Files Added Or Materially Updated
+
+### Backend
+
+- `backend/services/ad_network_service.py`
+- `backend/services/mcp_registry.py`
+- `backend/api/routes_catalyst.py`
+- `backend/api/routes_crm.py`
+- `backend/api/routes_oracle.py`
+- `backend/oracle/action_service.py`
+- `backend/oracle/router_v1.py`
+- `backend/main.py`
+
+### Tests
+
+- `backend/tests/test_catalyst_routes.py`
+- `backend/tests/test_oracle_routes.py`
+- `backend/tests/test_crm_websocket.py`
+
+## Verification Completed
+
+- `python -m pytest Project_Velocity/backend/tests/test_catalyst_routes.py Project_Velocity/backend/tests/test_oracle_routes.py Project_Velocity/backend/tests/test_crm_websocket.py Project_Velocity/backend/tests/test_crm_routes.py Project_Velocity/backend/tests/oracle/test_persona_service.py Project_Velocity/backend/tests/test_nemoclaw_runtime.py`
+- result: `9 passed`
+
+- `npm run build`
+- result: passed
+
+## Production Notes
+
+- Google Ads support is now integrated at the root contract level, but live mutate behavior still depends on valid provider credentials and provider-managed operations.
+- Brave Search becomes the preferred external search provider when `BRAVE_API_KEY` is present; otherwise the root falls back to DuckDuckGo.
+- Oracle writebacks currently target leads as the canonical CRM entity. Additional entity writebacks should follow the same `oracle_actions` ledger rather than introducing side-channel writes.
+
+## Residual Work After This Closure
+
+These are still separate follow-up items, not blockers for closing this integration pass:
+
+- deeper Google Ads mutate coverage beyond provider-managed passthroughs
+- frontend consumption of the CRM websocket stream
+- broader Oracle writebacks beyond `lead`
+- stricter auth and role gating for Oracle action application
+- richer Catalyst campaign creation UX for platform-specific fields
+- prompt inventory and persona-to-runtime mapping docs cleanup
+
+## Acceptance Criteria Met
+
+- root app shell preserved
+- root FastAPI entrypoint preserved
+- root Sentinel ownership preserved
+- no Go runtime adopted
+- no second backend center introduced
+- MCP external search executes for real
+- CRM has a live websocket sync surface
+- Oracle has a persisted action/writeback contract
+- Catalyst backend exposes Google-aware parity routes in the root

.Agent Context/Sprint 1/nemoclaw_setup_truth.md

@@ -1,6 +1,6 @@
 # NemoClaw Setup Truth
 
-Updated: April 2, 2026
+Updated: April 12, 2026
 
 ## 1. Purpose
 
@@ -10,15 +10,24 @@ This is not the original intended architecture. This is the current operational
 
 ## 2. High-Level Summary
 
-Project Velocity uses the term "NemoClaw" for the reasoning and prompt layer attached to the Sentinel QD Engine. In practice, this is now split into two different concerns:
+Project Velocity uses the term "NemoClaw" for the reasoning and prompt layer attached to the Sentinel QD Engine. In practice, this is now split into three different concerns:
 
 1. Prompted reasoning used by the FastAPI backend
 2. OpenShell / gateway infrastructure that remains installed on the AWS node
+3. Python-native append layers used by Oracle planning, MCP-style tool registration, and workflow dispatch preview
 
 The active FastAPI inference path is NVIDIA-hosted OpenAI-compatible chat completions.
 
 The OpenShell gateway and Ollama are still installed and running as adjacent infrastructure, but they are not the active primary scoring path used by `backend/services/nemoclaw_client.py`.
 
+The root codebase now also includes Python-native compatibility layers inspired by Sourik's Go runtime:
+
+- `backend/services/nemoclaw_runtime.py`
+- `backend/services/mcp_registry.py`
+- `backend/oracle/persona_service.py`
+
+These append the current root without replacing the active NVIDIA-hosted inference path.
+
 ## 3. Node and Network Truth
 
 AWS region: `us-east-1`
@@ -81,6 +90,24 @@ PostgreSQL 14 data directory.
 `backend/services/nemoclaw_client.py`
 Primary reasoning client used by the FastAPI backend.
 
+`backend/services/nemoclaw_runtime.py`
+Python-native append layer for workflow dispatch planning, webhook verification, and claim-style helper behavior.
+
+`backend/services/mcp_registry.py`
+Python-native MCP/search tool registry append layer used by Oracle helper surfaces.
+
+`backend/oracle/persona_service.py`
+Subordinate Oracle persona planning layer that recommends component templates, renders prompt assets, and augments Oracle v1.
+
+`backend/api/routes_crm.py`
+Root PostgreSQL-first CRM append layer for `leads`, `chat_logs`, `kanban`, and analytics routes.
+
+`backend/api/routes_oracle.py`
+Root Oracle helper append layer for workflow preview and MCP tool discovery.
+
+`backend/oracle/router_v1.py`
+Mounted Oracle v1 API surface for canvas, prompts, persona helpers, and collaboration.
+
 `backend/routers/videos.py`
 Marketing-video catalog endpoint for the Sentinel live-session picker.
 
@@ -182,6 +209,28 @@ No longer the primary path for backend scoring.
 5. The backend calls NVIDIA hosted completions using `nvidia/nemotron-3-super-120b-a12b`
 6. The result updates QD score state and is broadcast back over WebSocket
 
+### Current Oracle canvas planning append flow
+
+1. Frontend can call `/api/oracle/v1/canvas-pages/{pageId}/prompts`
+2. `backend/oracle/prompt_orchestrator.py` builds a retrieval plan
+3. `backend/oracle/persona_service.py` recommends reusable component templates and emits a planning note block
+4. `backend/services/nemoclaw_runtime.py` produces a workflow dispatch preview for ComfyUI-backed execution
+5. `backend/oracle/data_access_gateway.py` runs only whitelisted PostgreSQL queries
+6. Oracle commits the resulting components into the active canvas revision
+
+### Current CRM and analytics append flow
+
+1. Root FastAPI mounts `backend/api/routes_crm.py`
+2. Canonical root endpoints now exist for:
+   - `/api/leads`
+   - `/api/leads/demographics`
+   - `/api/chat-logs`
+   - `/api/kanban/board`
+   - `/api/kanban/move`
+   - `/api/analytics/sentiment-scatter`
+3. These routes use the root asyncpg pool and PostgreSQL-first storage contract
+4. CRM WebSocket sync is still intentionally deferred
+
 ### Current lead-tagging flow
 
 1. Broker or system calls `/api/sentinel/tag-lead`
@@ -232,6 +281,22 @@ Why it still exists:
 
 What it is not:
 - It is not the current primary inference path for backend scoring
+- It is not the root source of truth for Oracle or CRM orchestration
+
+## 8.5 Python-Native Append Responsibilities
+
+These are now part of root truth:
+
+- Oracle persona prompt loading and render helpers live in Python, not Go
+- MCP/search registration lives in Python, not Go
+- Workflow dispatch planning for Oracle-to-Comfy orchestration lives in Python, not Go
+- Claim-style helper behavior is appended in Python as a compatibility layer, not as a second backend center
+
+What remains deferred:
+
+- Full production webhook runtime parity with Sourik's Go stack
+- Full external search provider execution inside the MCP layer
+- Autonomous posting and non-root agent/webhook services
 
 ## 9. Prompts