### Software Requirements Specification (SRS): The Sentinel Quantum Dynamics Engine & Active Pipeline Updated: April 2, 2026 Status: Current truth after code implementation and AWS/NVMe deployment recovery #### 1. Purpose The Sentinel is the biometric and engagement intelligence module inside Project Velocity. It combines local perception capture, broker-facing live session controls, scene-aware biometric scoring, trackable asset notifications, and automated lead enrichment into one operational pipeline. The current implementation is no longer a purely planned architecture. The backend, frontend session flow, PostgreSQL schema, CCTV ingestion path, scene upload path, notification center, and test harness all exist in the repository. The live backend deployment has also been rebuilt on the AWS NVMe-backed node. #### 2. Current Architectural Truth The architecture now operates as four cooperating layers: 1. Perception Layer MediaPipe Face Landmarker runs in the frontend via `app/src/hooks/useMediapipeFaceLandmarker.ts`. The browser captures webcam frames locally and emits compact blend-shape packets rather than raw video. 2. Application Layer FastAPI runs from `backend/main.py`. It exposes Sentinel WebSocket endpoints, Vault endpoints, Scenes upload endpoints, CCTV ingestion endpoints, and authentication. 3. Data Layer PostgreSQL is the system of record. The core schema is in `backend/db/schema.sql` and `backend/db/schema_addendum.sql`. Session, CCTV, vault, consent, scene maps, and omnichannel logs are persisted here. 4. Reasoning Layer NemoClaw prompt logic is implemented in `backend/services/nemoclaw_client.py`. The current production-truth inference path is NVIDIA-hosted OpenAI-compatible chat completions, not local Ollama. OpenShell and Ollama still exist on the node as related runtime infrastructure, but the FastAPI backend currently uses NVIDIA as the primary scoring path. #### 3. Current Runtime Topology AWS node public IP as of April 2, 2026: `54.152.236.10` Ports: - `22` SSH - `443` nginx TLS reverse proxy - `127.0.0.1:8001` FastAPI/Uvicorn backend - `127.0.0.1:5432` PostgreSQL - `8080` OpenShell gateway / NemoClaw gateway bootstrap target - `11434` local Ollama runtime Storage and runtime paths: - Backend release root: `/opt/dlami/nvme/velocity/current` - Backend environment file: `/opt/dlami/nvme/velocity/env` - Backend virtualenv: `/opt/dlami/nvme/velocity/venv` - Backend TLS cert/key: `/opt/dlami/nvme/velocity/tls/velocity.crt` and `/opt/dlami/nvme/velocity/tls/velocity.key` - Prompt directory: `/opt/dlami/nvme/nemoclaw/prompts` - PostgreSQL data directory: `/opt/dlami/nvme/pgdata/14/velocity` - Asset directory: `/opt/dlami/nvme/assets` - Marketing-video directory: `/opt/dlami/nvme/assets/videos` - Marketing-video catalog: `/opt/dlami/nvme/assets/videos/catalog.json` Systemd units: - `velocity-backend.service` - `postgresql@14-velocity.service` - `nginx` - `nemoclaw-velocity.service` #### 4. Frontend Truth Implemented frontend components: - `app/src/components/modules/Sentinel.tsx` Provides the Sentinel subtab structure with `Overview` and `Live Session`. - `app/src/components/modules/sentinel/SentinelLiveSession.tsx` Implements the live workflow with: `select-video -> select-mode -> select-lead -> session -> summary` It now also includes scene CSV upload in the video-management flow, searchable assigned-mode lead selection, and hover-preview marketing video cards sourced from the backend catalog. - `app/src/components/modules/sentinel/PerceptionPlayer.tsx` Plays the video, opens webcam capture, emits biometric packets every 500ms, receives QD updates, and closes the session through the backend on video end. - `app/src/components/layout/NotificationCenter.tsx` Displays live notifications from the backend for vault opens, QD spikes, and lead-tagging events. - `app/src/hooks/useVelocitySocket.ts` Uses the correct backend WebSocket namespace: `/api/sentinel/ws/notifications` `/api/sentinel/ws/perception` #### 5. Backend Truth Implemented backend routers: - `backend/routers/sentinel.py` Handles: `/api/sentinel/ws/notifications` `/api/sentinel/ws/perception` `/api/sentinel/consent` `/api/sentinel/session/complete` `/api/sentinel/tag-lead` `/api/sentinel/qd-score/{lead_id}` - `backend/routers/vault.py` Handles trackable asset delivery and `WS_ASSET_OPENED` broadcast. - `backend/routers/scenes.py` Handles: `POST /api/scenes/upload` `GET /api/scenes/{video_asset_id}` - `backend/routers/videos.py` Handles: `GET /api/videos/marketing` Returns catalog-driven or directory-scanned marketing videos for the live-session picker. - `backend/routers/cctv.py` Handles: `POST /api/cctv/event` `POST /api/cctv/finalize-auto-mode` Implemented backend services: - `backend/services/nemoclaw_client.py` NVIDIA API primary inference client, scene-aware QD scoring, lead tagging, CCTV visitor profiling, and malformed JSON recovery. - `backend/services/auto_mode_matcher.py` Post-session matching for auto mode using plate history, tag overlap, and fallback auto-lead creation. - `backend/db/pool.py` asyncpg connection pool for FastAPI. #### 6. Database Truth The following tables are part of the current truth: - `users_and_roles` - `leads_intelligence` - `velocity_vault_assets` - `omnichannel_logs` - `consent_log` - `video_scene_maps` - `perception_sessions` - `cctv_events` The addendum schema is no longer optional planning material. It is part of the required production schema. #### 6A. Marketing Video Asset Truth The Sentinel live-session picker is no longer hardcoded to mock cards. The current truth is: - The frontend requests `GET /api/videos/marketing` - The backend reads `/opt/dlami/nvme/assets/videos/catalog.json` when present - The backend falls back to recursive video discovery inside `/opt/dlami/nvme/assets/videos` - The selected asset is streamed through `/assets/videos/` - The four current walkthrough assets are: - `eden-devprayag.mp4` - `sugam-prakriti.mp4` - `atri-aqua.mp4` - `atri-surya-toron.mp4` This matters because adding more walkthrough assets is now a backend-content operation, not a frontend rebuild requirement. #### 7. NemoClaw / Reasoning Truth The original plan assumed local agent-contained reasoning through OpenShell plus local Ollama. That is no longer the active scoring path. Current truth: - The backend client uses NVIDIA-hosted OpenAI-compatible completions as primary inference. - Current primary model: `nvidia/nemotron-3-super-120b-a12b` - Current fallback model: `nvidia/llama-3.3-nemotron-super-49b-v1` - Local Ollama remains installed on `11434` but is not the active backend primary. - OpenShell gateway remains present on `8080` and is bootstrapped by `nemoclaw-velocity.service`. Reason for this change: - The local Ollama path was not reliably producing the strict JSON contract needed for QD scoring. - The NVIDIA path is OpenAI-compatible and easier to integrate directly from FastAPI. - The NVIDIA path is now what the code and live environment are configured to use. #### 8. Milestone Truth Milestone 1: Complete Database schema, auth dependencies, store/types, and live PostgreSQL deployment exist. Milestone 2: Complete Vault generation, notification ingestion, and frontend notification center are implemented. Milestone 3: Complete Perception pipeline frontend, live session flow, and packet emission are implemented. Milestone 4: Complete NemoClaw prompt system and backend scoring/tagging routes are implemented. The inference truth changed from local Ollama-first planning to NVIDIA-primary production use. Milestone 5: Code complete, operational feed integration still pending CCTV router, OCR bridge script, and auto-mode matcher exist. What remains is wiring a real ONVIF/RTSP/OCR producer into `/api/cctv/event`. Milestone 6: Complete Scene upload backend, scene lookup, frontend upload UI, and scene-aware packet correlation are implemented. Milestone 7: Complete NVMe-backed backend deployment, env file, PostgreSQL, systemd, nginx TLS, and prompt placement are in place. Milestone 8: Complete at repo test level, still desirable to run a broker-facing live acceptance pass Backend pytest tests and frontend Playwright tests exist and pass locally. #### 9. Known Risks 1. NVIDIA model JSON consistency The NVIDIA model is reachable and active, but it does not always honor strict JSON perfectly for the full QD prompt. The client now includes malformed-response recovery, but this remains the main live inference risk. 2. Dynamic public IP churn The node public IP changed during execution. The system still needs either an Elastic IP or a stable DNS record. 3. Self-signed TLS nginx currently serves TLS, but a trusted certificate path is still preferable for production rollout. 4. External CCTV source not yet hard-connected The ingest API exists, but a real external bridge process still needs to be pointed at it in production. 5. Video catalog hygiene If operators copy MP4s to NVMe without updating `catalog.json`, the fallback scan will keep the assets visible, but broker-facing order and labels may become inconsistent. #### 10. Acceptance Criteria The Sentinel is considered aligned with current truth when all of the following hold: - `https://54.152.236.10/health` returns backend status `ok` - PostgreSQL is live on NVMe and all eight tables exist - `velocity-backend.service` is active - `nginx` is active on `443` - `nemoclaw-velocity.service` is active - `POST /api/scenes/upload` accepts a valid CSV - `POST /api/cctv/event` persists a CCTV event - `POST /api/sentinel/session/complete` finalizes assigned or auto sessions - `GET /api/videos/marketing` returns the current marketing-video catalog - Live QD updates are visible in the frontend through the Sentinel WebSocket path - Playwright tests and backend tests pass #### 11. Appendices Related documents: - `velocity_status_report.md` - `nemoclaw_setup_truth.md` - `The Sentinel Bibel.md`