Project_Velocity/.Agent Context/Sprint 1/Software Requirements Specification (SRS)_ The Sentinel _Quantum Dynamics_ Engine & Active Pipeline.md

Software Requirements Specification (SRS): The Sentinel Quantum Dynamics Engine & Active Pipeline

Updated: April 2, 2026 Status: Current truth after code implementation and AWS/NVMe deployment recovery

1. Purpose

The Sentinel is the biometric and engagement intelligence module inside Project Velocity. It combines local perception capture, broker-facing live session controls, scene-aware biometric scoring, trackable asset notifications, and automated lead enrichment into one operational pipeline.

The current implementation is no longer a purely planned architecture. The backend, frontend session flow, PostgreSQL schema, CCTV ingestion path, scene upload path, notification center, and test harness all exist in the repository. The live backend deployment has also been rebuilt on the AWS NVMe-backed node.

2. Current Architectural Truth

The architecture now operates as four cooperating layers:

1. Perception Layer MediaPipe Face Landmarker runs in the frontend via app/src/hooks/useMediapipeFaceLandmarker.ts. The browser captures webcam frames locally and emits compact blend-shape packets rather than raw video.

2. Application Layer FastAPI runs from backend/main.py. It exposes Sentinel WebSocket endpoints, Vault endpoints, Scenes upload endpoints, CCTV ingestion endpoints, and authentication.

3. Data Layer PostgreSQL is the system of record. The core schema is in backend/db/schema.sql and backend/db/schema_addendum.sql. Session, CCTV, vault, consent, scene maps, and omnichannel logs are persisted here.

4. Reasoning Layer NemoClaw prompt logic is implemented in backend/services/nemoclaw_client.py. The current production-truth inference path is NVIDIA-hosted OpenAI-compatible chat completions, not local Ollama. OpenShell and Ollama still exist on the node as related runtime infrastructure, but the FastAPI backend currently uses NVIDIA as the primary scoring path.

3. Current Runtime Topology

AWS node public IP as of April 2, 2026: 54.152.236.10

Ports:

• 22 SSH
• 443 nginx TLS reverse proxy
• 127.0.0.1:8001 FastAPI/Uvicorn backend
• 127.0.0.1:5432 PostgreSQL
• 8080 OpenShell gateway / NemoClaw gateway bootstrap target
• 11434 local Ollama runtime

Storage and runtime paths:

• Backend release root: /opt/dlami/nvme/velocity/current
• Backend environment file: /opt/dlami/nvme/velocity/env
• Backend virtualenv: /opt/dlami/nvme/velocity/venv
• Backend TLS cert/key: /opt/dlami/nvme/velocity/tls/velocity.crt and /opt/dlami/nvme/velocity/tls/velocity.key
• Prompt directory: /opt/dlami/nvme/nemoclaw/prompts
• PostgreSQL data directory: /opt/dlami/nvme/pgdata/14/velocity
• Asset directory: /opt/dlami/nvme/assets
• Marketing-video directory: /opt/dlami/nvme/assets/videos
• Marketing-video catalog: /opt/dlami/nvme/assets/videos/catalog.json

Systemd units:

• velocity-backend.service
• postgresql@14-velocity.service
• nginx
• nemoclaw-velocity.service

4. Frontend Truth

Implemented frontend components:

• app/src/components/modules/Sentinel.tsx Provides the Sentinel subtab structure with Overview and Live Session.

• app/src/components/modules/sentinel/SentinelLiveSession.tsx Implements the live workflow with: select-video -> select-mode -> select-lead -> session -> summary It now also includes scene CSV upload in the video-management flow, searchable assigned-mode lead selection, and hover-preview marketing video cards sourced from the backend catalog.

• app/src/components/modules/sentinel/PerceptionPlayer.tsx Plays the video, opens webcam capture, emits biometric packets every 500ms, receives QD updates, and closes the session through the backend on video end.

• app/src/components/layout/NotificationCenter.tsx Displays live notifications from the backend for vault opens, QD spikes, and lead-tagging events.

• app/src/hooks/useVelocitySocket.ts Uses the correct backend WebSocket namespace: /api/sentinel/ws/notifications /api/sentinel/ws/perception

5. Backend Truth

Implemented backend routers:

• backend/routers/sentinel.py Handles: /api/sentinel/ws/notifications /api/sentinel/ws/perception /api/sentinel/consent /api/sentinel/session/complete /api/sentinel/tag-lead /api/sentinel/qd-score/{lead_id}

• backend/routers/vault.py Handles trackable asset delivery and WS_ASSET_OPENED broadcast.

• backend/routers/scenes.py Handles: POST /api/scenes/upload GET /api/scenes/{video_asset_id}

• backend/routers/videos.py Handles: GET /api/videos/marketing Returns catalog-driven or directory-scanned marketing videos for the live-session picker.

• backend/routers/cctv.py Handles: POST /api/cctv/event POST /api/cctv/finalize-auto-mode

Implemented backend services:

• backend/services/nemoclaw_client.py NVIDIA API primary inference client, scene-aware QD scoring, lead tagging, CCTV visitor profiling, and malformed JSON recovery.

• backend/services/auto_mode_matcher.py Post-session matching for auto mode using plate history, tag overlap, and fallback auto-lead creation.

• backend/db/pool.py asyncpg connection pool for FastAPI.

6. Database Truth

The following tables are part of the current truth:

• users_and_roles
• leads_intelligence
• velocity_vault_assets
• omnichannel_logs
• consent_log
• video_scene_maps
• perception_sessions
• cctv_events

The addendum schema is no longer optional planning material. It is part of the required production schema.

6A. Marketing Video Asset Truth

The Sentinel live-session picker is no longer hardcoded to mock cards. The current truth is:

• The frontend requests GET /api/videos/marketing
• The backend reads /opt/dlami/nvme/assets/videos/catalog.json when present
• The backend falls back to recursive video discovery inside /opt/dlami/nvme/assets/videos
• The selected asset is streamed through /assets/videos/<file>
• The four current walkthrough assets are:
  • eden-devprayag.mp4
  • sugam-prakriti.mp4
  • atri-aqua.mp4
  • atri-surya-toron.mp4

This matters because adding more walkthrough assets is now a backend-content operation, not a frontend rebuild requirement.

7. NemoClaw / Reasoning Truth

The original plan assumed local agent-contained reasoning through OpenShell plus local Ollama. That is no longer the active scoring path.

Current truth:

• The backend client uses NVIDIA-hosted OpenAI-compatible completions as primary inference.
• Current primary model: nvidia/nemotron-3-super-120b-a12b
• Current fallback model: nvidia/llama-3.3-nemotron-super-49b-v1
• Local Ollama remains installed on 11434 but is not the active backend primary.
• OpenShell gateway remains present on 8080 and is bootstrapped by nemoclaw-velocity.service.

Reason for this change:

• The local Ollama path was not reliably producing the strict JSON contract needed for QD scoring.
• The NVIDIA path is OpenAI-compatible and easier to integrate directly from FastAPI.
• The NVIDIA path is now what the code and live environment are configured to use.

8. Milestone Truth

Milestone 1: Complete Database schema, auth dependencies, store/types, and live PostgreSQL deployment exist.

Milestone 2: Complete Vault generation, notification ingestion, and frontend notification center are implemented.

Milestone 3: Complete Perception pipeline frontend, live session flow, and packet emission are implemented.

Milestone 4: Complete NemoClaw prompt system and backend scoring/tagging routes are implemented. The inference truth changed from local Ollama-first planning to NVIDIA-primary production use.

Milestone 5: Code complete, operational feed integration still pending CCTV router, OCR bridge script, and auto-mode matcher exist. What remains is wiring a real ONVIF/RTSP/OCR producer into /api/cctv/event.

Milestone 6: Complete Scene upload backend, scene lookup, frontend upload UI, and scene-aware packet correlation are implemented.

Milestone 7: Complete NVMe-backed backend deployment, env file, PostgreSQL, systemd, nginx TLS, and prompt placement are in place.

Milestone 8: Complete at repo test level, still desirable to run a broker-facing live acceptance pass Backend pytest tests and frontend Playwright tests exist and pass locally.

9. Known Risks

1. NVIDIA model JSON consistency The NVIDIA model is reachable and active, but it does not always honor strict JSON perfectly for the full QD prompt. The client now includes malformed-response recovery, but this remains the main live inference risk.

2. Dynamic public IP churn The node public IP changed during execution. The system still needs either an Elastic IP or a stable DNS record.

3. Self-signed TLS nginx currently serves TLS, but a trusted certificate path is still preferable for production rollout.

4. External CCTV source not yet hard-connected The ingest API exists, but a real external bridge process still needs to be pointed at it in production.

5. Video catalog hygiene If operators copy MP4s to NVMe without updating catalog.json, the fallback scan will keep the assets visible, but broker-facing order and labels may become inconsistent.

10. Acceptance Criteria

The Sentinel is considered aligned with current truth when all of the following hold:

• https://54.152.236.10/health returns backend status ok
• PostgreSQL is live on NVMe and all eight tables exist
• velocity-backend.service is active
• nginx is active on 443
• nemoclaw-velocity.service is active
• POST /api/scenes/upload accepts a valid CSV
• POST /api/cctv/event persists a CCTV event
• POST /api/sentinel/session/complete finalizes assigned or auto sessions
• GET /api/videos/marketing returns the current marketing-video catalog
• Live QD updates are visible in the frontend through the Sentinel WebSocket path
• Playwright tests and backend tests pass

11. Appendices

Related documents:

• velocity_status_report.md
• nemoclaw_setup_truth.md
• The Sentinel Bibel.md