Project_Velocity/.Agent Context/Sprint 1/The Sentinel Bibel.md

The Sentinel Bibel

Updated: April 2, 2026

1. What The Sentinel Is

The Sentinel is the biometric, session-intelligence, and attention-scoring engine inside Project Velocity. It exists to help brokers understand how a prospect reacts to a guided property walkthrough in real time, without streaming raw webcam video to a remote backend.

It fuses:

• browser-side facial blend-shape capture
• scene-aware timestamp correlation
• CRM context
• asset-open notifications
• CCTV-assisted auto-mode enrichment
• lead tagging and session finalization

The output is not just a number. The output is operational intelligence.

2. Core Principles

Local-first perception

The webcam is processed in the browser through MediaPipe. The backend receives only compact biometric packets, not raw video.

NVMe-first deployment

Runtime data, prompts, PostgreSQL, certificates, and backend release assets belong on /opt/dlami/nvme, not the root volume.

PostgreSQL is the source of truth

No Supabase database runtime is used for the Sentinel workflow. Session, consent, vault, CCTV, and sentiment history all land in PostgreSQL.

Broker-facing intelligence must be real time

The system pushes QD updates and vault-open signals back through WebSockets so the broker can act during or immediately after the session.

Architecture follows truth, not plan nostalgia

The original plan centered more heavily on local Ollama/OpenShell inference. The current deployed truth is NVIDIA-primary reasoning with OpenShell/Ollama still present as neighboring infrastructure.

3. How The Sentinel Works

Assigned Mode

1. Broker selects a marketing video
2. Broker selects Assigned Mode
3. Broker selects an existing lead
4. Prospect watches the video while the browser emits biometric packets
5. Backend scores QD using scene context plus CRM context
6. Lead score updates and notifications are visible in real time

Auto Mode

1. Broker selects a marketing video
2. Broker selects Auto Mode
3. Session runs without a pre-bound lead
4. Browser emits biometric packets tied to a session UUID
5. CCTV events enrich the same session through /api/cctv/event
6. Session closes through /api/sentinel/session/complete
7. Auto-mode matcher links an existing lead or creates a new one

4. The Files That Matter

Frontend

app/src/components/modules/Sentinel.tsx Sentinel shell and tab structure.

app/src/components/modules/sentinel/SentinelLiveSession.tsx The broker-facing session wizard. This is the orchestration UI.

app/src/components/modules/sentinel/PerceptionPlayer.tsx Video playback, webcam capture, packet emission, and session closeout.

app/src/hooks/useMediapipeFaceLandmarker.ts MediaPipe bootstrap and frame detection.

app/src/hooks/useVelocitySocket.ts Notification and perception WebSocket connection logic.

app/src/components/layout/NotificationCenter.tsx Top-bar operational notification panel.

Backend

backend/main.py FastAPI entrypoint and router registration.

backend/routers/sentinel.py Perception WebSocket, notifications, consent, session completion, tag-lead route, QD score endpoint.

backend/routers/vault.py Trackable links and vault-open broadcast flow.

backend/routers/scenes.py Scene CSV upload and scene retrieval.

backend/routers/videos.py Marketing-video catalog loading and fallback filesystem discovery.

backend/routers/cctv.py CCTV event ingestion and auto-mode finalization.

backend/services/nemoclaw_client.py NVIDIA-primary prompt client and response parsing.

backend/services/auto_mode_matcher.py Auto-mode session-to-lead attribution logic.

backend/db/schema.sql Core Sentinel relational model.

backend/db/schema_addendum.sql Scene maps, perception sessions, CCTV events.

5. The Tables That Matter

users_and_roles RBAC and broker identity.

leads_intelligence Lead record, tags, QD score, broker assignment.

velocity_vault_assets Trackable asset share records.

omnichannel_logs Event history and sentiment spikes.

consent_log Biometric consent actions.

video_scene_maps Scene labels and timestamp ranges for each marketing video.

perception_sessions Assigned or auto-mode session records.

cctv_events OCR/vehicle/visitor-derived session evidence.

6. Ports and Paths

Public/backend-facing truth:

• 22 SSH
• 443 nginx TLS
• 127.0.0.1:8001 Uvicorn
• 127.0.0.1:5432 PostgreSQL
• 8080 OpenShell gateway
• 11434 Ollama

Paths:

• /opt/dlami/nvme/velocity/current
• /opt/dlami/nvme/velocity/env
• /opt/dlami/nvme/velocity/venv
• /opt/dlami/nvme/velocity/tls
• /opt/dlami/nvme/nemoclaw/prompts
• /opt/dlami/nvme/pgdata/14/velocity
• /opt/dlami/nvme/assets
• /opt/dlami/nvme/assets/videos
• /opt/dlami/nvme/assets/videos/catalog.json

7. What Is True About NemoClaw

The Sentinel still conceptually uses NemoClaw as its reasoning layer, but the active truth is:

• The backend uses NVIDIA-hosted completions as primary inference
• The active primary model is nvidia/nemotron-3-super-120b-a12b
• The backend prompt layer is implemented in repo code, not hidden inside a separate agent black box
• OpenShell and Ollama still exist on the node, but they are not the active primary scoring path

This distinction matters. If someone assumes the live QD score is coming from local Ollama, they are wrong.

8. Event Flows

QD update flow

Frontend packet -> /api/sentinel/ws/perception -> scene lookup in video_scene_maps -> QD scoring in nemoclaw_client.py -> perception_sessions and omnichannel_logs -> QD broadcast -> frontend QD ring and notifications

Marketing video flow

Frontend live-session picker -> GET /api/videos/marketing -> catalog read from /opt/dlami/nvme/assets/videos/catalog.json -> fallback recursive scan of /opt/dlami/nvme/assets/videos -> MP4 served through /assets/videos/... -> hover-preview card and full PerceptionPlayer playback

Vault flow

Broker generates velocity link -> prospect opens /vault/{tracking_hash} -> backend logs WS_ASSET_OPENED -> notification broadcast -> NotificationCenter shows the event

Auto-mode CCTV flow

OCR bridge posts to /api/cctv/event -> backend profiles plate/vehicle via cctv_profiler -> cctv_events row is written -> session evidence is updated -> session closes -> auto_mode_matcher.py links or creates lead

9. What Can Break

Structured output from the model

The NVIDIA path is reachable, but model output is not always perfect JSON under the full prompt. This is the main runtime fragility.

Wrong WebSocket path

The correct Sentinel socket namespace is /api/sentinel/ws/.... Any client using /ws/... directly will miss the router prefix and fail.

Root-volume drift

If deployment scripts start writing runtime state back to root instead of NVMe, disk-pressure regressions will return.

Video catalog drift

If new walkthrough files are copied to NVMe without catalog updates, the system still loads them, but the presentation order and labels can become unreliable.

Dynamic public IP assumptions

If external scripts hardcode an old public IP, they will fail. Current truth must be rechecked whenever AWS reassigns the address.

10. What “Done” Actually Means

The Sentinel is done at the engineering level when:

• all milestone code exists
• backend services start cleanly on NVMe-backed runtime
• PostgreSQL schema is live
• frontend build succeeds
• backend tests pass
• Playwright tests pass
• health endpoint is live over TLS

The Sentinel is done at the operational level only when:

• a stable DNS or Elastic IP is assigned
• trusted TLS is installed
• production CCTV/OCR source is wired in
• a live broker walkthrough has been accepted on the deployed system

11. Commands Worth Remembering

curl -k https://54.152.236.10/health
sudo systemctl status velocity-backend.service
sudo systemctl status postgresql@14-velocity.service
sudo systemctl status nginx
sudo systemctl status nemoclaw-velocity.service
sudo -u postgres psql -d velocity -c '\dt'

12. Final Rule

Do not maintain two truths.

If deployment reality changes, update:

• the SRS
• nemoclaw_setup_truth.md
• this Bibel

before anyone else builds on stale assumptions.