sagnik/Project_Velocity
2
1
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Oracle CRM Page, Synthetic Client Data and Live Snapshot when hitting emotion hotpoint

Browse Source
This commit is contained in:
Sagnik
2026-04-19 00:43:01 +05:30
parent f616a33ab0
commit 4b21c2cad6
197 changed files with 105054 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files

867
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Colony Database Schema and Root API Spec.md Normal file
View File

@@ -0,0 +1,867 @@
# Colony Database Schema and Root API Spec
**Date:** 2026-04-14
**Status:** Draft implementation artifact
**Purpose:** Define the PostgreSQL schema, root repository behavior, and FastAPI contract required to support the colony orchestration layer inside the current Project Velocity backend.
## 1. Purpose
This document specifies the database and root API layer for the colony system.
The colony runtime itself should not become the canonical persistence owner. The Project Velocity Python root must remain the system of record for mission persistence, policy decisions, and domain writeback approvals.
## 2. Design Principles
### 2.1 Root Owns Persistence
The TypeScript colony service may keep ephemeral runtime state in memory, but canonical mission and artifact records must be persisted through the Python root.
### 2.2 JSONB Plus Indexed Columns
Every artifact should be stored in canonical `jsonb` form, but high-value lookup fields must also be denormalized into indexed scalar columns.
### 2.3 Mission-Centric Linking
All records must be queryable by `mission_id` first. This is the primary unit of orchestration, debugging, audit, and approval.
### 2.4 No Hidden Writebacks
All proposed mutations to CRM or Oracle state must be stored as explicit proposals before approval or rejection.
## 3. Root Schema Files
Add:
- `backend/db/schema_colony.sql`
Optionally add:
- `backend/db/schema_colony_indexes.sql`
- `backend/db/schema_colony_seed.sql`
For Sprint 1, a single `schema_colony.sql` is sufficient if it remains readable.
## 4. Required Tables
Phase 1 requires the following root-owned tables:
- `colony_missions`
- `colony_tasks`
- `colony_prompt_packages`
- `colony_research_artifacts`
- `colony_librarian_passes`
- `colony_worker_results`
- `colony_aggregation_packets`
- `colony_review_packets`
- `colony_policy_decisions`
- `colony_writeback_proposals`
- `colony_event_log`
- `colony_pheromone_signals`
## 5. Detailed Table Plan
### 5.1 `colony_missions`
Purpose:
Stores the root mission envelope and lifecycle state.
Recommended columns:
```sql
CREATE TABLE IF NOT EXISTS colony_missions (
mission_id UUID PRIMARY KEY,
mission_type TEXT NOT NULL,
origin_surface TEXT NOT NULL,
tenant_id TEXT NOT NULL,
actor_id TEXT NOT NULL,
actor_role TEXT,
risk_level TEXT NOT NULL,
sensitivity_class TEXT NOT NULL,
status TEXT NOT NULL,
review_status TEXT,
time_budget_ms INTEGER,
token_budget INTEGER,
user_goal TEXT NOT NULL,
normalized_goal TEXT NOT NULL,
context_refs JSONB NOT NULL DEFAULT '{}'::jsonb,
requested_outputs JSONB NOT NULL DEFAULT '[]'::jsonb,
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
completed_at TIMESTAMPTZ
);
```
Indexes:
- `(tenant_id, created_at DESC)`
- `(mission_type, created_at DESC)`
- `(origin_surface, created_at DESC)`
- `(status, created_at DESC)`
### 5.2 `colony_tasks`
Purpose:
Stores the task graph produced by the planner and its runtime state.
Recommended columns:
```sql
CREATE TABLE IF NOT EXISTS colony_tasks (
task_id TEXT PRIMARY KEY,
mission_id UUID NOT NULL REFERENCES colony_missions(mission_id) ON DELETE CASCADE,
role_type TEXT NOT NULL,
status TEXT NOT NULL,
objective TEXT NOT NULL,
depends_on JSONB NOT NULL DEFAULT '[]'::jsonb,
required_capabilities JSONB NOT NULL DEFAULT '[]'::jsonb,
allowed_tools JSONB NOT NULL DEFAULT '[]'::jsonb,
required_data_scopes JSONB NOT NULL DEFAULT '[]'::jsonb,
success_criteria JSONB NOT NULL DEFAULT '[]'::jsonb,
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
completed_at TIMESTAMPTZ
);
```
Indexes:
- `(mission_id, created_at ASC)`
- `(mission_id, status, updated_at DESC)`
- `(mission_id, role_type, updated_at DESC)`
### 5.3 `colony_prompt_packages`
Purpose:
Stores prompt packages produced by the prompt master.
Recommended columns:
```sql
CREATE TABLE IF NOT EXISTS colony_prompt_packages (
package_id TEXT PRIMARY KEY,
mission_id UUID NOT NULL REFERENCES colony_missions(mission_id) ON DELETE CASCADE,
task_id TEXT NOT NULL REFERENCES colony_tasks(task_id) ON DELETE CASCADE,
role_type TEXT NOT NULL,
template_id TEXT,
template_version TEXT,
model_route TEXT,
tool_scope JSONB NOT NULL DEFAULT '[]'::jsonb,
constraints JSONB NOT NULL DEFAULT '{}'::jsonb,
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```
### 5.4 `colony_research_artifacts`
Purpose:
Stores external research artifacts and normalized citations.
Recommended columns:
```sql
CREATE TABLE IF NOT EXISTS colony_research_artifacts (
artifact_id TEXT PRIMARY KEY,
mission_id UUID NOT NULL REFERENCES colony_missions(mission_id) ON DELETE CASCADE,
task_id TEXT REFERENCES colony_tasks(task_id) ON DELETE CASCADE,
provider TEXT NOT NULL,
query TEXT NOT NULL,
result_count INTEGER NOT NULL DEFAULT 0,
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```
### 5.5 `colony_librarian_passes`
Purpose:
Stores scoped route cards and internal access passes issued to workers.
Recommended columns:
```sql
CREATE TABLE IF NOT EXISTS colony_librarian_passes (
pass_id TEXT PRIMARY KEY,
mission_id UUID NOT NULL REFERENCES colony_missions(mission_id) ON DELETE CASCADE,
task_id TEXT NOT NULL REFERENCES colony_tasks(task_id) ON DELETE CASCADE,
allowed_resource_families JSONB NOT NULL DEFAULT '[]'::jsonb,
expires_at TIMESTAMPTZ,
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```
### 5.6 `colony_worker_results`
Purpose:
Stores normalized worker outputs.
Recommended columns:
```sql
CREATE TABLE IF NOT EXISTS colony_worker_results (
result_id TEXT PRIMARY KEY,
mission_id UUID NOT NULL REFERENCES colony_missions(mission_id) ON DELETE CASCADE,
task_id TEXT NOT NULL REFERENCES colony_tasks(task_id) ON DELETE CASCADE,
confidence DOUBLE PRECISION,
output_text TEXT,
citations JSONB NOT NULL DEFAULT '[]'::jsonb,
tool_trace_refs JSONB NOT NULL DEFAULT '[]'::jsonb,
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```
### 5.7 `colony_aggregation_packets`
Purpose:
Stores synthesized pre-review outputs.
Recommended columns:
```sql
CREATE TABLE IF NOT EXISTS colony_aggregation_packets (
packet_id TEXT PRIMARY KEY,
mission_id UUID NOT NULL REFERENCES colony_missions(mission_id) ON DELETE CASCADE,
summary TEXT,
contradiction_count INTEGER NOT NULL DEFAULT 0,
coverage_gap_count INTEGER NOT NULL DEFAULT 0,
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```
### 5.8 `colony_review_packets`
Purpose:
Stores review results and final approved outputs.
Recommended columns:
```sql
CREATE TABLE IF NOT EXISTS colony_review_packets (
packet_id TEXT PRIMARY KEY,
mission_id UUID NOT NULL REFERENCES colony_missions(mission_id) ON DELETE CASCADE,
review_status TEXT NOT NULL,
issue_count INTEGER NOT NULL DEFAULT 0,
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```
### 5.9 `colony_policy_decisions`
Purpose:
Stores explicit allow or deny decisions.
Recommended columns:
```sql
CREATE TABLE IF NOT EXISTS colony_policy_decisions (
decision_id TEXT PRIMARY KEY,
mission_id UUID NOT NULL REFERENCES colony_missions(mission_id) ON DELETE CASCADE,
task_id TEXT,
decision_type TEXT NOT NULL,
subject TEXT NOT NULL,
decision TEXT NOT NULL,
reason TEXT,
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```
### 5.10 `colony_writeback_proposals`
Purpose:
Stores proposed domain mutations before approval.
Recommended columns:
```sql
CREATE TABLE IF NOT EXISTS colony_writeback_proposals (
proposal_id TEXT PRIMARY KEY,
mission_id UUID NOT NULL REFERENCES colony_missions(mission_id) ON DELETE CASCADE,
target_entity_type TEXT NOT NULL,
target_entity_id TEXT NOT NULL,
action_type TEXT NOT NULL,
approval_status TEXT NOT NULL DEFAULT 'pending',
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
resolved_at TIMESTAMPTZ
);
```
### 5.11 `colony_event_log`
Purpose:
Stores lifecycle events for audit and replay.
Recommended columns:
```sql
CREATE TABLE IF NOT EXISTS colony_event_log (
event_id BIGSERIAL PRIMARY KEY,
mission_id UUID NOT NULL REFERENCES colony_missions(mission_id) ON DELETE CASCADE,
task_id TEXT,
event_type TEXT NOT NULL,
event_source TEXT NOT NULL,
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```
### 5.12 `colony_pheromone_signals`
Purpose:
Stores compact reinforcement signals for future routing improvements.
Recommended columns:
```sql
CREATE TABLE IF NOT EXISTS colony_pheromone_signals (
signal_id BIGSERIAL PRIMARY KEY,
mission_type TEXT NOT NULL,
role_type TEXT NOT NULL,
signal_type TEXT NOT NULL,
subject TEXT NOT NULL,
signal_strength DOUBLE PRECISION NOT NULL,
evidence_count INTEGER NOT NULL DEFAULT 1,
payload JSONB NOT NULL DEFAULT '{}'::jsonb,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
```
## 6. Root Python Modules
Create:
- `backend/services/colony_repository.py`
- `backend/services/colony_gateway.py`
- `backend/services/colony_policy_bridge.py`
- `backend/api/routes_colony.py`
### 6.1 `colony_repository.py`
Responsibilities:
- ensure schema readiness
- insert and update mission rows
- insert artifact rows
- retrieve mission detail view
- retrieve mission artifact collection
- resolve writeback proposals
### 6.2 `colony_gateway.py`
Responsibilities:
- build HTTP client to colony service
- submit mission envelopes
- fetch mission state
- fetch artifact snapshots
- handle retries and timeout classification
### 6.3 `colony_policy_bridge.py`
Responsibilities:
- classify mission risk
- derive tool scope
- derive model routing tier
- validate writeback proposals
- produce `policy_decision` artifacts
### 6.4 `routes_colony.py`
Responsibilities:
- expose root-owned public endpoints
- authenticate callers
- authorize by origin surface and mission type
- persist mission creation
- call colony runtime
- expose approval and rejection endpoints
## 7. Root API Surface
### 7.1 `POST /api/colony/missions`
Purpose:
Create a new mission.
Request body:
```json
{
"mission_type": "oracle_advisory",
"origin_surface": "oracle",
"user_goal": "Summarize high-intent waterfront leads and recommend actions.",
"context_refs": {
"lead_ids": ["lead_123"],
"page_id": "page_abc",
"branch_id": "main"
},
"options": {
"include_external_research": true
}
}
```
Response:
```json
{
"status": "ok",
"data": {
"mission_id": "uuid",
"mission_type": "oracle_advisory",
"status": "accepted"
}
}
```
### 7.2 `GET /api/colony/missions/{mission_id}`
Purpose:
Fetch mission summary state.
Response should include:
- mission status
- review status
- current stage
- trace summary
- final output if completed
### 7.3 `GET /api/colony/missions/{mission_id}/artifacts`
Purpose:
Fetch mission artifact collection for operator inspection.
Response should include:
- task graph
- prompt packages
- research artifacts
- worker results
- aggregation packet
- review packet
- policy decisions
- writeback proposals
### 7.4 `POST /api/colony/missions/{mission_id}/approve`
Purpose:
Approve pending writeback proposal or gated finalization step.
### 7.5 `POST /api/colony/missions/{mission_id}/reject`
Purpose:
Reject pending writeback proposal or gated finalization step.
### 7.6 `GET /api/colony/health`
Purpose:
Expose root-to-colony connectivity and root persistence readiness.
Response should include:
- route health
- db readiness
- colony service reachability
- last connectivity check timestamp
## 8. Mission Lifecycle in Root
Mission lifecycle states should be:
- `accepted`
- `planning`
- `packaging`
- `retrieving`
- `executing`
- `aggregating`
- `reviewing`
- `awaiting_approval`
- `completed`
- `failed`
- `rejected`
These states should be stored in `colony_missions.status`.
## 9. Writeback Release Flow
The writeback release sequence should be:
1. reviewer produces `writeback_proposal`
2. proposal stored in `colony_writeback_proposals`
3. root policy bridge evaluates proposal
4. if mission requires approval, proposal waits in `pending`
5. approval endpoint resolves proposal
6. only then does root call existing CRM or Oracle write path
No colony worker should bypass this flow.
## 10. Root Repository Contract
The Python root needs one repository module that hides SQL and exposes a stable service contract to `routes_colony.py`, `routes_oracle.py`, and `routes_crm.py`.
Recommended file:
- `backend/services/colony_repository.py`
Required methods:
- `create_mission(mission_envelope: dict) -> dict`
- `update_mission_status(mission_id: str, status: str, review_status: str | None = None) -> dict`
- `store_task_graph(mission_id: str, tasks: list[dict]) -> int`
- `store_prompt_packages(mission_id: str, packages: list[dict]) -> int`
- `store_research_artifacts(mission_id: str, artifacts: list[dict]) -> int`
- `store_librarian_passes(mission_id: str, passes: list[dict]) -> int`
- `store_worker_results(mission_id: str, results: list[dict]) -> int`
- `store_aggregation_packet(mission_id: str, packet: dict) -> dict`
- `store_review_packet(mission_id: str, packet: dict) -> dict`
- `store_policy_decision(mission_id: str, decision: dict) -> dict`
- `store_writeback_proposal(mission_id: str, proposal: dict) -> dict`
- `append_event(mission_id: str, event_type: str, payload: dict) -> dict`
- `append_pheromone_signal(mission_id: str, signal: dict) -> dict`
- `get_mission_summary(mission_id: str) -> dict | None`
- `get_mission_artifacts(mission_id: str) -> dict`
- `list_missions(filters: dict) -> list[dict]`
- `resolve_writeback_proposal(mission_id: str, proposal_id: str, resolution: str, actor_id: str) -> dict`
Repository rules:
- repository methods must be idempotent where retries are expected
- artifact inserts should use unique IDs controlled by the caller, not generated ad hoc in SQL
- repository methods should raise typed exceptions for `not_found`, `conflict`, and `validation_error`
- root services should never handcraft SQL in route modules
## 11. Root API Contract
The root FastAPI layer should remain the only externally consumed colony API in Sprint 1.
Recommended files:
- `backend/api/routes_colony.py`
- `backend/services/colony_gateway.py`
### 11.1 `POST /api/colony/missions`
Purpose:
Create a mission, forward it to the TypeScript colony service, and persist the accepted mission envelope immediately.
Request body shape:
```json
{
"mission_type": "oracle_advisory",
"origin_surface": "oracle",
"tenant_id": "tenant-desineuron",
"actor_id": "user-123",
"actor_role": "advisor",
"user_goal": "Prepare a project-aware response for this lead.",
"context_refs": {
"lead_id": "lead-001",
"page_id": "oracle-main"
},
"requested_outputs": [
"final_answer",
"writeback_proposal"
],
"payload": {
"prompt": "What is the best next action for this lead?"
}
}
```
Response shape:
```json
{
"mission_id": "uuid",
"status": "accepted",
"review_status": "not_required",
"current_stage": "planning",
"accepted_at": "timestamp",
"links": {
"self": "/api/colony/missions/uuid",
"artifacts": "/api/colony/missions/uuid/artifacts"
}
}
```
### 11.2 `GET /api/colony/missions/{mission_id}`
Purpose:
Fetch mission summary state.
Response should include:
- mission status
- review status
- current stage
- trace summary
- final output if completed
- pending approval summary if applicable
### 11.3 `GET /api/colony/missions/{mission_id}/artifacts`
Purpose:
Fetch mission artifact collection for operator inspection.
Response should include:
- task graph
- prompt packages
- research artifacts
- worker results
- aggregation packet
- review packet
- policy decisions
- writeback proposals
This endpoint should support:
- `?include_payloads=true|false`
- `?artifact_type=worker_results`
- `?task_id=...`
### 11.4 `POST /api/colony/missions/{mission_id}/approve`
Purpose:
Approve pending writeback proposal or gated finalization step.
Request shape:
```json
{
"proposal_id": "proposal-001",
"resolution_note": "approved after operator review"
}
```
### 11.5 `POST /api/colony/missions/{mission_id}/reject`
Purpose:
Reject pending writeback proposal or gated finalization step.
Request shape:
```json
{
"proposal_id": "proposal-001",
"resolution_note": "insufficient evidence for automated writeback"
}
```
### 11.6 `GET /api/colony/health`
Purpose:
Expose root-to-colony connectivity and root persistence readiness.
Response should include:
- route health
- db readiness
- colony service reachability
- last connectivity check timestamp
- schema version loaded by root
## 12. Mission Lifecycle in Root
Mission lifecycle states should be:
- `accepted`
- `planning`
- `packaging`
- `retrieving`
- `executing`
- `aggregating`
- `reviewing`
- `awaiting_approval`
- `completed`
- `failed`
- `rejected`
These states should be stored in `colony_missions.status`.
Review states should be:
- `not_required`
- `pending`
- `approved`
- `rejected`
Writeback proposal states should be:
- `pending`
- `approved`
- `rejected`
- `expired`
- `superseded`
State transition rules:
- `accepted` may move only to `planning` or `failed`
- `planning` may move only to `packaging`, `failed`, or `rejected`
- `packaging` may move only to `retrieving`, `executing`, `failed`, or `rejected`
- `retrieving` may move only to `executing`, `failed`, or `rejected`
- `executing` may move only to `aggregating`, `failed`, or `rejected`
- `aggregating` may move only to `reviewing`, `failed`, or `rejected`
- `reviewing` may move only to `completed`, `awaiting_approval`, `failed`, or `rejected`
- `awaiting_approval` may move only to `completed`, `rejected`, or `failed`
- terminal states are `completed`, `failed`, and `rejected`
## 13. Writeback Release Flow
The writeback release sequence should be:
1. reviewer produces `writeback_proposal`
2. proposal stored in `colony_writeback_proposals`
3. root policy bridge evaluates proposal
4. if mission requires approval, proposal waits in `pending`
5. approval endpoint resolves proposal
6. only then does root call existing CRM or Oracle write path
No colony worker should bypass this flow.
Additional rules:
- every executed writeback must store `resolved_by`, `resolved_at`, and `execution_result`
- root must store the target route or service name that performed the approved mutation
- a rejected proposal must remain queryable for audit and future training
## 14. Root Query Shapes
### 14.1 Mission Summary Query
Need one summary query that joins:
- `colony_missions`
- latest `colony_review_packets`
- latest `colony_writeback_proposals`
This should return one operator-facing mission summary object.
### 14.2 Mission Artifact Query
Need one artifact query bundle that returns all mission-linked rows by table.
This should be grouped in a stable response shape so the frontend and operator tools do not need to know the physical table order.
### 14.3 Mission Event Replay Query
Need one query returning ordered `colony_event_log` rows by `created_at ASC`.
### 14.4 Mission Inbox Query
Need one query to list:
- missions currently awaiting approval
- missions failed in the last 24 hours
- missions completed but unresolved for operator follow-up
This powers internal operations and demo readiness.
## 15. Concurrency, Idempotency, and Retry Rules
Root API and repository behavior must be explicit here because the colony runtime will retry on network and model failures.
Required rules:
- `POST /api/colony/missions` should accept a caller-supplied `request_id` header for idempotent creation
- duplicate mission creation with the same `request_id` should return the existing mission record
- artifact writes should use deterministic IDs where practical
- approval endpoints must reject double approval of the same proposal
- mission status updates should use optimistic concurrency with `updated_at` or revision number checks
- failed root-to-colony dispatch attempts should still preserve the accepted mission record with a clear failure event
## 16. Security and Access Control
Root API enforcement must happen before requests reach the TypeScript service.
Required controls:
- mission creation requires authenticated actor context
- Oracle-origin missions can only reference Oracle-allowed data scopes
- CRM-origin missions can only reference CRM-allowed data scopes unless explicitly elevated
- artifact inspection endpoint requires privileged operator or internal service role
- approval and rejection endpoints require writeback authority, not just read authority
- root must redact sensitive fields before returning artifacts to standard UI consumers
## 17. Migration and Rollout Plan
Database rollout order:
1. apply `schema_colony.sql`
2. verify tables and indexes
3. deploy `colony_repository.py`
4. deploy `colony_gateway.py`
5. deploy `routes_colony.py`
6. mount routes in `backend/main.py`
7. enable mission create and health only
8. enable mission status
9. enable artifacts
10. enable approval routes
Rollout rule:
Do not expose approval endpoints in production before repository writes and artifact replay are proven stable.
## 18. Acceptance Criteria
This layer is complete when:
- schema file exists and applies cleanly
- root endpoints exist and validate payloads
- root persists mission and artifacts
- root can fetch mission and artifact state
- writeback proposals are stored and require root-mediated approval
- mission status transitions are enforced centrally
- duplicate mission create requests are handled safely
- no existing root routes regress
## 19. Ticket Breakdown
Minimum ticket set:
1. create `schema_colony.sql`
2. implement `colony_repository.py`
3. implement `colony_gateway.py`
4. implement `routes_colony.py`
5. mount colony router in `backend/main.py`
6. add idempotency and approval tests
7. add operator artifact inspection tests
## 20. Bottom Line
The database and root API layer are what turn the colony from a framework experiment into a product subsystem. Without this layer, the orchestration runtime remains difficult to govern and difficult to sell.

185
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Deployment, Operations, and Release Readiness Spec.md Normal file
View File

@@ -0,0 +1,185 @@
# Deployment, Operations, and Release Readiness Spec
**Date:** 2026-04-14
**Status:** Draft implementation artifact
**Purpose:** Define how the colony system is deployed, operated, observed, released, and judged ready for internal demos and early sales use.
## 1. Purpose
The colony cannot be sold if it only exists as a local developer runtime. It needs a disciplined deployment and operating model that fits current Project Velocity infrastructure.
## 2. Deployment Topology
Recommended topology for Sprint 1:
- FastAPI root remains the public application authority
- TypeScript colony service is an internal service behind the root
- PostgreSQL remains canonical persistence
- Nemoclaw and MCP services remain root-governed append layers
Traffic pattern:
1. UI calls FastAPI root
2. root authenticates and normalizes mission
3. root calls colony service over private service boundary
4. colony persists artifacts back through root APIs or root persistence bridge
5. root returns status and reviewed output to UI
## 3. Environment Contract
### 3.1 Root Backend
Required environment values:
- `COLONY_SERVICE_BASE_URL`
- `COLONY_SERVICE_API_KEY`
- `COLONY_ENABLED`
- `COLONY_TIMEOUT_MS`
- `COLONY_DEFAULT_TIME_BUDGET_MS`
- `COLONY_DEFAULT_TOKEN_BUDGET`
### 3.2 Colony Service
Required environment values:
- `PORT`
- `ROOT_API_BASE_URL`
- `ROOT_API_KEY`
- `DEFAULT_MODEL_ROUTE`
- `RESEARCH_PROVIDER`
- `BROWSER_PROVIDER`
- `MAX_CONCURRENT_MISSIONS`
- `MAX_WORKERS_PER_MISSION`
- `MISSION_TIMEOUT_MS`
- `TASK_TIMEOUT_MS`
## 4. Release Environments
Need three environments:
- local development
- shared staging
- production
Rules:
- staging must be used for Oracle and CRM mission replay before production enablement
- production must begin with assisted-mode missions only
- no production writeback automation before approval route testing is complete
## 5. Observability Requirements
Required operational outputs:
- structured logs
- mission traces
- stage latency metrics
- provider failure metrics
- approval queue metrics
- mission success and failure counters by mission type
Minimum dashboards:
- mission health dashboard
- policy block dashboard
- provider health dashboard
- approval backlog dashboard
## 6. Release Gates
The colony is not release-ready until all gates below pass.
### Gate 1: Technical Integrity
- schema applies cleanly
- root and colony health endpoints are green
- one Oracle mission completes in staging
- one CRM mission completes in staging
### Gate 2: Governance Integrity
- blocked tool case is denied correctly
- blocked writeback case is denied correctly
- approved writeback case requires explicit operator action
### Gate 3: Operational Integrity
- missions are replayable through artifact inspection
- failures preserve enough artifacts for debugging
- approval queue is visible to operators
### Gate 4: Sales Integrity
- output is stable enough for live demo
- reviewer packet can explain why the answer is trustworthy
- operator can inspect mission evidence quickly
## 7. Failure Runbooks
Need documented responses for:
- colony service unavailable
- root-to-colony authentication failure
- provider outage
- malformed contract payload
- approval queue backlog
- stuck mission in non-terminal state
Required runtime behavior:
- root returns structured degraded-state response
- mission remains auditable
- operator can mark mission failed or replay it
## 8. Rollout Strategy
Recommended rollout:
1. enable health and dry-run mission creation
2. enable Oracle assisted missions in staging
3. enable CRM assisted missions in staging
4. enable Oracle assisted missions in production
5. enable CRM assisted missions in production
6. enable Catalyst strategy missions in staging
Do not enable autonomous external research by default in production on day one.
## 9. Sales Readiness Criteria
The system is sellable for early Project Velocity demos only if:
- one Oracle mission reliably returns project-aware reviewed output
- one CRM mission reliably returns lead intelligence with evidence trail
- operator can show auditability in under two minutes
- approval workflow prevents accidental mutations
- mission failures are graceful and legible
It is not sellable if:
- output quality depends on manual developer intervention
- mission replay is impossible
- provider outages create silent failure
- writebacks can happen without approval
## 10. Ownership Model
Operational ownership should be split clearly:
- root owner: backend routes, auth, persistence, approval flows
- colony owner: runtime, workers, orchestration behavior
- policy owner: governance, model routing, tool permissions
- product owner: mission definitions, demo scenarios, release decision
## 11. Ticket Breakdown
1. define environment contracts
2. implement health checks
3. add mission trace and metrics
4. create staging rollout checklist
5. create production assisted-mode rollout checklist
6. add approval queue observability
7. document runbooks for outage and failed mission recovery
## 12. Bottom Line
The colony becomes commercially usable only when it is deployable, inspectable, and fail-safe. Release readiness is not a polish task. It is part of the core product contract.

497
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Execution Backlog_ Acceptance Ownership Sales Readiness.md Normal file
View File

@@ -0,0 +1,497 @@
# Execution Backlog_ Acceptance Ownership Sales Readiness
**Date:** 2026-04-14
**Status:** Draft execution backlog
**Purpose:** Translate the implementation blueprint into a sequenced delivery backlog with acceptance gates, ownership guidance, and sales-readiness criteria.
## 1. Purpose
This document defines the executable work program for the colony orchestration layer.
## 2. Delivery Principles
The delivery program should follow six principles.
### 2.1 Sellable Before Comprehensive
The first release must be narrow enough to be dependable. A small system that works under real operator pressure is more valuable than a broad colony that only works in demos.
### 2.2 Domain Missions Before General Intelligence
Do not begin with free-form everything agents. Start with typed missions:
- Oracle advisory
- CRM lead intelligence
- Catalyst strategy brief
### 2.3 Governance Before Autonomy
Any action that reads sensitive data, reaches the public web, or proposes writeback must be governed before being automated.
### 2.4 Artifacts Before UX Polish
The internal artifacts, schemas, and traces must be correct before product polish begins. Without them, debugging and improvement loops stay weak.
### 2.5 Root Compatibility Is Mandatory
Any colony feature that fights the current root architecture is wrong for Sprint 1.
### 2.6 Review Once, Review Well
Use one explicit review pass by default. Do not create endless recursive reviewer loops.
## 3. Workstream Backlog
The backlog below is ordered. Teams should not skip ahead unless a predecessor is stable enough for the next dependency.
### Workstream A: Colony Service Bootstrap
Objective:
Stand up `services/colony-orchestrator/` as a buildable TypeScript service.
Tasks:
- create package scaffold
- define environment config contract
- import or adapt Open Multi Agent execution concepts
- expose service bootstrap and health endpoint
Definition of done:
- service builds locally
- service starts locally
- service exposes `/health`
### Workstream B: Contract Layer
Objective:
Make mission and artifact exchange explicit and versioned.
Tasks:
- define mission envelope schema
- define task graph schema
- define prompt package schema
- define librarian pass schema
- define research artifact schema
- define worker result schema
- define aggregation packet schema
- define review packet schema
- define policy decision schema
Definition of done:
- every colony stage reads and writes only typed contracts
- invalid payloads fail fast
### Workstream C: Root Persistence Layer
Objective:
Add mission persistence to the current PostgreSQL root.
Tasks:
- create `backend/db/schema_colony.sql`
- create root repository helper module
- add mission storage methods
- add artifact storage methods
Definition of done:
- missions and artifacts can be inserted, updated, and queried from root Python
### Workstream D: Root Gateway and Routes
Objective:
Expose a root-owned API for colony mission execution.
Tasks:
- implement `backend/services/colony_gateway.py`
- implement `backend/api/routes_colony.py`
- mount colony router in `backend/main.py`
Definition of done:
- root can create and fetch missions through `/api/colony/*`
### Workstream E: Planner and Prompt Master
Objective:
Create the first meaningful colony reasoning layer.
Tasks:
- mission normalization
- task DAG planning
- prompt package generation
- prompt template registry
- prompt exemplar registry
Definition of done:
- one mission can be decomposed into a persisted task graph and prompt package set
### Workstream F: Librarian and Researcher
Objective:
Make context routing and evidence gathering real.
Tasks:
- internal source catalog
- route card generation
- cache index design
- search provider abstraction
- browser provider abstraction
- citation normalization
Definition of done:
- workers can receive internal passes and external evidence packets
### Workstream G: Workers, Aggregator, Reviewer
Objective:
Complete the minimum viable colony loop.
Tasks:
- worker factory
- worker execution runner
- result normalizer
- aggregator packet builder
- contradiction detector
- reviewer packet builder
Definition of done:
- a mission can execute from planning through reviewed final output
### Workstream H: Governance Bridge
Objective:
Connect colony execution to root policy and Nemoclaw-derived controls.
Tasks:
- risk classifier
- tool policy
- model routing policy
- writeback policy
- root policy bridge
Definition of done:
- at least one disallowed tool and one disallowed writeback are blocked and logged
### Workstream I: Oracle Adapter
Objective:
Make Oracle the first colony consumer.
Tasks:
- mission mapping for Oracle prompt submit
- Oracle context enrichment
- reviewed result return contract
- writeback proposal return contract
Definition of done:
- one Oracle advisory mission works end to end through the colony
### Workstream J: CRM Adapter
Objective:
Make CRM the second colony consumer.
Tasks:
- mission mapping for lead intelligence
- lead and chat context enrichment
- message suggestion output contract
- next-step recommendation contract
Definition of done:
- one CRM lead intelligence mission works end to end through the colony
### Workstream K: Catalyst Adapter
Objective:
Prepare the next monetizable mission surface.
Tasks:
- define strategy brief mission type
- connect to campaign and analytics state
- produce planning-grade artifact only
Definition of done:
- Catalyst mission can generate reviewed strategy brief in test mode
### Workstream L: Observability and Ops
Objective:
Make the system operable under real internal use.
Tasks:
- mission trace IDs
- structured logs
- duration metrics
- token usage metrics
- failure taxonomy
- replay and inspection support
Definition of done:
- every mission can be inspected after execution
## 4. Acceptance Gates
No stage should advance to broader rollout until these gates pass.
### Gate 1: Service Integrity
Pass conditions:
- colony service builds
- root backend still builds and starts
- no route collisions
- no regression in Oracle, CRM, Sentinel, or Catalyst route mounting
### Gate 2: Contract Integrity
Pass conditions:
- all mission and artifact schemas validate
- malformed payloads fail deterministically
- persisted records match schema versions
### Gate 3: Planner Integrity
Pass conditions:
- mission produces a task DAG
- tasks include dependencies and role assignments
- prompt packages are generated per task
### Gate 4: Context Integrity
Pass conditions:
- librarian passes are scoped
- research artifacts include provenance
- workers do not receive unrestricted context blobs
### Gate 5: Execution Integrity
Pass conditions:
- worker tasks execute
- aggregation packet is produced
- one-pass review packet is produced
- final output is returned with traceability
### Gate 6: Governance Integrity
Pass conditions:
- blocked tool requests are denied
- blocked writeback proposals are denied
- model routing class is recorded
- policy decisions are auditable
### Gate 7: Oracle Mission Readiness
Pass conditions:
- Oracle can submit mission
- mission returns reviewed response
- optional writeback proposal is structured
- Oracle v1 remains intact
### Gate 8: CRM Mission Readiness
Pass conditions:
- CRM can submit mission
- lead context is used
- result includes actionable insights
- no regression in existing CRM routes
### Gate 9: Internal Sales Readiness
Pass conditions:
- one internal operator can use the flow with low supervision
- outputs are stable enough for internal client-facing preparation
- failures are diagnosable without reading raw model output
## 5. Ownership Model
Ownership should be explicit even if one person temporarily covers multiple roles.
### Orchestration Kernel Owner
Owns:
- colony runtime
- task graph execution
- worker provisioning
- review loop mechanics
### Root Integration Owner
Owns:
- Python gateway
- FastAPI route integration
- PostgreSQL persistence layer
- adapter surfaces in root
### Prompt Systems Owner
Owns:
- prompt template registry
- prompt exemplar registry
- prompt package quality
- prompt evaluation traces
### Policy and Safety Owner
Owns:
- risk classification
- tool scope policy
- model routing policy
- writeback release policy
### Domain Adapter Owner
Owns:
- Oracle mission mapping
- CRM mission mapping
- Catalyst mission mapping
- Sentinel evidence attachment path
### QA and Observability Owner
Owns:
- scenario validation
- failure matrix
- trace verification
- release criteria evidence
## 6. Sales Readiness Criteria
This system is not sellable when it is merely architecturally elegant. It is sellable when it can be demonstrated reliably against buyer-relevant workflows.
### 6.1 Minimum Sellable Capability
The minimum sellable capability is:
- one Oracle mission that produces a reviewed advisory result
- one CRM mission that produces a reviewed lead intelligence result
- clear auditability and explainability
- stable failure behavior
### 6.2 Buyer-Relevant Proofs
To support sales, the team should be able to demonstrate:
- the system decomposes a goal transparently
- the system uses internal business data in a scoped way
- the system can enrich with external evidence
- the system reviews its own result before surfacing it
- the system does not perform uncontrolled autonomous actions
### 6.3 Demo Safety Criteria
No sales demo should depend on brittle hidden state.
A demo-ready system must have:
- known mission templates
- seeded demo data
- stable prompt packages
- deterministic fallback behavior for missing data
- visible mission state and trace output
### 6.4 Internal Readiness Checklist
Before using the colony in commercial conversations, confirm:
- mission classes are frozen for demo use
- one-click seed or fixture data exists
- failure cases produce intelligible explanations
- operator instructions fit on one short runbook page
- every demo claim can be tied back to a working code path
## 7. Risks and Kill Criteria
### Risk 1: Too Much System, Too Little Product
If the team spends weeks building colony internals without Oracle and CRM mission proofs, the work is drifting.
### Risk 2: Governance Is Deferred
If tool scope and writeback gating are postponed, the colony will accumulate unsafe assumptions and later refactors will be painful.
### Risk 3: Prompt Master Is Hand-Waved
If prompt packaging remains a generic coordinator side effect, one of the system's biggest intended advantages is lost.
### Risk 4: Library Routing Is Faked
If all workers still get all context, the system is not implementing the actual biomimetic model and cost will grow quickly.
### Kill Criteria
Pause or redesign if any of the following happen:
- colony introduces route or data ownership conflicts with root
- missions cannot be replayed or diagnosed
- Oracle and CRM adapters remain unproven after kernel work
- policy model is too weak to gate sensitive actions
## 8. Immediate Next Actions
The immediate next actions should be executed in this order:
1. Freeze the current design docs in the biomimetic folder as the architecture source set.
2. Create `services/colony-orchestrator/` skeleton.
3. Create `backend/db/schema_colony.sql`.
4. Create `backend/services/colony_gateway.py`.
5. Create `backend/api/routes_colony.py`.
6. Implement mission envelope and artifact schemas.
7. Implement planner and prompt package generation.
8. Wire Oracle advisory mission.
9. Wire CRM lead intelligence mission.
10. Add governance and acceptance tests.
## 9. Bottom Line
The delivery program should be judged by one question:
Can this system become a stable, demonstrable revenue-supporting capability for Project Velocity quickly enough to matter?
If the team follows this backlog in order, the answer can become yes without turning the architecture into another speculative side project.

658
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Implementation Blueprint_ Repository and Runtime Mapping.md Normal file
View File

@@ -0,0 +1,658 @@
# Implementation Blueprint_ Repository and Runtime Mapping
**Date:** 2026-04-14
**Status:** Draft implementation blueprint
**Purpose:** Convert the biomimetic orchestration design into a repository-level build plan tied directly to the current Project Velocity codebase.
## 1. Purpose
This document defines the exact implementation skeleton for the colony orchestration layer inside Project Velocity.
## 2. Current State Mapping
Project Velocity already has enough root infrastructure to support a colony orchestration layer without introducing a second backend center.
### 2.1 Root FastAPI and Router Truth
The current root backend in `backend/main.py` already mounts the relevant operational surfaces:
- `backend/api/routes_catalyst.py`
- `backend/api/routes_crm.py`
- `backend/api/routes_oracle.py`
- `backend/oracle/router_v1.py`
- `backend/routers/sentinel.py`
- `backend/routers/cctv.py`
- `backend/routers/scenes.py`
- `backend/routers/videos.py`
- `backend/routers/vault.py`
This means the colony layer must integrate into the current root, not compete with it.
### 2.2 Oracle Truth
Oracle already contains a meaningful planning spine:
- `backend/oracle/prompt_orchestrator.py`
- `backend/oracle/persona_service.py`
- `backend/oracle/policy_service.py`
- `backend/oracle/data_access_gateway.py`
- `backend/oracle/action_service.py`
- `backend/oracle/router_v1.py`
This is critical. The colony layer must not replace Oracle v1. It must become the next orchestration layer beneath or beside the current prompt orchestration concepts.
### 2.3 CRM Truth
CRM is now real enough to serve as an internal mission substrate:
- `backend/api/routes_crm.py` exposes leads, chat logs, kanban, demographics, seed flow, and analytics scatter
- `/ws/crm` already exists in `backend/main.py`
- frontend CRM bootstrap already exists under `app/src/hooks/useCrmBootstrap.ts`
The colony therefore has a real internal state store to reason over. It should use CRM as one of its first mission domains.
### 2.4 Nemoclaw and MCP Truth
Project Velocity already includes root-owned Python append layers:
- `backend/services/nemoclaw_runtime.py`
- `backend/services/mcp_registry.py`
- `backend/services/nemoclaw_client.py`
These are not yet a full orchestration platform, but they provide three essential things:
- workflow dispatch preview semantics
- external and internal tool abstraction
- model-facing reasoning utilities
### 2.5 Frontend Truth
The frontend already exposes product surfaces that will eventually call the colony:
- Oracle page shell
- CRM and pipeline surfaces
- Catalyst and marketing tab
- Sentinel live session and perception surfaces
The correct implementation path is therefore back-to-front:
- build colony runtime and contracts first
- expose root API entry points second
- consume them from Oracle and CRM first
## 3. Target Runtime Shape
The target runtime must use a split-brain architecture with one root authority and one specialized orchestration service.
### 3.1 Root Authority
The current Python root remains authoritative for:
- authentication
- PostgreSQL access
- domain data models
- Oracle page and canvas persistence
- CRM persistence
- Sentinel ingestion and response
- Catalyst persistence and route ownership
### 3.2 Colony Runtime
The new TypeScript colony service becomes authoritative for:
- mission normalization
- task graph planning
- prompt package generation
- worker provisioning
- inter-agent execution
- aggregation and review
- orchestration traces
### 3.3 Guard Plane
Nemoclaw-derived governance remains external to worker prompts and should mediate:
- tool scope
- network egress class
- model routing class
- writeback validation
- audit events
### 3.4 Why This Split Is Correct
This split is the least disruptive implementation path because:
- Project Velocity already has a mature Python root
- Open Multi Agent is TypeScript-native and should remain where it is strongest
- domain systems should not be reimplemented in the colony runtime
- orchestration should not directly own CRM, Oracle, or Sentinel tables
## 4. File-by-File Repository Blueprint
The following structure should be created.
### 4.1 New Top-Level Service
Create:
```text
services/
colony-orchestrator/
```
This service is the fork-and-extend boundary for Open Multi Agent concepts.
### 4.2 TypeScript Service Tree
Create:
```text
services/colony-orchestrator/
package.json
tsconfig.json
README.md
src/
index.ts
config/
env.ts
models.ts
core/
colony-runtime.ts
mission-runner.ts
task-graph.ts
scheduler.ts
worker-factory.ts
contracts/
mission.ts
task.ts
prompt-package.ts
research-artifact.ts
librarian-pass.ts
worker-result.ts
aggregation-packet.ts
review-packet.ts
policy-decision.ts
planner/
mission-normalizer.ts
planner-agent.ts
decomposition-policy.ts
prompt-master/
prompt-master-agent.ts
template-registry.ts
exemplar-registry.ts
prompt-package-builder.ts
librarian/
librarian-agent.ts
source-catalog.ts
route-card-builder.ts
cache-index.ts
researcher/
researcher-agent.ts
search-provider.ts
browser-provider.ts
citation-normalizer.ts
workers/
worker-agent.ts
worker-context-builder.ts
result-normalizer.ts
aggregation/
aggregator-agent.ts
evidence-matrix.ts
contradiction-detector.ts
review/
reviewer-agent.ts
review-policy.ts
release-gate.ts
policy/
tool-policy.ts
model-routing-policy.ts
writeback-policy.ts
risk-classifier.ts
adapters/
oracle-adapter.ts
crm-adapter.ts
catalyst-adapter.ts
sentinel-adapter.ts
python-root-client.ts
persistence/
mission-store.ts
artifact-store.ts
pheromone-store.ts
telemetry/
audit-log.ts
mission-trace.ts
metrics.ts
tests/
contracts/
planner/
prompt-master/
librarian/
researcher/
integration/
```
### 4.3 New Python Root Files
Create:
```text
backend/
api/
routes_colony.py
services/
colony_gateway.py
colony_repository.py
colony_policy_bridge.py
db/
schema_colony.sql
```
### 4.4 Why These Files Exist
`routes_colony.py` is the root public API surface for colony mission submission and inspection.
`colony_gateway.py` is the Python integration client that talks to the TypeScript colony service.
`colony_repository.py` is the root-owned persistence adapter for colony mission records that must live near current PostgreSQL patterns.
`colony_policy_bridge.py` is the place where root policy, Nemoclaw-derived guard logic, and domain-specific action constraints are translated into colony-consumable decisions.
`schema_colony.sql` is the root schema source for mission and artifact persistence.
## 5. Python Root Integration Plan
The Python root must integrate the colony in a way that preserves current ownership.
### 5.1 New Root Routes
Mount a new router in `backend/main.py`:
```python
from backend.api.routes_colony import router as colony_router
app.include_router(colony_router, prefix="/api/colony", tags=["Colony"])
```
### 5.2 Required Root Endpoints
Phase 1 root endpoints should be:
- `POST /api/colony/missions`
- `GET /api/colony/missions/{mission_id}`
- `GET /api/colony/missions/{mission_id}/artifacts`
- `POST /api/colony/missions/{mission_id}/approve`
- `POST /api/colony/missions/{mission_id}/reject`
- `GET /api/colony/health`
### 5.3 Root Responsibilities
The Python root should:
- authenticate and authorize caller context
- attach tenant and actor context
- classify requested mission type
- persist mission metadata
- call colony runtime
- validate requested writebacks
- surface mission state to existing product surfaces
### 5.4 Explicit Non-Responsibilities
The Python root should not:
- perform colony DAG decomposition itself
- own worker prompt package generation
- run direct multi-agent loops inside FastAPI request handlers
Those belong in the colony service.
## 6. TypeScript Colony Service Plan
The colony service should be implemented as a narrowly scoped internal service, not as a second public product shell.
### 6.1 Source of Truth for Execution
The service should own:
- mission runtime state
- task graph state
- prompt package generation
- agent execution
- review packets
- orchestration traces
### 6.2 Upstream Fork Strategy
Do not copy the entire Open Multi Agent repository directly into the root app. Instead:
1. create `services/colony-orchestrator/`
2. import or adapt upstream concepts into `src/core/`
3. keep Velocity-specific roles outside the upstream-like core folders
The forked pieces should roughly correspond to:
- `OpenMultiAgent` -> `colony-runtime.ts`
- `TaskQueue` -> `task-graph.ts`
- `Scheduler` -> `scheduler.ts`
- `AgentPool` -> `worker-factory.ts` plus internal pool manager
- `team/messaging` and `memory/shared` -> mission-local context bus and artifact retrieval
### 6.3 Colony Role Mapping
Map the biomimetic metaphor to concrete runtime classes:
- queen -> root product surface plus mission ID ownership
- planner ant -> `planner/planner-agent.ts`
- prompt master ant -> `prompt-master/prompt-master-agent.ts`
- researcher ant -> `researcher/researcher-agent.ts`
- librarian ant -> `librarian/librarian-agent.ts`
- worker ants -> `workers/worker-agent.ts`
- aggregator ant -> `aggregation/aggregator-agent.ts`
- reviewer ant -> `review/reviewer-agent.ts`
### 6.4 Execution Model
Per mission:
1. `mission-normalizer.ts` creates mission envelope
2. `planner-agent.ts` creates task graph
3. `prompt-package-builder.ts` creates prompt package per task
4. `librarian-agent.ts` creates route cards and scoped passes
5. `researcher-agent.ts` creates evidence artifacts where required
6. `worker-agent.ts` executes task-specific work
7. `aggregator-agent.ts` synthesizes outputs
8. `reviewer-agent.ts` performs one-pass review
9. final packet is returned to root
### 6.5 Mission Classes for Phase 1
Implement only three mission classes first:
- `oracle_advisory`
- `crm_lead_intelligence`
- `catalyst_strategy_brief`
Do not start with generalized free-form autonomous missions. Use typed mission classes only.
## 7. Mission, Artifact, and Persistence Plan
The current Open Multi Agent model is intentionally short-lived and mostly in-memory. Project Velocity needs persistence from day one.
### 7.1 Root Database Objects
Create the following tables in `backend/db/schema_colony.sql`:
- `colony_missions`
- `colony_tasks`
- `colony_prompt_packages`
- `colony_research_artifacts`
- `colony_librarian_passes`
- `colony_worker_results`
- `colony_aggregation_packets`
- `colony_review_packets`
- `colony_policy_decisions`
- `colony_pheromone_signals`
### 7.2 Mandatory Columns
Every table should include:
- primary ID
- mission ID foreign key where applicable
- `tenant_id`
- `actor_id`
- `status`
- `payload jsonb`
- `created_at`
- `updated_at`
### 7.3 Mission Envelope Contract
Mission payload should minimally contain:
- `mission_id`
- `mission_type`
- `origin_surface`
- `user_goal`
- `normalized_goal`
- `tenant_id`
- `actor_id`
- `risk_level`
- `sensitivity_class`
- `time_budget_ms`
- `token_budget`
- `requested_outputs`
### 7.4 Why Persistence Is Mandatory
Persistence is not optional because the product needs:
- sales-demo reliability
- auditability
- mission replay and debugging
- operator review visibility
- future approvals and resumability
## 8. Adapter Plan for Oracle, CRM, Catalyst, and Sentinel
### 8.1 Oracle Adapter
The Oracle adapter should be the first-class Phase 1 adapter.
It should:
- accept a prompt from Oracle UI or Oracle helper routes
- translate it into `oracle_advisory` mission type
- enrich with tenant, actor, page, branch, and target lead context
- return structured renderable packet plus optional writeback proposal
Do not bypass `backend/oracle/router_v1.py`. Instead add a colony-backed path under Oracle helper or Oracle v1 integration.
### 8.2 CRM Adapter
The CRM adapter should:
- accept a lead or kanban stage context
- create `crm_lead_intelligence` mission type
- pull relevant lead, chat log, and stage data through root APIs or repository methods
- return structured insights, next-step suggestions, and optional message drafts
### 8.3 Catalyst Adapter
Catalyst should be Phase 1.5 or Phase 2, but the file boundary should exist now.
It should eventually create `catalyst_strategy_brief` missions using:
- campaign state
- analytics
- audience context
- ad network context
### 8.4 Sentinel Adapter
Sentinel should initially be an evidence provider, not an autonomous mission origin.
That means Sentinel data can be attached to missions as:
- perception evidence
- emotional interest signals
- room-level events
but Sprint 1 should not make Sentinel the lead actor in colony execution.
## 9. Governance and Nemoclaw Guard Plan
Governance is where this system either becomes production-ready or stays a demo.
### 9.1 Policy Layers
There should be four policy layers:
1. mission admission policy
2. tool and data scope policy
3. model routing policy
4. writeback release policy
### 9.2 Mission Admission Policy
Before the colony starts, classify:
- mission type
- sensitivity
- autonomy tier
- required review level
### 9.3 Tool and Data Scope Policy
Every worker must receive an explicit allowed tool list and allowed data scope list. No worker should receive generic open access to all MCP tools or all root data.
### 9.4 Model Routing Policy
At minimum define:
- `tier_local_private`
- `tier_cloud_standard`
- `tier_high_reasoning`
This routing decision must be computed outside the worker prompt and carried into the prompt package.
### 9.5 Writeback Release Policy
No worker should directly mutate CRM or Oracle state.
All writebacks should go through:
- `review-packet`
- root policy bridge
- root writeback validator
### 9.6 Current Code to Reuse
Directly leverage:
- `backend/oracle/action_service.py`
- `backend/oracle/policy_service.py`
- `backend/services/nemoclaw_runtime.py`
- `backend/services/mcp_registry.py`
These should not be discarded. They should be bridged into the colony runtime.
## 10. Testing and Operationalization Plan
Testing must be mission-centered, not only framework-centered.
### 10.1 Required Test Layers
Create tests for:
- schema validation
- planner outputs
- prompt package generation
- librarian routing passes
- research artifact normalization
- worker result normalization
- aggregation packets
- review decisions
- policy rejection
- root adapter integration
### 10.2 Root Integration Tests
Add Python integration tests for:
- `POST /api/colony/missions`
- Oracle adapter mission submission
- CRM adapter mission submission
- writeback approval and rejection flow
### 10.3 Operational Readiness
The service is only considered production-candidate when it has:
- health endpoint
- structured logs
- trace correlation IDs
- mission replay view
- failure reason visibility
- environment-driven provider config
## 11. Build Order
The implementation order should be fixed.
### Step 1
Create `services/colony-orchestrator/` with:
- package setup
- base runtime shell
- contract files
### Step 2
Create `backend/db/schema_colony.sql` and repository helpers.
### Step 3
Create `backend/services/colony_gateway.py` and `backend/api/routes_colony.py`.
### Step 4
Implement:
- mission normalizer
- planner
- prompt package builder
### Step 5
Implement:
- librarian
- researcher provider interfaces
### Step 6
Implement:
- worker execution
- aggregator
- reviewer
### Step 7
Implement:
- Oracle adapter
- CRM adapter
### Step 8
Implement governance bridge and writeback approval flow.
### Step 9
Run end-to-end acceptance on Oracle and CRM mission classes.
## 12. Non-Negotiable Decisions
These decisions should be frozen unless strong evidence proves them wrong:
- no second Python backend center
- no replacement of Oracle v1
- no uncontrolled context sharing
- no prompt-only governance
- no direct worker writebacks to root state
- no generic mission types before typed domain missions succeed
## 13. Bottom Line
The colony layer should be added as a new orchestration service plus a thin root integration layer.
The build should start where Project Velocity is already strong:
- Oracle
- CRM
- root PostgreSQL
- existing policy and Nemoclaw helper surfaces
That is the shortest path from architecture to a sellable working product.

380
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Implementation Ticket Breakdown and Dependency Matrix.md Normal file
View File

@@ -0,0 +1,380 @@
# Implementation Ticket Breakdown and Dependency Matrix
**Date:** 2026-04-14
**Status:** Draft implementation artifact
**Purpose:** Convert the biomimetic architecture into an execution-ready ticket graph with dependencies, gating logic, and parallelization boundaries.
## 1. Purpose
This document is the handoff layer between architecture and implementation management. It defines what tickets exist, what depends on what, and what can be built in parallel without creating merge chaos.
## 2. Dependency Rules
The system should be built in layers.
Layer 1:
- contracts
- root schema
- root repository
Layer 2:
- root gateway and routes
- colony service bootstrap
- root client and persistence facades
Layer 3:
- planner
- prompt master
- librarian
Layer 4:
- researcher
- workers
- scheduler
Layer 5:
- aggregation
- reviewer
- policy bridge
Layer 6:
- Oracle adapter
- CRM adapter
- Catalyst adapter
Layer 7:
- observability
- release operations
- operator tooling
## 3. Ticket Matrix
### T1. Contract Layer
Scope:
- create or finalize TypeScript contracts and Zod schemas
- align contracts with mission schema doc
Depends on:
- none
Blocks:
- all runtime work
### T2. Root Database Schema
Scope:
- create `backend/db/schema_colony.sql`
- add indexes
Depends on:
- T1
Blocks:
- repository and root APIs
### T3. Root Repository
Scope:
- implement `colony_repository.py`
Depends on:
- T2
Blocks:
- gateway, routes, approval flows
### T4. Colony Service Bootstrap
Scope:
- scaffold `services/colony-orchestrator`
- add `app.ts`, `index.ts`, health route
Depends on:
- T1
Blocks:
- service-side feature tickets
### T5. Python Root Client and Persistence Facades
Scope:
- implement `python-root-client.ts`
- implement `mission-store.ts`
- implement `artifact-store.ts`
Depends on:
- T3
- T4
Blocks:
- planner-through-review pipeline
### T6. Root Gateway and Routes
Scope:
- implement `colony_gateway.py`
- implement `routes_colony.py`
- mount in `backend/main.py`
Depends on:
- T3
- T4
Blocks:
- external mission initiation
### T7. Planner
Scope:
- mission normalization
- task DAG generation
Depends on:
- T1
- T5
Blocks:
- prompt master
- worker execution
### T8. Prompt Master
Scope:
- template registry
- exemplar registry
- prompt package builder
Depends on:
- T7
Blocks:
- worker execution
### T9. Librarian
Scope:
- source catalog
- route-card builder
- cache index
Depends on:
- T7
Blocks:
- worker execution
### T10. Researcher
Scope:
- search provider abstraction
- browser provider abstraction
- citation normalization
Depends on:
- T1
- T5
Blocks:
- research-enabled missions
### T11. Workers and Scheduler
Scope:
- worker factory
- scheduler
- worker runner
Depends on:
- T8
- T9
Blocks:
- aggregation
### T12. Aggregation and Review
Scope:
- aggregator
- contradiction detector
- reviewer
- release gate
Depends on:
- T11
Blocks:
- reviewed output delivery
### T13. Policy and Governance Bridge
Scope:
- risk classifier
- tool policy
- model routing policy
- writeback policy
Depends on:
- T6
- T12
Blocks:
- production-grade release
### T14. Oracle Adapter
Scope:
- map Oracle prompt flow to colony missions
Depends on:
- T12
- T13
### T15. CRM Adapter
Scope:
- map CRM lead intelligence flow to colony missions
Depends on:
- T12
- T13
### T16. Catalyst Adapter
Scope:
- map Catalyst strategy missions
Depends on:
- T12
- T13
### T17. Observability and Operations
Scope:
- traces
- metrics
- runbooks
- release dashboards
Depends on:
- T6
- T12
### T18. Sales Demo Readiness
Scope:
- curated demo missions
- operator script
- approval queue visibility
- artifact inspection path
Depends on:
- T14
- T15
- T17
## 4. Parallelization Guidance
Safe parallel tracks:
- T2 and T4 after T1
- T7 and T10 after prerequisite facades exist
- T14 and T15 after core and governance are stable
Unsafe parallel tracks:
- implementing adapters before contracts stabilize
- building reviewer before worker result contract exists
- building production demo flows before observability exists
## 5. Acceptance Gates by Ticket Group
Gate A:
- T1 through T6 complete
- root and service are connected
Gate B:
- T7 through T12 complete
- one reviewed mission can execute end to end
Gate C:
- T13 through T15 complete
- one Oracle and one CRM mission are governed
Gate D:
- T17 and T18 complete
- system is demo-safe and sales-usable
## 6. Suggested Ownership Split
- engineer 1: root schema, repository, gateway, routes
- engineer 2: service bootstrap, contracts, persistence facades
- engineer 3: planner, prompt master, librarian
- engineer 4: workers, aggregation, review
- engineer 5: governance, adapters, observability
If the team is smaller, keep ownership by layer, not by file count.
## 7. Bottom Line
This dependency matrix should be used to create actual tickets. It keeps the team from building the colony out of order and makes it clear what can proceed in parallel without damaging the architecture.

255
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Introduction for Sourik.md Normal file
View File

@@ -0,0 +1,255 @@
# Introduction for Sourik_ Biomimetic Agentic Orchestration Layer
**Date:** 2026-04-14
**Status:** Team onboarding artifact
**Primary Reader:** Sourik
**Purpose:** Provide a single entry point into the biomimetic orchestration work so you can understand the intent, architecture, current implementation direction, and the exact document set without reconstructing the system from scattered files.
## 1. What This Project Is
This folder defines the next orchestration layer for Project Velocity.
The goal is not to build another chatbot and not to replace the current Python root. The goal is to build a governed colony-style agentic orchestration system that can sit on top of the current Project Velocity runtime and make Oracle, CRM, and later Catalyst operate through structured multi-agent missions instead of loose prompt chains.
The core design idea is biomimetic:
- the user-facing intelligence behaves like the queen interface
- planning is handled by a decomposition layer
- prompt specialization is handled by a prompt master layer
- internal knowledge routing is handled by a librarian layer
- external research is handled by a researcher layer
- execution is handled by specialized workers
- synthesis is handled by an aggregator
- quality control is handled by a reviewer
- governance remains outside worker prompts through a Nemoclaw-style guard plane
In practical engineering terms, this means:
- FastAPI root remains the authority for auth, persistence, domain ownership, and approvals
- a TypeScript colony service becomes the orchestration kernel
- Nemoclaw-derived policy remains the guard and writeback control plane
- Oracle and CRM are the first real mission consumers
## 2. Why This Matters for Project Velocity
Project Velocity already has many of the right pieces:
- a real FastAPI backend
- Oracle surfaces
- CRM routes and state
- Catalyst backend and UI
- Sentinel sensing and analysis paths
- MCP and Nemoclaw append layers
What it does not yet have is one disciplined orchestration spine that can:
- decompose work safely
- distribute work by role
- scope context instead of flooding every worker
- preserve artifacts and evidence
- review results before delivery
- propose writebacks without bypassing root governance
This folder is the design and implementation program for that spine.
## 3. The Architectural Truth You Should Assume
If you work on this system, assume the following are already decided:
1. Python FastAPI root is the canonical product center.
2. The colony service must integrate into the current root, not compete with it.
3. Open Multi Agent concepts are being forked and reshaped, not adopted blindly.
4. Nemoclaw concepts are being absorbed as governance and policy patterns, not as a second runtime center.
5. Internal artifacts must be structured and persisted.
6. Direct worker writeback is forbidden.
7. Oracle advisory missions and CRM intelligence missions are the first practical targets because they are closest to real product value and sales readiness.
## 4. How to Read This Folder
Do not read this as a flat pile of documents. It has a deliberate reading order.
### 4.1 Start Here: Concept and Intent
Read these first:
- [Project Velocity - Biomimetic Agentic Orchestration Layer (First Principles).md](./Project%20Velocity%20-%20Biomimetic%20Agentic%20Orchestration%20Layer%20(First%20Principles).md)
- [Product Requirements Document (PRD)_ Biomimetic Agentic Orchestration Layer.md](./Product%20Requirements%20Document%20(PRD)_%20Biomimetic%20Agentic%20Orchestration%20Layer.md)
- [Software Requirements Specification (SRS)_ Biomimetic Agentic Orchestration Layer.md](./Software%20Requirements%20Specification%20(SRS)_%20Biomimetic%20Agentic%20Orchestration%20Layer.md)
These three define:
- why the system exists
- what problem it solves
- what the product and system boundaries are
- what must be true for it to be technically legitimate
### 4.2 Then Read the Delivery and Build Plan
Read next:
- [Sprint 1 Plan_ Biomimetic Agentic Orchestration Layer.md](./Sprint%201%20Plan_%20Biomimetic%20Agentic%20Orchestration%20Layer.md)
- [Implementation Blueprint_ Repository and Runtime Mapping.md](./Implementation%20Blueprint_%20Repository%20and%20Runtime%20Mapping.md)
- [Execution Backlog_ Acceptance Ownership Sales Readiness.md](./Execution%20Backlog_%20Acceptance%20Ownership%20Sales%20Readiness.md)
These define:
- the build order
- the runtime split
- the exact repository direction
- the acceptance gates
- the path to something that is demo-safe and sellable
### 4.3 Then Read the Contract and Runtime Specs
Read next:
- [Mission Contracts and JSON Schemas Blueprint.md](./Mission%20Contracts%20and%20JSON%20Schemas%20Blueprint.md)
- [Colony Database Schema and Root API Spec.md](./Colony%20Database%20Schema%20and%20Root%20API%20Spec.md)
- [TypeScript Colony Service Internal Module Spec.md](./TypeScript%20Colony%20Service%20Internal%20Module%20Spec.md)
These define:
- what artifacts exist
- how they are shaped
- where they persist
- what the root API looks like
- how the TypeScript service should be structured internally
### 4.4 Then Read the Domain and Role Specs
Read next:
- [Oracle and CRM Adapter Detailed Implementation Spec.md](./Oracle%20and%20CRM%20Adapter%20Detailed%20Implementation%20Spec.md)
- [Prompt Master, Librarian, and Researcher Role Spec.md](./Prompt%20Master,%20Librarian,%20and%20Researcher%20Role%20Spec.md)
These define:
- the first domain adapters that matter
- the highest-leverage internal role boundaries
- how execution quality is controlled before workers even start
### 4.5 Finally Read the Operating and Ticketing Layer
Read last:
- [Deployment, Operations, and Release Readiness Spec.md](./Deployment,%20Operations,%20and%20Release%20Readiness%20Spec.md)
- [Implementation Ticket Breakdown and Dependency Matrix.md](./Implementation%20Ticket%20Breakdown%20and%20Dependency%20Matrix.md)
These define:
- how this becomes deployable
- how it becomes safe to demo
- how it becomes sliceable into tickets without architectural drift
## 5. The Most Important Project Decisions in This Folder
If you only retain a few truths from the full set, retain these:
### 5.1 Root Owns Truth
The colony service does not become the source of truth for domain state. Root FastAPI owns persistence, approvals, and domain write execution.
### 5.2 Colony Owns Orchestration
The TypeScript colony service owns:
- mission normalization
- task graph planning
- prompt packaging
- worker execution
- aggregation
- review
### 5.3 Nemoclaw Owns Guard Semantics
Nemoclaw-style concepts are being used for:
- policy
- tool scope
- model routing
- writeback control
- auditability
### 5.4 Oracle and CRM Are the First Serious Use Cases
This is not an abstract framework exercise. The first meaningful mission classes are:
- Oracle advisory missions
- CRM lead intelligence missions
Catalyst follows once those work reliably.
### 5.5 Artifacts Matter More Than Clever Prompts
The colony is being designed around explicit artifacts:
- mission envelope
- task graph
- prompt package
- librarian pass
- research artifact
- worker result
- aggregation packet
- review packet
- policy decision
- writeback proposal
This is one of the main differences between a demo swarm and a productizable orchestration layer.
## 6. What You Should Pay Attention To First
Given your likely integration role, the highest-value docs for you are:
- [Implementation Blueprint_ Repository and Runtime Mapping.md](./Implementation%20Blueprint_%20Repository%20and%20Runtime%20Mapping.md)
- [TypeScript Colony Service Internal Module Spec.md](./TypeScript%20Colony%20Service%20Internal%20Module%20Spec.md)
- [Oracle and CRM Adapter Detailed Implementation Spec.md](./Oracle%20and%20CRM%20Adapter%20Detailed%20Implementation%20Spec.md)
- [Colony Database Schema and Root API Spec.md](./Colony%20Database%20Schema%20and%20Root%20API%20Spec.md)
- [Implementation Ticket Breakdown and Dependency Matrix.md](./Implementation%20Ticket%20Breakdown%20and%20Dependency%20Matrix.md)
These together answer the five questions that matter most for implementation:
1. Where does the system live in the repo?
2. What does the TypeScript colony service need to look like?
3. How does it connect to the Python root?
4. What are the first domain missions?
5. In what order should the team build this without creating chaos?
## 7. What This Folder Is Not
This folder is not:
- a code-complete implementation
- a speculative research memo with no execution path
- a replacement plan for Project Velocity root
- a prompt-only system design
It is a build program. It is intended to be implementable by a real team under real product pressure.
## 8. Expected Outcome If We Build This Correctly
If this work is executed correctly, Project Velocity gets:
- a mission-oriented orchestration kernel
- safer domain-aware multi-agent behavior
- stronger Oracle and CRM intelligence flows
- artifact-level auditability
- a clearer path from internal automation to demo-safe commercial product behavior
That is the actual value of this project.
## 9. Suggested Next Step for You
The correct next move is:
1. read the five implementation-critical docs listed above
2. challenge any unclear runtime boundary early
3. convert the dependency matrix into actual implementation tickets
4. start with contract layer, root schema, root gateway, and colony service bootstrap before adapter work
Do not start from UI polish or generic multi-agent experimentation. Start from the runtime spine.
## 10. Bottom Line
This folder is the design and implementation package for a biomimetic orchestration layer that absorbs the useful parts of Open Multi Agent and Nemoclaw into Project Velocity without breaking the current root architecture.
If you approach it in reading order, the logic is coherent. If you approach it as disconnected files, it will feel heavier than it really is. The intended outcome is a governed colony runtime that can make Project Velocity more operational, more defensible, and more sellable.

457
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Mission Contracts and JSON Schemas Blueprint.md Normal file
View File

@@ -0,0 +1,457 @@
# Mission Contracts and JSON Schemas Blueprint
**Date:** 2026-04-14
**Status:** Draft implementation artifact
**Purpose:** Define the concrete contract model, JSON payload shapes, schema boundaries, versioning rules, and validation plan for the colony orchestration layer.
## 1. Purpose
This document converts the colony architecture into explicit contracts. The system must not rely on informal prompt chaining between stages. Every stage must read and write a typed artifact.
## 2. Contract Design Rules
### 2.1 Version Every Artifact
Every artifact must carry:
- `schema_name`
- `schema_version`
This is mandatory from day one so prompt package evolution and mission replay do not become brittle later.
### 2.2 Keep Root Metadata on Every Artifact
Every artifact should include:
- `mission_id`
- `tenant_id`
- `actor_id`
- `created_at`
- `producer`
This prevents orphan artifacts and makes debugging much easier.
### 2.3 Store Canonical Payloads as JSON
Root PostgreSQL persistence should store the canonical artifact payload in `jsonb`, even if some fields are also denormalized for indexing.
### 2.4 Prefer Small Composable Artifacts
Do not use one giant colony state blob. The correct design is multiple artifacts with explicit linkage.
## 3. Core Artifact Set
Phase 1 artifact set:
- `mission_envelope`
- `mission_task_graph`
- `prompt_package`
- `research_artifact`
- `librarian_pass`
- `worker_result`
- `aggregation_packet`
- `review_packet`
- `policy_decision`
- `writeback_proposal`
## 4. Mission Envelope Schema
### 4.1 Purpose
Defines the normalized mission before decomposition.
### 4.2 Suggested JSON Shape
```json
{
"schema_name": "mission_envelope",
"schema_version": "v1",
"mission_id": "uuid",
"mission_type": "oracle_advisory",
"origin_surface": "oracle",
"tenant_id": "tenant_velocity",
"actor_id": "oracle_operator",
"actor_role": "sales_director",
"user_goal": "Summarize the highest intent waterfront prospects and recommend next actions.",
"normalized_goal": "Produce a prioritized lead intelligence brief for waterfront prospects with actions.",
"risk_level": "medium",
"sensitivity_class": "internal_business",
"time_budget_ms": 45000,
"token_budget": 180000,
"requested_outputs": [
"reviewed_summary",
"writeback_proposal"
],
"context_refs": {
"lead_ids": ["lead_123"],
"page_id": null,
"campaign_ids": []
},
"created_at": "2026-04-14T12:00:00Z",
"producer": "python_root_gateway"
}
```
### 4.3 Required Fields
Required:
- `schema_name`
- `schema_version`
- `mission_id`
- `mission_type`
- `origin_surface`
- `tenant_id`
- `actor_id`
- `user_goal`
- `normalized_goal`
- `risk_level`
- `sensitivity_class`
- `created_at`
- `producer`
## 5. Mission Task Graph Schema
### 5.1 Purpose
Defines the DAG produced by the planner.
### 5.2 Suggested JSON Shape
```json
{
"schema_name": "mission_task_graph",
"schema_version": "v1",
"mission_id": "uuid",
"tasks": [
{
"task_id": "task_plan_01",
"role_type": "prompt_master",
"objective": "Prepare role-specific prompt packages for all execution tasks.",
"depends_on": [],
"required_capabilities": ["prompt_templates", "task_prompting"],
"allowed_tools": [],
"required_data_scopes": ["mission_metadata"],
"success_criteria": [
"prompt packages exist for all downstream tasks"
]
}
],
"created_at": "2026-04-14T12:00:02Z",
"producer": "planner_agent"
}
```
### 5.3 Required Rules
- every `task_id` must be unique within mission
- `depends_on` must only reference existing task IDs
- role types must be from an allowed enum in Phase 1
Phase 1 allowed role types:
- `planner`
- `prompt_master`
- `researcher`
- `librarian`
- `worker`
- `aggregator`
- `reviewer`
## 6. Prompt Package Schema
### 6.1 Purpose
Defines the exact worker-facing prompt instruction package.
### 6.2 Suggested JSON Shape
```json
{
"schema_name": "prompt_package",
"schema_version": "v1",
"mission_id": "uuid",
"task_id": "task_worker_01",
"package_id": "pkg_001",
"role_type": "worker",
"template_id": "crm_lead_intelligence_worker",
"template_version": "v1",
"examples_used": [
"crm_worker_example_01"
],
"system_prompt": "You are a lead intelligence specialist...",
"task_prompt": "Analyze the provided lead context...",
"constraints": {
"must_cite_sources": true,
"must_not_writeback_directly": true
},
"tool_scope": [
"crm_search",
"local_property_rag",
"external_search"
],
"model_route": "tier_high_reasoning",
"created_at": "2026-04-14T12:00:05Z",
"producer": "prompt_master_agent"
}
```
## 7. Research Artifact Schema
### 7.1 Purpose
Represents external evidence.
### 7.2 Suggested JSON Shape
```json
{
"schema_name": "research_artifact",
"schema_version": "v1",
"mission_id": "uuid",
"task_id": "task_research_01",
"artifact_id": "res_001",
"provider": "brave",
"query": "Dubai waterfront premium real estate 2026 demand",
"results": [
{
"title": "Market report title",
"url": "https://example.com/report",
"snippet": "Short snippet",
"source_type": "web",
"retrieved_at": "2026-04-14T12:01:00Z",
"confidence": 0.82
}
],
"created_at": "2026-04-14T12:01:01Z",
"producer": "researcher_agent"
}
```
## 8. Librarian Pass Schema
### 8.1 Purpose
Represents scoped internal routing instructions for workers.
### 8.2 Suggested JSON Shape
```json
{
"schema_name": "librarian_pass",
"schema_version": "v1",
"mission_id": "uuid",
"task_id": "task_worker_01",
"pass_id": "pass_001",
"allowed_resource_families": [
"crm.leads",
"crm.chat_logs"
],
"recommended_queries": [
{
"tool": "crm_search",
"query": "waterfront whale prospects"
}
],
"cached_previews": [
{
"label": "Lead summary",
"entity_type": "lead",
"entity_id": "lead_123",
"preview": "Whale prospect interested in marina-facing unit"
}
],
"expires_at": "2026-04-14T13:00:00Z",
"created_at": "2026-04-14T12:00:08Z",
"producer": "librarian_agent"
}
```
## 9. Worker Result Schema
### 9.1 Purpose
Represents normalized output from one worker.
### 9.2 Suggested JSON Shape
```json
{
"schema_name": "worker_result",
"schema_version": "v1",
"mission_id": "uuid",
"task_id": "task_worker_01",
"result_id": "wrk_001",
"output_text": "Lead 123 has the highest close probability...",
"structured_output": {
"priority": "high",
"recommended_action": "schedule executive follow-up"
},
"citations": [
{
"kind": "internal",
"ref": "lead_123"
}
],
"confidence": 0.79,
"tool_trace_refs": [
"toolcall_001",
"toolcall_002"
],
"created_at": "2026-04-14T12:02:10Z",
"producer": "worker_agent"
}
```
## 10. Aggregation Packet Schema
### 10.1 Purpose
Represents the synthesized draft before review.
### 10.2 Suggested JSON Shape
```json
{
"schema_name": "aggregation_packet",
"schema_version": "v1",
"mission_id": "uuid",
"packet_id": "agg_001",
"summary": "Top waterfront leads are concentrated in two whale segments...",
"evidence_matrix": [
{
"claim": "Lead 123 is high priority",
"support_refs": ["wrk_001", "lead_123"]
}
],
"contradictions": [],
"coverage_gaps": [],
"recommended_output": {
"format": "structured_brief"
},
"created_at": "2026-04-14T12:03:00Z",
"producer": "aggregator_agent"
}
```
## 11. Review Packet Schema
### 11.1 Purpose
Represents the one-pass review outcome.
### 11.2 Suggested JSON Shape
```json
{
"schema_name": "review_packet",
"schema_version": "v1",
"mission_id": "uuid",
"packet_id": "rev_001",
"review_status": "approved_with_edits",
"issues": [
{
"severity": "medium",
"message": "One recommendation needs clearer business justification."
}
],
"required_edits": [
"add one sentence linking lead score to action priority"
],
"approved_output": {
"summary": "Reviewed final answer here"
},
"created_at": "2026-04-14T12:03:20Z",
"producer": "reviewer_agent"
}
```
## 12. Policy Decision Schema
### 12.1 Purpose
Represents an explicit allow or deny decision from the guard plane.
### 12.2 Suggested JSON Shape
```json
{
"schema_name": "policy_decision",
"schema_version": "v1",
"mission_id": "uuid",
"decision_id": "pol_001",
"decision_type": "tool_scope",
"subject": "external_search",
"decision": "allow",
"reason": "mission sensitivity permits public web search",
"created_at": "2026-04-14T12:00:03Z",
"producer": "colony_policy_bridge"
}
```
## 13. Writeback Proposal Schema
### 13.1 Purpose
Represents a proposed root mutation. This is a proposal only, never a direct execution artifact.
### 13.2 Suggested JSON Shape
```json
{
"schema_name": "writeback_proposal",
"schema_version": "v1",
"mission_id": "uuid",
"proposal_id": "wb_001",
"target_entity_type": "lead",
"target_entity_id": "lead_123",
"action_type": "lead_stage_update",
"payload": {
"kanban_status": "Negotiation"
},
"requires_approval": true,
"created_at": "2026-04-14T12:03:25Z",
"producer": "reviewer_agent"
}
```
## 14. Root Table Mapping
Recommended root table to artifact mapping:
- `colony_missions` -> mission envelope
- `colony_tasks` -> task graph task records
- `colony_prompt_packages` -> prompt package
- `colony_research_artifacts` -> research artifact
- `colony_librarian_passes` -> librarian pass
- `colony_worker_results` -> worker result
- `colony_aggregation_packets` -> aggregation packet
- `colony_review_packets` -> review packet
- `colony_policy_decisions` -> policy decision
## 15. Validation Strategy
Validation should occur in three layers:
1. TypeScript runtime validation in colony service
2. Python root payload validation before persistence
3. database-level required column and foreign key validation
## 16. Immediate Implementation Order
Implement schemas in this order:
1. mission envelope
2. task graph
3. prompt package
4. worker result
5. review packet
6. policy decision
7. librarian pass
8. research artifact
9. aggregation packet
10. writeback proposal
## 17. Bottom Line
The colony will only be dependable if every stage is contract-first. This document defines the contract spine that allows the rest of the implementation to stay disciplined.

292
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Oracle and CRM Adapter Detailed Implementation Spec.md Normal file
View File

@@ -0,0 +1,292 @@
# Oracle and CRM Adapter Detailed Implementation Spec
**Date:** 2026-04-14
**Status:** Draft implementation artifact
**Purpose:** Define the exact adapter behavior, request and response contracts, file placements, runtime flow, and acceptance criteria for the two first colony mission consumers: Oracle and CRM.
## 1. Purpose
Oracle and CRM are the first two adapters because they already have the strongest root primitives and the clearest near-term business value.
## 2. Adapter Strategy
The adapters must not bypass current root ownership. They are translation layers between:
- current Project Velocity product surfaces
- the new colony runtime
The adapters are therefore integration boundaries, not new domain roots.
## 3. Oracle Adapter Spec
### 3.1 Oracle Mission Type
Primary Phase 1 mission type:
- `oracle_advisory`
### 3.2 Oracle Entry Points
The adapter should be callable from:
- Oracle helper routes under `backend/api/routes_oracle.py`
- Oracle v1 prompt submission flow under `backend/oracle/router_v1.py`
### 3.3 Oracle Context Inputs
Oracle mission creation should include:
- `tenant_id`
- `actor_id`
- `actor_role`
- `page_id`
- `branch_id`
- `prompt`
- `conversation_context`
- optional `target_lead_id`
### 3.4 Oracle Adapter File Placement
TypeScript:
- `services/colony-orchestrator/src/adapters/oracle-adapter.ts`
Python root:
- `backend/services/colony_gateway.py`
- `backend/api/routes_colony.py`
- adapter-specific entry hook in `backend/api/routes_oracle.py`
### 3.5 Oracle Mission Creation Flow
1. Oracle prompt arrives in current root route.
2. Root builds mission envelope with type `oracle_advisory`.
3. Root persists mission envelope.
4. Root sends mission request to colony service.
5. Colony planner creates task graph.
6. Prompt master creates prompt packages.
7. Librarian prepares root-data route cards.
8. Worker tasks execute against scoped CRM, Oracle, and MCP tools.
9. Aggregator creates answer packet.
10. Reviewer returns reviewed output and optional writeback proposal.
11. Root returns reviewed packet to Oracle UI.
### 3.6 Oracle Output Contract
The reviewed Oracle response should include:
- final summary text
- structured sections for renderable Oracle use
- citations or internal references where applicable
- optional `writeback_proposal`
- mission metadata
### 3.7 Oracle Writeback Rules
Oracle adapter may propose:
- lead note update
- lead score update
- lead stage update
- Oracle action creation
Oracle adapter may not directly commit these changes from colony workers. All mutations must pass through root writeback policy.
### 3.8 Oracle Acceptance Criteria
Oracle adapter is complete for Sprint 1 when:
- a live Oracle prompt can create a mission
- the mission returns a reviewed response
- the response references real root data
- a writeback proposal can be generated
- no regression occurs in existing Oracle v1 canvas behavior
## 4. CRM Adapter Spec
### 4.1 CRM Mission Type
Primary Phase 1 mission type:
- `crm_lead_intelligence`
### 4.2 CRM Entry Points
The adapter should be callable from:
- new colony routes
- CRM UI actions triggered from pipeline, lead inspector, or dashboard surfaces
### 4.3 CRM Context Inputs
CRM mission creation should include:
- `tenant_id`
- `actor_id`
- `actor_role`
- `lead_id`
- optional `kanban_status`
- optional `query_focus`
- optional `include_external_research`
### 4.4 CRM Adapter File Placement
TypeScript:
- `services/colony-orchestrator/src/adapters/crm-adapter.ts`
Python root:
- `backend/services/colony_gateway.py`
- `backend/api/routes_colony.py`
- optional helper integration in `backend/api/routes_crm.py`
### 4.5 CRM Mission Creation Flow
1. CRM action selects a lead or cohort.
2. Root creates `crm_lead_intelligence` mission envelope.
3. Root fetches lead and related chat context.
4. Root sends mission envelope to colony.
5. Planner creates tasks for context analysis, optional external research, synthesis, and review.
6. Prompt master creates role-specific packages.
7. Librarian provides scoped passes to leads and chat logs.
8. Researcher provides public market or project evidence only when requested.
9. Worker results are aggregated and reviewed.
10. Root returns structured CRM guidance to UI.
### 4.6 CRM Output Contract
Reviewed CRM output should include:
- lead priority assessment
- intent summary
- recommended next step
- message draft suggestions
- risk flags
- optional writeback proposal
### 4.7 CRM Writeback Rules
CRM adapter may propose:
- lead stage move
- lead note append
- lead qualification update
- follow-up task creation
CRM adapter may not directly write these changes from colony workers.
### 4.8 CRM Acceptance Criteria
CRM adapter is complete for Sprint 1 when:
- one lead mission runs end to end
- reviewed output uses real lead and chat data
- output includes actionable next-step suggestions
- optional writeback proposal is generated safely
- existing CRM routes continue to function unchanged
## 5. Shared Adapter Contracts
### 5.1 Root Request Shape
Both adapters should submit missions to root in a shape like:
```json
{
"mission_type": "oracle_advisory",
"origin_surface": "oracle",
"tenant_id": "tenant_velocity",
"actor_id": "oracle_operator",
"actor_role": "sales_director",
"user_goal": "Original operator prompt",
"context_refs": {
"lead_ids": ["lead_123"],
"page_id": "page_abc",
"branch_id": "main"
}
}
```
### 5.2 Root Response Shape
Both adapters should receive:
```json
{
"mission_id": "uuid",
"status": "completed",
"review_status": "approved_with_edits",
"final_output": {},
"writeback_proposal": null,
"trace_summary": {
"task_count": 5,
"duration_ms": 12850
}
}
```
## 6. Adapter-Specific Files to Modify
### 6.1 Python Files
Create or modify:
- `backend/api/routes_colony.py`
- `backend/services/colony_gateway.py`
- `backend/services/colony_repository.py`
- `backend/main.py`
- `backend/api/routes_oracle.py`
- `backend/api/routes_crm.py`
### 6.2 TypeScript Files
Create:
- `services/colony-orchestrator/src/adapters/oracle-adapter.ts`
- `services/colony-orchestrator/src/adapters/crm-adapter.ts`
- `services/colony-orchestrator/src/adapters/python-root-client.ts`
## 7. Test Plan
### 7.1 Oracle Adapter Tests
Required:
- mission creation test
- mission status retrieval test
- reviewed output shape test
- writeback proposal validation test
- Oracle regression test
### 7.2 CRM Adapter Tests
Required:
- mission creation test
- real lead context enrichment test
- chat log usage test
- external research optionality test
- writeback proposal validation test
## 8. Demo and Sales Relevance
These two adapters are the fastest route to a system that supports sales.
Oracle proves:
- strategic advisory
- reviewed synthesis
- structured delivery
CRM proves:
- actionable lead intelligence
- grounded recommendations
- commercial follow-up value
If these two adapters work reliably, Project Velocity can begin demonstrating a real colony orchestration capability rather than only promising one.
## 9. Bottom Line
Oracle and CRM should be built first because they best match the current root state, the architecture design, and the shortest path to revenue-supporting demonstrations.

296
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Product Requirements Document (PRD)_ Biomimetic Agentic Orchestration Layer.md Normal file
View File

@@ -0,0 +1,296 @@
# Product Requirements Document (PRD)_ Biomimetic Agentic Orchestration Layer
**Date:** 2026-04-14
**Status:** Draft
**Product Name:** `Project Velocity Colony Orchestration Layer`
## Purpose
This PRD defines the product goals, user value, scope, major capabilities, and measurable outcomes for the biomimetic orchestration layer that will combine Project Velocity's Nemoclaw-derived governance layer with a forked Open Multi Agent execution kernel.
## Problem Statement
Project Velocity has many valuable intelligent surfaces, but they are still too fragmented:
- Oracle has strong planning and canvas concepts
- CRM now has a real backend and writeback paths
- Catalyst has real backend behavior and UI
- Sentinel has live signal ingestion and reasoning paths
- MCP and Nemoclaw append layers exist
What is missing is a robust orchestration layer that can coordinate these systems reliably, safely, and modularly.
The current state creates five business problems:
1. Valuable domain logic exists in separate islands.
2. Prompt engineering is not yet a governed subsystem.
3. Tool and data routing are still too implicit.
4. Multi-step goals require bespoke chains instead of reusable mission flows.
5. Result quality varies because decomposition, evidence gathering, and review are not standardized.
## Product Vision
Create a colony-style orchestration platform that receives a high-level user goal and turns it into a governed mission executed by specialized agents that:
- plan
- research
- retrieve internal context
- execute task-specific work
- aggregate results
- review the result once
- return a final answer or domain action
The product must feel cohesive at the user layer and modular at the engineering layer.
## Product Goals
### Primary Goals
The system must let Project Velocity convert ambiguous goals into structured, auditable, multi-agent execution across Oracle, CRM, Catalyst, and future surfaces.
The system must preserve governance and safety through external policy controls rather than trusting worker prompts alone.
The system must remain extensible, so new agent roles, domain adapters, tools, and model providers can be introduced without redesigning the whole stack.
### Secondary Goals
The system should improve latency and quality by parallelizing independent work and reducing unnecessary context sharing.
The system should learn from prior successful patterns through prompt templates, artifact registries, and pheromone-style telemetry.
The system should allow progressive productization:
- internal operator use first
- assisted customer-facing use next
- higher autonomy only after domain hardening
## Non-Goals
This release is not intended to be:
- a generic public AI operating system from day one
- a long-running checkpoint-heavy workflow engine in Sprint 1
- a replacement for the root FastAPI backend
- a replacement for Oracle v1 canvas
- a replacement for Sentinel ownership
## Product Users
### Primary Users
Internal operators using Oracle, CRM, Catalyst, or research workflows.
### Secondary Users
Sales and strategy operators who need:
- lead analysis
- project intelligence
- campaign planning
- internal synthesis
### System-Level Consumers
Project Velocity modules themselves:
- Oracle
- CRM
- Catalyst
- Sentinel
- future Dream Weaver orchestration
## Core User Stories
### Mission Decomposition
As an operator, I want to submit a high-level goal and have the system break it into explicit, traceable task units so I can trust how work is being performed.
### Prompt Specialization
As an operator, I want the system to create better task-specific prompts than a generic coordinator would produce, so specialist agents receive clearer instructions.
### Internal Knowledge Routing
As an operator, I want relevant CRM, Oracle, project, and asset context routed automatically to workers without exposing unrelated data.
### External Research
As an operator, I want the system to gather external evidence with citations and provenance so outputs are grounded rather than invented.
### Aggregation and Review
As an operator, I want a final result that has already been synthesized and reviewed once so I see a clean answer, not a pile of partial agent outputs.
### Safe Writeback
As a product owner, I want writebacks to CRM or Oracle to be explicit, validated, and auditable before they change product state.
## Product Capabilities
### Capability 1: Mission Intake
The system accepts a goal from:
- Oracle
- CRM
- Catalyst
- internal operations
It normalizes the goal into a mission object with budgets, risk, scope, and output targets.
### Capability 2: Planner
The planner converts the mission into a task DAG with:
- dependencies
- role requirements
- evidence needs
- success criteria
### Capability 3: Prompt Master
The prompt master produces worker-ready prompt packages using:
- templates
- exemplars
- mission constraints
- task type metadata
### Capability 4: Research
The research subsystem acquires public information through search and browsing and returns evidence packets with citations.
### Capability 5: Librarian Routing
The librarian subsystem issues scoped access passes for internal knowledge and cached artifacts.
### Capability 6: Worker Execution
Workers execute task-specific prompts with bounded tools and scoped context.
### Capability 7: Aggregation and Review
Results are aggregated into one structured answer and then reviewed once before final delivery.
### Capability 8: Governance
Nemoclaw-style policy controls mediate:
- model routing
- tool access
- external network calls
- writeback actions
- audit logging
## Product Requirements
### Must-Have Requirements
The product must support mission decomposition, prompt packaging, internal retrieval routing, external research, worker execution, aggregation, review, and final delivery.
The product must expose audit traces for all mission steps.
The product must allow domain adapters for Oracle, CRM, Catalyst, and Sentinel.
The product must separate planning logic from execution logic.
The product must provide structured internal artifacts, not only free-form chat.
### Should-Have Requirements
The product should support multiple model providers in one mission.
The product should support human approval gates for sensitive missions.
The product should keep reusable prompt and artifact histories.
The product should expose observability for latency, cost, and failure hotspots.
### Could-Have Requirements
The product could later support:
- long-running mission resumes
- colony simulation dashboards
- community-contributed specialist packs
- adaptive self-tuning role selection
## Success Metrics
### Quality Metrics
- percentage of missions completed without manual intervention
- percentage of final outputs approved without manual rewrite
- evidence citation coverage for research missions
- contradiction detection rate during review
### Operational Metrics
- average mission latency by mission type
- average token cost by mission type
- retry frequency
- writeback rejection rate
### Product Metrics
- number of Velocity surfaces successfully using the colony layer
- number of mission templates reusable across domains
- reduction in bespoke one-off prompt chains
## Release Strategy
### Phase 1
Build the colony kernel and run Oracle and CRM missions in assisted mode.
### Phase 2
Add Catalyst missions and stronger research workflows.
### Phase 3
Introduce Sentinel evidence feeds and more controlled automation.
## Key Trade-Offs
Open Multi Agent gives speed, simplicity, and dynamic decomposition, but it does not provide production-grade persistence or governance by itself.
Nemoclaw-style governance gives safety and bounded execution, but if over-applied it can slow down iteration and developer ergonomics.
The correct product strategy is therefore:
- Open Multi Agent fork as execution kernel
- Nemoclaw-derived policy layer as guard plane
- Project Velocity root as source of truth for domain state
## Risks
### Architectural Risk
If the system becomes too generic, it will not solve Velocity's real domain problems.
### Safety Risk
If governance remains prompt-level instead of infrastructure-level, the system will be brittle.
### Product Risk
If prompt packaging and librarian routing are not formalized early, the colony will look clever in demos but fail under production variance.
## Acceptance Criteria
The PRD is satisfied when the first working product release can:
- receive a real goal from Oracle or CRM
- create a mission and task graph
- assign specialized workers
- route internal and external context correctly
- produce an aggregated and reviewed answer
- log every material step
- block disallowed tool or writeback behavior
## Bottom Line
This product is the missing orchestration spine for Project Velocity.
It should not be built as a monolithic assistant. It should be built as a biomimetic colony runtime with strict interfaces, external governance, and domain-aware adapters.

690
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Project Velocity - Biomimetic Agentic Orchestration Layer (First Principles).md Normal file
View File

@@ -0,0 +1,690 @@
# Project Velocity - Biomimetic Agentic Orchestration Layer (First Principles)
**Date:** 2026-04-14
**Status:** Draft for design alignment
**Working Name:** `Project Velocity Colony Orchestration Layer`
**Purpose:** Define the product and architecture from first principles before implementation, using current Nemoclaw realities and the public Open Multi Agent framework as the two main input layers.
## Executive Summary
Project Velocity needs an orchestration layer that can turn an ambiguous high-level goal into reliable, auditable, domain-aware work across research, CRM, Oracle, Catalyst, Sentinel, and eventually Dream Weaver and iOS-assisted flows.
The missing piece is not another chatbot. The missing piece is a colony runtime:
- one system that decomposes goals into work
- routes the right context to the right specialist
- controls tool use and data access
- aggregates and reviews results before they touch the user
- stays explainable and governable under production constraints
The best public runtime reference for the orchestration kernel is Open Multi Agent:
- dynamic goal-to-task decomposition
- dependency-aware task DAG execution
- shared memory and agent messaging
- multi-model teams
- MCP connectivity
- built-in tool registry and execution loop
The best reference for safety and operational control is the current Nemoclaw layer already present in Project Velocity:
- policy and workflow boundary thinking
- inference routing
- auditability
- model isolation as a design value
- root-owned Oracle, MCP, and workflow helpers already adapted into Python
This document proposes a biomimetic orchestration system inspired by ant colonies and bee hives, but expressed as concrete software architecture rather than metaphor alone.
## Assumptions and Constraints
### Assumptions
Open Multi Agent is a legitimate public upstream and can be studied, forked, and modified under its MIT license. The public repository and its source files are treated as a primary technical reference.
The current Project Velocity root remains authoritative for:
- FastAPI
- PostgreSQL
- Oracle v1 canvas and prompt orchestration
- Sentinel backend ownership
- Catalyst backend ownership
- CRM canonical storage
The orchestration layer is therefore an internal subsystem, not a second product root.
The system will be used first inside Project Velocity, but should be designed so it can later become a reusable platform across Oracle, Catalyst, Sentinel, CRM, and future agentic modules.
### Constraints
The architecture must not rely on leaked or provenance-unclear code. Public ideas can inspire design. Public licensed code can be forked. Leaked Anthropic artifacts should not be incorporated into product code or product docs as implementation source material.
The system must support production governance:
- explainable task decomposition
- traceable tool usage
- bounded model context
- scoped data access
- safe egress and write controls
The system must work with both cloud and local models, because Project Velocity already spans NVIDIA-hosted inference, local model ambitions, and domain-specific privacy needs.
The system must remain modular. If Open Multi Agent becomes too constraining later, Project Velocity should be able to replace the execution kernel without rewriting all domain contracts.
## Reference Sources and Rationale
### Local Sources
Primary local sources:
- `.Agent Context/Tech/Tools Understanding.md`
- `.Agent Context/Tech/README-Open Multi-Agent.md`
- `.Agent Context/Sprint 1/nemoclaw_setup_truth.md`
- current root code under `backend/oracle`, `backend/services`, `backend/api`, and Sentinel routes
These sources describe the current Project Velocity truth and the internal understanding of Nemoclaw and Open Multi Agent.
### Upstream Public Source
Primary upstream source:
- `https://github.com/JackChen-me/open-multi-agent`
The inspected codebase shows a small TypeScript framework centered around:
- `src/orchestrator/orchestrator.ts`
- `src/task/queue.ts`
- `src/orchestrator/scheduler.ts`
- `src/agent/runner.ts`
- `src/agent/pool.ts`
- `src/team/team.ts`
- `src/memory/shared.ts`
- `src/tool/mcp.ts`
The key takeaway is that Open Multi Agent is intentionally narrow:
- strong at dynamic decomposition and short-lived multi-agent execution
- intentionally weak at persistence, long-lived workflow recovery, and policy infrastructure
That is useful. It gives Project Velocity a sharp orchestration kernel, but it still needs a colony control layer around it.
## System Vision
The target system is a colony-style orchestration plane for Project Velocity.
It should behave like this:
1. A user gives a goal to the system.
2. The system converts that goal into a structured mission.
3. The mission is decomposed into a task graph.
4. A prompt-design specialist prepares role-specific task prompts.
5. Internal and external context is routed to specialists based on need.
6. Specialists execute in parallel where possible.
7. Outputs are aggregated, reviewed once, and normalized.
8. The final result returns to the user or back into a Project Velocity subsystem.
This is not one monolithic agent. It is a managed ecology of small, bounded agents with shared purpose.
## Biomimicry Metaphor in Architecture
The biomimicry metaphor should not remain branding. It should shape architectural behavior.
### Queen
The queen is not the worker. In software terms, the queen is the authoritative interface and coordination identity.
Responsibilities:
- receive intent
- maintain mission identity
- preserve user continuity
- own the final delivery boundary
The queen should not personally execute every step. That would collapse the colony into a single overloaded agent.
### Planner Ant / Foreman Ant
This is the decomposition specialist. It turns a mission into a DAG of work.
In Open Multi Agent terms, this aligns with the coordinator agent. In Project Velocity, this should become a first-class mission planner with stronger contracts than a generic coordinator prompt.
### Prompt Master Ant
This role creates worker-specific prompts using:
- prompt templates
- task type libraries
- past prompt exemplars
- runtime constraints
- tool and policy scope
This is a critical improvement over vanilla orchestration. Open Multi Agent assumes task prompts are mostly generated inline. Project Velocity should separate task decomposition from prompt engineering for execution.
### Researcher Ant
This role handles external evidence gathering:
- search
- browsing
- citation capture
- GUI browser where needed
- headless browser where sufficient
This ant should produce structured evidence artifacts, not prose alone.
### Librarian Ant
This role handles internal memory, repositories, CRM, and domain stores.
Its job is not just retrieval. Its job is routing. It creates scoped access passes for workers so they know:
- which sources are relevant
- which data is allowed
- which indexes to search
- which cached snippets already exist
This is the colony equivalent of pheromone trails plus route cards.
### Worker Ants
These are ephemeral specialists, spawned with:
- task objective
- prompt package
- tool package
- librarian pass
- research evidence where needed
They should be disposable and reproducible.
### Assistant Ants
These are lightweight helper processes that fetch:
- cached summaries
- thumbnails
- source previews
- snippet citations
They reduce bandwidth and keep workers from redoing simple retrieval.
### Aggregator Ant
This role composes the result packet:
- original goal
- enhanced mission
- task outputs
- evidence
- contradictions
- unresolved risks
### Reviewer Ant
This role performs one bounded review pass against:
- completeness
- internal consistency
- policy compliance
- citation and provenance quality
- user-goal alignment
One review loop is a good default. More loops increase cost and delay unless risk level justifies them.
### Pheromone Layer
In software, pheromones correspond to lightweight reinforced signals:
- prompt-template success rates
- tool effectiveness by task type
- source usefulness
- failure hotspots
- cache affinity
This should not be a giant opaque memory store. It should be compact telemetry that influences planning and routing.
### Waggle Dance
The bee waggle dance is a good design metaphor for compressed broadcasting. Not every ant needs the full artifact. The system should support:
- short mission summaries
- confidence and urgency signals
- route hints
- evidence pointers
That means the colony needs both full artifact channels and compressed signal channels.
## First-Principles Architecture
The orchestration layer should be built from the following software principles.
### Principle 1: Separate intent, planning, execution, and governance
If one agent owns everything, the system becomes hard to reason about and easy to break. The colony should split:
- intent ingestion
- task planning
- prompt preparation
- execution
- review
- policy enforcement
### Principle 2: Scope context aggressively
A common failure mode in agent systems is uncontrolled context flooding. Workers should receive:
- only the data needed for the task
- compressed mission context
- explicitly granted tool scope
- explicit evidence packets
This matches the librarian-pass model.
### Principle 3: Prefer artifacts over free-form chat
Each stage should emit structured artifacts, not only prose. Examples:
- mission spec
- task graph
- prompt package
- research packet
- library pass
- worker result
- aggregation packet
- review findings
This keeps the system composable and testable.
### Principle 4: Keep the colony stateful, keep workers mostly stateless
The colony needs memory. Individual workers should be cheap to create and destroy. State should live in:
- mission store
- artifact store
- pheromone store
- audit log
- policy and prompt registries
### Principle 5: Governance must be external to worker prompts
Prompt-only safety is not enough. This is where Nemoclaw matters. The worker should not be trusted to self-police. Instead:
- tool permissions must be granted externally
- network egress must be bounded externally
- data writebacks must be validated externally
- model routing should be policy-driven
### Principle 6: Parallelism should be earned, not assumed
The framework can run tasks in parallel. The product should decide when:
- tasks are independent
- tasks share too much mutable state
- cost or latency budgets require serialization
- review or quorum is needed before downstream work
### Principle 7: Review should target risk, not everything equally
Not every mission needs the same rigor. The system should escalate review by:
- financial risk
- user-facing impact
- external actuation
- writeback side effects
- low-confidence evidence
## Proposed Functional Architecture and Key Roles
```mermaid
flowchart TD
U[User or Product Surface] --> Q[Queen Interface]
Q --> M[Mission Normalizer]
M --> P[Planner Ant]
P --> PM[Prompt Master Ant]
P --> R[Researcher Ant]
P --> L[Librarian Ant]
PM --> W1[Worker Ants]
R --> W1
L --> W1
W1 --> A[Aggregator Ant]
A --> V[Reviewer Ant]
V --> Q
Q --> O[User or Calling Product]
G[Governance and Nemoclaw Guard Plane] --> P
G --> PM
G --> W1
G --> V
S[Pheromone and Artifact Stores] --> P
S --> PM
S --> L
S --> A
```
### Layer 1: Colony Kernel
This is the forked Open Multi Agent core, extended but still recognizable:
- mission coordinator
- task DAG compiler
- scheduler
- agent pool
- tool dispatcher
- short-lived memory and message primitives
### Layer 2: Colony Control Plane
This is the new Project Velocity layer that Open Multi Agent does not provide:
- mission registry
- artifact schemas
- prompt registry
- policy engine
- librarian and research services
- pheromone scoring
- review gates
- observability
### Layer 3: Nemoclaw Guard Plane
This layer governs:
- model routing
- network permissions
- tool access policies
- filesystem access
- human approvals for sensitive actions
- audit logs
### Layer 4: Velocity Domain Adapters
Domain adapters map colony artifacts to actual product surfaces:
- Oracle adapter
- CRM adapter
- Catalyst adapter
- Sentinel adapter
- Dream Weaver adapter
## Data Model and Interfaces
The system should be artifact-native. Core objects should include:
### Mission Envelope
Fields:
- `mission_id`
- `origin_surface`
- `user_goal`
- `normalized_goal`
- `risk_level`
- `time_budget_ms`
- `token_budget`
- `sensitivity_class`
- `requested_outputs`
### Task Graph
Fields:
- `task_id`
- `mission_id`
- `role_type`
- `objective`
- `depends_on`
- `required_capabilities`
- `allowed_tools`
- `required_data_scopes`
- `success_criteria`
### Prompt Package
Fields:
- `prompt_package_id`
- `task_id`
- `role_prompt`
- `template_id`
- `template_version`
- `examples_used`
- `constraints`
- `tool_instructions`
### Research Artifact
Fields:
- `artifact_id`
- `task_id`
- `source_url`
- `source_type`
- `retrieved_at`
- `summary`
- `snippet`
- `citation_confidence`
### Librarian Pass
Fields:
- `pass_id`
- `worker_task_id`
- `allowed_indexes`
- `allowed_entities`
- `recommended_paths`
- `cached_previews`
- `expiry`
### Worker Result
Fields:
- `result_id`
- `task_id`
- `output`
- `structured_output`
- `citations`
- `confidence`
- `tool_trace`
- `cost`
- `duration_ms`
### Aggregation Packet
Fields:
- `mission_id`
- `summary`
- `evidence_matrix`
- `contradictions`
- `coverage_gaps`
- `recommended_answer`
### Review Packet
Fields:
- `mission_id`
- `review_status`
- `issues`
- `required_edits`
- `approved_output`
## Interaction and Workflow Description
### Standard Mission Flow
```mermaid
flowchart LR
A[Goal Submitted] --> B[Mission Normalization]
B --> C[Task Graph Planning]
C --> D[Prompt Package Generation]
D --> E[Research and Librarian Routing]
E --> F[Worker Execution]
F --> G[Aggregation]
G --> H[Single Review Pass]
H --> I[Final Delivery]
```
### Internal Coordination Model
The planner creates the mission map. The prompt master transforms tasks into execution-ready prompt packages. The librarian prepares scoped passes. The researcher gathers external evidence. Workers consume these inputs and produce worker results. The aggregator converts worker results into one coherent answer. The reviewer performs one bounded audit pass. The queen delivers the result.
This flow is clearer and safer than a generic one coordinator, many workers, final answer pipeline because it prevents prompt design, evidence gathering, data routing, and synthesis from collapsing into one role.
## Improvements and Recommendations
### 1. Do not make the queen a single execution bottleneck
Your instinct is correct, but the queen should be the mission owner, not the main reasoner. Use the queen as interface and final sign-off identity. Use planner plus prompt master beneath it.
### 2. Split external research from internal retrieval
Researcher and librarian should remain separate. External evidence and internal knowledge have different trust models, latencies, and security constraints.
### 3. Introduce an immune system role
The metaphor needs one more biological component: an immune role. This should be the policy and anomaly layer that spots:
- tool misuse
- prompt injection
- suspicious source behavior
- uncontrolled recursion
- invalid writeback attempts
This can be implemented partly through Nemoclaw and partly through colony review logic.
### 4. Use pheromone decay
Do not keep every success signal forever. Prompt and route heuristics should decay over time unless reinforced by fresh outcomes. Otherwise the system will overfit stale patterns.
### 5. Add quorum mode for sensitive operations
For external actions or high-value CRM writebacks, require either:
- reviewer approval
- human approval
- dual-agent agreement
### 6. Keep a strict artifact schema
The more the colony relies on free-form prose between agents, the less governable it becomes. Use JSON artifacts internally and render prose only at the delivery layer.
### 7. Make Project Velocity domains first-class specializations
The colony should not remain generic for too long. It should gain specialized ant classes for:
- Oracle composition and visualization
- CRM and lead operations
- Catalyst marketing planning
- Sentinel session interpretation
- Dream Weaver asset workflow dispatch
### 8. Refuse all context to every ant
That feels powerful but is structurally wrong. It increases cost, hallucination risk, leakage risk, and reasoning confusion. The librarian-pass model is the correct design.
### 9. Treat prompt engineering as a subsystem, not a field
The prompt master should have:
- prompt templates
- prompt exemplars
- evaluation traces
- role-specific quality metrics
This becomes a product asset, not just a string builder.
### 10. Keep legal provenance clean
Use public code, public docs, and internal original work only. Do not build from leaked Claude framework material.
## Migration and Fork Strategy
### What to fork from Open Multi Agent
Fork and extend:
- orchestrator model
- task queue
- scheduler
- agent pool
- tool runner
- MCP connectivity
### What not to inherit blindly
Do not inherit its current limitations as product assumptions:
- no persistence
- no checkpointing
- no colony-level review layer
- no domain adapters
- no policy-first execution boundaries
### What to absorb from Nemoclaw
Absorb conceptually and through current root-owned code:
- governance outside worker prompts
- policy-controlled egress
- model routing by sensitivity
- auditability
- writeback controls
### Proposed Runtime Boundary
Recommended shape:
- TypeScript colony service forks Open Multi Agent and becomes the colony kernel
- FastAPI remains source of truth for product data and domain APIs
- Python Nemoclaw-root helpers remain available for Oracle, MCP, and workflow planning
- the colony talks to FastAPI through internal service contracts, not by bypassing the root
## What This Means for Project Velocity
This colony layer is valuable because Project Velocity already has the right pieces but not yet the right orchestration coherence.
Today the product has:
- Oracle reasoning surfaces
- CRM data and writeback paths
- Sentinel scoring and live sessions
- Catalyst execution surfaces
- MCP and Nemoclaw append layers
What it lacks is a disciplined colony runtime that can coordinate these without turning every subsystem into a bespoke prompt chain.
The correct next move is to define the system formally, then build the minimal colony kernel that can support one or two real mission classes first.
## Recommended Initial Mission Classes
The first implementation should focus on three mission types:
1. Oracle advisory missions
These already align with prompt orchestration, structured output, and writeback planning.
2. CRM intelligence missions
Lead research, prioritization, messaging strategy, and action recommendation are ideal for multi-agent decomposition.
3. Catalyst marketing missions
Campaign research, audience strategy, creative ideation, and review are naturally multi-role.
Sentinel should integrate as an evidence provider before becoming a fully autonomous mission class.
## Bottom Line
Yes, the idea makes sense.
The strongest version of it is not Jarvis with many ants. The strongest version is:
- a mission-oriented colony kernel
- a prompt-master subsystem
- a librarian routing subsystem
- a research subsystem
- bounded worker execution
- an aggregation and review pair
- a Nemoclaw-governed policy shell
- clean artifact schemas and domain adapters for Project Velocity
That is the right abstraction to pursue.

230
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Prompt Master, Librarian, and Researcher Role Spec.md Normal file
View File

@@ -0,0 +1,230 @@
# Prompt Master, Librarian, and Researcher Role Spec
**Date:** 2026-04-14
**Status:** Draft implementation artifact
**Purpose:** Define the three highest-leverage role systems inside the colony so their behavior is deterministic, governable, and implementable without ad hoc interpretation.
## 1. Purpose
The planner, prompt master, librarian, and researcher chain is where colony quality is won or lost. If this layer is vague, workers receive inconsistent context, external evidence quality collapses, and the final review stage becomes overloaded.
This document defines the exact role boundaries for:
- prompt master
- librarian
- researcher
## 2. Why These Roles Need a Separate Spec
These roles are special because they shape every downstream worker.
Workers may be numerous and disposable. These three roles are not. They are multiplier roles:
- prompt master multiplies task clarity
- librarian multiplies internal retrieval quality
- researcher multiplies external evidence quality
If these roles are weak, adding more workers only scales confusion.
## 3. Prompt Master
### 3.1 Purpose
Convert planned tasks into execution-grade prompt packages.
### 3.2 Inputs
- mission envelope
- task graph
- mission budgets
- mission risk class
- domain adapter metadata
- template registry
- exemplar registry
### 3.3 Outputs
- one `prompt_package` per executable task
- optional prompt package for aggregator
- optional prompt package for reviewer
### 3.4 Required Behavior
The prompt master must:
- select the right prompt structure by mission type and role type
- bind explicit success criteria from the task graph
- encode tool limits in the prompt package metadata, not only in prose
- choose a model route appropriate for the task
- distinguish between task instructions and domain context
- keep prompts concise enough to preserve budget for retrieval and evidence
### 3.5 Forbidden Behavior
The prompt master must not:
- invent data access beyond the librarian pass
- hide approval requirements inside free-form prose
- issue direct write instructions to workers
- collapse multiple tasks into one giant prompt for convenience
### 3.6 Acceptance Criteria
- every executable task has a prompt package
- every prompt package references its `task_id`
- every prompt package declares a model route and tool scope
- packages are reproducible from the same inputs
## 4. Librarian
### 4.1 Purpose
Convert mission context and task intent into scoped internal access guidance.
### 4.2 Inputs
- mission envelope
- task graph
- prompt packages
- domain source catalog
- tenant and actor scope
- cache index
### 4.3 Outputs
- one `librarian_pass` per worker task
- optional route cards for aggregator and reviewer
### 4.4 What a Librarian Pass Must Contain
- `pass_id`
- `mission_id`
- `task_id`
- `resource_families`
- allowed entity IDs when relevant
- recommended retrieval order
- summary snippets
- freshness hints
- expiry time if needed
### 4.5 Required Behavior
The librarian must:
- narrow context, not widen it
- prefer route cards over raw data dumps
- separate mandatory sources from optional sources
- tell the worker where evidence likely lives
- surface cached snippets when available
### 4.6 Forbidden Behavior
The librarian must not:
- copy entire datasets into worker context
- silently elevate data access
- assume all workers need all sources
- become a general-purpose reasoner
### 4.7 Acceptance Criteria
- workers can locate domain evidence without broad data flooding
- passes are task-specific
- sensitive scopes are explicit and auditable
## 5. Researcher
### 5.1 Purpose
Gather and normalize external evidence for tasks that require web or browser-based research.
### 5.2 Inputs
- mission envelope
- task graph
- researcher-designated task objectives
- tool policy decisions
- search provider configuration
- browser provider configuration
### 5.3 Outputs
- `research_artifact` objects with normalized citations
### 5.4 Required Behavior
The researcher must:
- search only when policy permits
- produce citations with source URL, title, provider, and retrieval timestamp
- separate observed evidence from inferred interpretation
- include enough provenance for reviewer audit
- support both search-engine retrieval and browser-based extraction
### 5.5 Forbidden Behavior
The researcher must not:
- claim unsupported facts
- inject uncited claims into downstream workers
- access blocked public-web domains
- decide writeback actions
### 5.6 Acceptance Criteria
- every research artifact contains normalized citations
- policy-blocked research requests fail clearly
- downstream workers receive research packets in a stable format
## 6. Handshake Contract Between the Three Roles
Fixed sequence:
1. planner creates task graph
2. prompt master emits prompt packages
3. librarian emits passes
4. researcher emits external evidence when required
5. worker execution starts only after the required packets exist
Required join key:
- `mission_id`
- `task_id`
Required invariant:
No worker should execute unless the following minimum packet exists:
- prompt package
- librarian pass
- policy decision for required tools
For research tasks, research artifacts are also required.
## 7. Operational Metrics
Track these role-level metrics:
- prompt package reuse rate
- average prompt package size by mission type
- librarian pass hit rate
- retrieval miss rate
- average research citation count
- reviewer rejection rate caused by weak prompting
- reviewer rejection rate caused by weak evidence
## 8. Ticket Breakdown
1. implement prompt template registry
2. implement exemplar registry
3. implement prompt package builder
4. implement source catalog
5. implement route-card builder
6. implement cache index
7. implement search provider normalization
8. implement browser provider normalization
9. integrate role outputs into worker launch contract
## 9. Bottom Line
Prompt master, librarian, and researcher are not helper utilities. They are the colony’s quality-control spine before execution. Their contracts must be explicit from the first implementation pass.

400
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Software Requirements Specification (SRS)_ Biomimetic Agentic Orchestration Layer.md Normal file
View File

@@ -0,0 +1,400 @@
# Software Requirements Specification (SRS)_ Biomimetic Agentic Orchestration Layer
**Date:** 2026-04-14
**Status:** Draft
**System Name:** `Project Velocity Colony Orchestration Layer`
## 1. Introduction
### 1.1 Purpose
This SRS defines the functional and non-functional requirements for a biomimetic agentic orchestration layer that integrates:
- a forked Open Multi Agent runtime as execution kernel
- Project Velocity Nemoclaw-derived policy and governance concepts
- Project Velocity root domain systems including Oracle, CRM, Catalyst, and Sentinel
### 1.2 Scope
The system accepts a high-level goal and coordinates a colony of specialized agents to produce a reviewed, auditable final output or domain writeback.
The system is intended to become the orchestration layer for Project Velocity domain workflows.
### 1.3 Intended Audience
This document is intended for:
- product and systems architects
- backend and orchestration engineers
- AI infrastructure engineers
- QA and validation engineers
- internal operators defining mission classes
## 2. References
Primary references used for this SRS:
- `.Agent Context/Tech/Tools Understanding.md`
- `.Agent Context/Tech/README-Open Multi-Agent.md`
- `.Agent Context/Sprint 1/nemoclaw_setup_truth.md`
- public Open Multi Agent repository `https://github.com/JackChen-me/open-multi-agent`
- current root Project Velocity code under `backend/oracle`, `backend/services`, `backend/api`, and Sentinel routes
## 3. System Overview
The system consists of:
- mission intake and normalization
- planner
- prompt master
- research subsystem
- librarian subsystem
- worker execution layer
- aggregation and review layer
- governance and policy layer
- domain adapters
The system must preserve the current Project Velocity ownership boundaries and must not replace the root backend.
## 4. Architectural Constraints
The system shall use a modular architecture.
The system shall not depend on leaked or provenance-unclear upstream code.
The system shall support mixed model providers.
The system shall keep policy enforcement external to worker prompts wherever possible.
The system shall preserve Oracle v1, FastAPI root ownership, and Sentinel ownership.
## 5. Functional Requirements
### FR-1 Mission Intake
The system shall accept a goal payload containing:
- source surface
- user prompt
- desired output type
- risk or sensitivity hint when available
### FR-2 Mission Normalization
The system shall normalize the goal into a canonical mission envelope with:
- mission identifier
- normalized objective
- risk classification
- time budget
- token budget
- requested outputs
### FR-3 Task Planning
The system shall decompose the mission into a directed acyclic graph of tasks.
Each task shall include:
- unique task identifier
- role type
- objective
- dependency list
- success criteria
- required capabilities
### FR-4 Prompt Package Generation
The system shall generate a prompt package per task using:
- prompt templates
- examples where configured
- task metadata
- mission constraints
- tool scope
### FR-5 External Research
The system shall support a research role capable of:
- executing search queries
- browsing webpages
- returning evidence packets with citation metadata
### FR-6 Internal Retrieval and Routing
The system shall support a librarian role capable of:
- identifying relevant internal data sources
- issuing scoped access passes
- returning cached previews and data pointers
### FR-7 Worker Provisioning
The system shall spawn worker tasks with:
- prompt package
- task objective
- tool permissions
- librarian pass
- research artifacts when applicable
### FR-8 Parallel Execution
The system shall execute independent tasks in parallel, subject to concurrency limits and policy constraints.
### FR-9 Aggregation
The system shall aggregate worker outputs into a structured aggregation packet.
### FR-10 Review
The system shall execute one bounded review pass over the aggregation packet before final delivery.
### FR-11 Final Delivery
The system shall deliver a final output packet to:
- the originating user-facing surface
- or a root domain subsystem
### FR-12 Governance
The system shall evaluate policy constraints before executing:
- sensitive tools
- external network calls
- filesystem writes where applicable
- domain writebacks
### FR-13 Audit Logging
The system shall emit mission-level and task-level audit records for:
- planning
- prompt package selection
- tool use
- model use
- writeback attempts
- review findings
### FR-14 Domain Adapters
The system shall expose adapter boundaries for:
- Oracle
- CRM
- Catalyst
- Sentinel
### FR-15 Artifact Persistence
The system shall persist mission and artifact records in a store controlled by Project Velocity rather than relying solely on in-memory framework state.
### FR-16 Prompt Registry
The system shall maintain a prompt registry with:
- prompt template identifiers
- versioning
- example sets
- role associations
### FR-17 Quorum and Approval Hooks
The system shall support configurable approval paths for high-risk missions, including:
- single human approval
- automatic review approval
- dual-agent review agreement
### FR-18 Failure Handling
The system shall handle:
- task failure
- task timeout
- policy rejection
- low-confidence evidence
- partial mission completion
and shall still return a structured final mission status.
## 6. External Interfaces
### 6.1 Mission Intake Interface
Recommended interface:
- internal HTTP or RPC endpoint from FastAPI to colony runtime
- JSON mission envelope request and response
### 6.2 Domain Adapter Interfaces
Each adapter should support:
- mission submission
- progress event subscription
- artifact retrieval
- writeback execution or rejection
### 6.3 Tool Interface
Tools shall expose:
- tool identifier
- input schema
- policy class
- timeout behavior
- result schema
### 6.4 Research Interface
Research outputs shall include:
- source URI
- title when available
- snippet
- summary
- retrieval timestamp
- confidence
### 6.5 Librarian Interface
Librarian passes shall include:
- allowed resource families
- recommended paths or queries
- cache pointers
- expiration metadata
## 7. Data Model
The following persistent entities are required:
- `missions`
- `mission_tasks`
- `prompt_packages`
- `research_artifacts`
- `library_passes`
- `worker_results`
- `aggregation_packets`
- `review_packets`
- `policy_decisions`
- `tool_invocations`
- `pheromone_signals`
Each entity shall include:
- a primary identifier
- timestamps
- provenance metadata
- mission linkage
## 8. Non-Functional Requirements
### 8.1 Reliability
The system shall degrade gracefully on partial task failure and shall produce a final mission state instead of hanging indefinitely.
### 8.2 Scalability
The system shall support configurable concurrency, multi-model execution, and task fan-out without requiring per-mission custom orchestration code.
### 8.3 Security
The system shall use external policy controls for sensitive tool and network operations.
### 8.4 Privacy
The system shall classify mission and data sensitivity and route high-sensitivity work to approved model paths only.
### 8.5 Observability
The system shall emit structured traces for:
- LLM calls
- tool calls
- task durations
- mission durations
- policy decisions
- writebacks
### 8.6 Fault Tolerance
The system should support resumable mission persistence in a later phase. Sprint 1 may use simpler persistence, but the schema and API boundaries shall not block future recovery features.
### 8.7 Licensing
The system shall use only code and artifacts with clear legal provenance.
Open Multi Agent may be forked and redistributed under MIT with attribution and license preservation.
Any direct vendor Nemoclaw code incorporation shall require explicit license verification. Until then, the design shall prefer Project Velocity-owned implementations inspired by Nemoclaw concepts.
## 9. Evaluation and Testing Strategy
The system shall be validated through:
- unit tests for planners, prompt packaging, policy checks, and adapters
- integration tests for full mission execution
- golden tests for prompt package generation
- replay tests for research and librarian flows
- failure injection tests
- performance tests on fan-out missions
- security tests for blocked tools and blocked egress
Acceptance should be measured per mission class, not only at framework level.
## 10. Migration and Fork Plan
### 10.1 Fork Strategy
The fork shall preserve upstream attribution and license files from Open Multi Agent.
The fork should remain isolated under a dedicated service or package boundary so upstream changes can still be reviewed selectively.
### 10.2 Compatibility Strategy
The fork should preserve the useful Open Multi Agent primitives:
- agent config model
- orchestrator model
- task queue model
- tool abstraction
- MCP connectivity
The fork should add Project Velocity-specific layers without tightly coupling them into upstream core classes unless necessary.
### 10.3 Contribution Strategy
The fork should define:
- contribution guidelines
- interface boundaries
- extension points for new ant roles
- prompt registry contribution rules
## 11. Open Questions
The following items require explicit decisions before implementation hardens:
- exact runtime boundary between TypeScript colony service and Python backend
- persistence backend for mission artifacts
- policy engine ownership split between Nemoclaw-derived logic and colony review logic
- how much of research browsing should be headless first versus GUI capable
- whether high-risk missions use human approval by default
## 12. Acceptance Criteria
The SRS is satisfied when an initial implementation can:
- accept a mission
- plan a task DAG
- generate prompt packages
- route internal and external context
- execute workers in parallel
- aggregate and review results
- expose audit traces
- enforce policy boundaries
- integrate with at least Oracle and CRM through explicit adapters

426
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Sourik Work Assignment_ Colony Runtime Entry Point.md Normal file
View File

@@ -0,0 +1,426 @@
# Sourik Work Assignment_ Colony Runtime Entry Point
**Date:** 2026-04-14
**Status:** Active assignment handoff
**Assignee:** Sourik
**Purpose:** Define the exact entry-point work expected from Sourik for the colony runtime, including current Project Velocity backend constraints, schema expectations, ingress and deployment assumptions, and collaboration rules that prevent merge conflicts or architectural drift.
## 1. Assignment Summary
Your assignment is to own the colony runtime entry point without breaking the current Project Velocity root.
This is not a greenfield rewrite. This is an integration-first build. The target is to establish the first working colony spine inside the existing Project Velocity architecture so that Oracle and CRM can become the first governed mission consumers.
Your work starts at the runtime entry boundary:
- TypeScript colony service bootstrap
- root-facing mission intake contract
- Python root to colony service connection shape
- file and module layout that matches the implementation blueprint
You are not being asked to redesign the system from scratch. The system design, runtime split, and domain order are already defined in this folder.
## 2. Primary Objective
Deliver the first real colony runtime entry path that can accept a mission from the Python root in a disciplined way and return status safely, without bypassing:
- current FastAPI authority
- current Project Velocity CRM routes and schema direction
- current Oracle planning spine
- current ingress and stable API direction
The first implementation should make this statement true:
“Project Velocity root can submit a structured mission to a TypeScript colony service over the stable backend path and retrieve its mission state without inventing a parallel backend center.”
## 3. What You Are Working Within
Assume the following current truths are already valid and must be respected.
### 3.1 Root Backend Truth
The current Python backend is already the canonical center.
Relevant existing root files and surfaces:
- [main.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\main.py)
- [routes_crm.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\api\routes_crm.py)
- [routes_oracle.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\api\routes_oracle.py)
- [router_v1.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\oracle\router_v1.py)
- [routes_catalyst.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\api\routes_catalyst.py)
- [sentinel.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\routers\sentinel.py)
Root rule:
Do not create a second Python backend center. Do not move domain ownership out of root.
### 3.2 Oracle Truth
Oracle already has a real planning and action spine.
Relevant files:
- [prompt_orchestrator.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\oracle\prompt_orchestrator.py)
- [persona_service.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\oracle\persona_service.py)
- [policy_service.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\oracle\policy_service.py)
- [data_access_gateway.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\oracle\data_access_gateway.py)
- [action_service.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\oracle\action_service.py)
Oracle rule:
Do not replace Oracle v1. The colony is an orchestration layer that should sit beside or beneath the current Oracle flow, not destroy it.
### 3.3 CRM Truth
CRM is no longer a mock-only subsystem. It already has operational routes and state surfaces.
Relevant files:
- [routes_crm.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\api\routes_crm.py)
- [useCrmBootstrap.ts](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\app\src\hooks\useCrmBootstrap.ts)
Root CRM behavior already includes:
- leads
- chat logs
- kanban
- demographics
- analytics scatter
- websocket sync
CRM rule:
Do not fork CRM schema logic into a separate service. Colony should consume CRM through root contracts and root persistence.
### 3.4 Nemoclaw and MCP Truth
These already exist as append layers in root:
- [nemoclaw_runtime.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\services\nemoclaw_runtime.py)
- [mcp_registry.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\services\mcp_registry.py)
- [nemoclaw_client.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\backend\services\nemoclaw_client.py)
Guard rule:
Use these as governance and tool abstractions. Do not invent a second unrelated guard layer.
## 4. What You Must Read First
Before implementation, read these in this order:
1. [Introduction for Sourik.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\Introduction for Sourik.md)
2. [Implementation Blueprint_ Repository and Runtime Mapping.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\Implementation Blueprint_ Repository and Runtime Mapping.md)
3. [TypeScript Colony Service Internal Module Spec.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\TypeScript Colony Service Internal Module Spec.md)
4. [Colony Database Schema and Root API Spec.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\Colony Database Schema and Root API Spec.md)
5. [Oracle and CRM Adapter Detailed Implementation Spec.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\Oracle and CRM Adapter Detailed Implementation Spec.md)
6. [Implementation Ticket Breakdown and Dependency Matrix.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\Implementation Ticket Breakdown and Dependency Matrix.md)
If you skip this reading order, you will likely recreate decisions that are already closed.
## 5. Your Concrete Scope
Your assignment scope is the colony runtime entry point.
That means the following.
### 5.1 Create the TypeScript Colony Service Skeleton
Target path:
- `services/colony-orchestrator/`
Start with the file tree already defined in:
- [Implementation Blueprint_ Repository and Runtime Mapping.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\Implementation Blueprint_ Repository and Runtime Mapping.md)
Minimum early files:
- `package.json`
- `tsconfig.json`
- `README.md`
- `src/index.ts`
- `src/app.ts`
- `src/config/env.ts`
- `src/contracts/*`
- `src/core/colony-runtime.ts`
- `src/persistence/mission-store.ts`
- `src/persistence/artifact-store.ts`
- `src/adapters/python-root-client.ts`
### 5.2 Establish the Internal Mission Intake Surface
Minimum internal colony service routes:
- `POST /missions`
- `GET /missions/:missionId/status`
- `GET /health`
These are internal service routes, not public internet routes.
### 5.3 Implement Contract-First Intake
The mission intake must validate a structured mission envelope and reject malformed input early.
Use:
- typed contracts
- Zod or equivalent runtime validation
- stable mission IDs
- stable stage reporting
### 5.4 Integrate Through Root, Not Around Root
Your service should assume the Python root is the caller and the persistence authority.
That means:
- root creates or accepts the mission envelope
- colony service executes the runtime pipeline
- root persists canonical mission state and artifacts
### 5.5 Prepare for Oracle and CRM as First Consumers
Do not spend your first pass on generic agent experimentation.
Build with these first mission classes in mind:
- `oracle_advisory`
- `crm_lead_intelligence`
These are the two mission classes the whole architecture is already optimized around.
## 6. What You Are Not Supposed to Do
Do not do the following:
- do not rewrite `backend/main.py` into a new architecture center
- do not replace Oracle prompt orchestration with an unrelated flow
- do not bypass CRM routes and talk directly to ad hoc tables
- do not create direct worker writebacks
- do not expose the colony service as the public external product API
- do not hardcode public IP addresses anywhere
- do not fork a second schema truth outside the current backend ownership model
If any proposed change pushes in those directions, stop and align before coding.
## 7. Current Backend and Schema Respect Rules
This project already has real work from multiple people. Your implementation must accommodate that.
### 7.1 Respect Existing Root API Ownership
If a domain already has a route module, integrate through it or beside it:
- `backend/api/routes_crm.py`
- `backend/api/routes_oracle.py`
- `backend/api/routes_catalyst.py`
Do not shadow them with parallel service endpoints that duplicate product authority.
### 7.2 Respect Current Schema Direction
The current colony persistence direction is already defined in:
- [Colony Database Schema and Root API Spec.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\Colony Database Schema and Root API Spec.md)
Phase 1 root tables are:
- `colony_missions`
- `colony_tasks`
- `colony_prompt_packages`
- `colony_research_artifacts`
- `colony_librarian_passes`
- `colony_worker_results`
- `colony_aggregation_packets`
- `colony_review_packets`
- `colony_policy_decisions`
- `colony_writeback_proposals`
- `colony_event_log`
- `colony_pheromone_signals`
Schema rule:
Do not improvise a different schema family unless there is a concrete, defensible reason and it is discussed first.
### 7.3 Respect Existing Action and Writeback Semantics
Writeback proposals must still flow through root approval and execution control.
Relevant existing root direction:
- Oracle actions already exist
- CRM writeback paths already exist
- colony should produce proposals, not side-effectful worker commits
## 8. Ingress and Stable API Information You Must Respect
This work must respect the current production networking direction described in:
- [Desineuron Stable Ingress Handoff.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Desineuron Stable Ingress Handoff.md)
### 8.1 Production Access Pattern
Current correct production pattern:
- public API hostname: `api.desineuron.in`
- public edge: ingress node with Elastic IP `98.87.120.120`
- backend target: current private IP of the EC2 instance tagged `DesineuronRole=velocity-backend`
- Linux box owns the route-sync timer pattern
### 8.2 Why This Matters to Your Work
Your work should assume:
- the frontend must not depend on ephemeral backend IPs
- internal colony service communication should also avoid stale hardcoded addressing
- all production-facing references should align with stable ingress and root-owned config
### 8.3 Repo Artifacts Already Supporting This Pattern
Relevant ingress/runtime files:
- [sync_velocity_route.py](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\infrastructure\desineuron_ingress\sync_velocity_route.py)
- [desineuron-velocity-route-sync.service](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\infrastructure\desineuron_ingress\desineuron-velocity-route-sync.service)
- [desineuron-velocity-route-sync.timer](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\infrastructure\desineuron_ingress\desineuron-velocity-route-sync.timer)
- [install_linux_velocity_route_sync.sh](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\infrastructure\desineuron_ingress\install_linux_velocity_route_sync.sh)
- [TEAM_HANDOFF_2026-04-08.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\infrastructure\desineuron_ingress\TEAM_HANDOFF_2026-04-08.md)
Relevant frontend/runtime files:
- [api.ts](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\app\src\lib\api.ts)
- [vite.config.ts](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\app\vite.config.ts)
- [remote_bootstrap_20260401.sh](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\remote_bootstrap_20260401.sh)
### 8.4 Networking Rules for Your Work
When you add configuration for the colony service:
- use environment-driven service URLs
- assume stable ingress for product-facing backend access
- do not pin anything to a one-off EC2 public IP
- assume backend app ports should remain private behind ingress security group control
## 9. Conflict Avoidance and Merge Discipline
This is important. Multiple people have already touched this codebase.
### 9.1 Respect Each Other’s Code
Do not revert unrelated changes.
If a file already has real work in it:
- read it first
- integrate with it
- extend it carefully
Do not “clean up” someone else’s work by replacing it wholesale unless there is explicit agreement.
### 9.2 Avoid Conflict Merges by Ownership Boundary
For your first pass, keep primary ownership focused on:
- `services/colony-orchestrator/*`
- colony-specific root bridge files once approved:
- `backend/api/routes_colony.py`
- `backend/services/colony_gateway.py`
- `backend/services/colony_repository.py`
- `backend/services/colony_policy_bridge.py`
- `backend/db/schema_colony.sql`
Try to avoid broad simultaneous edits to:
- `backend/api/routes_crm.py`
- `backend/api/routes_oracle.py`
- `backend/main.py`
until the colony service bootstrap and contracts are stable enough to integrate cleanly.
### 9.3 Make Small, Defensible Changes
Preferred merge behavior:
- contract layer first
- service bootstrap second
- root bridge third
- adapter entry hooks after that
That sequence keeps merges smaller and reduces cross-team breakage.
## 10. Recommended Build Order for You
Your best first execution order is:
1. scaffold `services/colony-orchestrator/`
2. add internal contracts and validation
3. add internal health route
4. add `python-root-client.ts`
5. define mission intake route and status route
6. align service outputs with root DB/API spec
7. only then start attaching Oracle and CRM adapter paths
Do not start with:
- external research providers
- UI hooks
- generic agent swarms
- direct domain writebacks
## 11. Deliverables Expected From You
Your first assignment is complete when the following are true:
### 11.1 Service Bootstrap Exists
The TypeScript colony service exists in repo with the correct top-level structure.
### 11.2 Mission Intake Works
The service can accept a validated mission envelope and return:
- mission ID
- status
- current stage
### 11.3 Root Integration Shape Is Clear
The bridge between Python root and colony service is explicit and compatible with the root API and persistence plan.
### 11.4 No Architectural Regression Occurs
Current Project Velocity routes and domain ownership remain intact.
### 11.5 No Networking Regression Occurs
Nothing in your work reintroduces ephemeral-IP assumptions or conflicts with the stable ingress/API direction.
## 12. Exact Docs That Define Your Assignment Boundary
You should treat these as your canonical references:
- [Introduction for Sourik.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\Introduction for Sourik.md)
- [Implementation Blueprint_ Repository and Runtime Mapping.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\Implementation Blueprint_ Repository and Runtime Mapping.md)
- [TypeScript Colony Service Internal Module Spec.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\TypeScript Colony Service Internal Module Spec.md)
- [Colony Database Schema and Root API Spec.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\Colony Database Schema and Root API Spec.md)
- [Oracle and CRM Adapter Detailed Implementation Spec.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\Oracle and CRM Adapter Detailed Implementation Spec.md)
- [Desineuron Stable Ingress Handoff.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Desineuron Stable Ingress Handoff.md)
- [Implementation Ticket Breakdown and Dependency Matrix.md](F:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\.Agent Context\Sprint 1\Biomimetic Agentic Orchestration Layer\Implementation Ticket Breakdown and Dependency Matrix.md)
## 13. Final Guidance
The right way to help this system is to strengthen the runtime spine, not to spread into every domain file at once.
If you build:
- the service skeleton
- the mission intake contract
- the root integration seam
- the non-conflicting file boundary
then the rest of the team can attach Oracle, CRM, governance, and review behavior without stepping on each other.
## 14. Bottom Line
Your assignment is to establish the colony runtime entry point in a way that respects the current Project Velocity backend, current schema direction, current stable ingress deployment pattern, and current multi-person code ownership reality.
Build the spine first. Keep the changes narrow. Respect root ownership. Respect existing routes and schema. Avoid hardcoded infrastructure assumptions. Avoid broad conflicting edits. If you do that, your work becomes the clean starting point the rest of the team can safely build on.

374
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/Sprint 1 Plan_ Biomimetic Agentic Orchestration Layer.md Normal file
View File

@@ -0,0 +1,374 @@
# Sprint 1 Plan_ Biomimetic Agentic Orchestration Layer
**Date:** 2026-04-14
**Status:** Proposed Sprint 1 execution plan
**Program:** Project Velocity Colony Orchestration Layer
## Purpose
This plan defines the first implementation sprint for the biomimetic colony orchestration layer. The sprint goal is not full autonomy. The sprint goal is to create the minimum production-grade orchestration spine that can be integrated into Project Velocity without destabilizing the current root.
## Sprint 1 Goals
Sprint 1 must validate four architectural truths:
1. A forked Open Multi Agent kernel can be adapted into a Project Velocity orchestration service.
2. Nemoclaw-derived governance can sit outside worker prompts and still allow useful execution.
3. Oracle and CRM can use the colony runtime through explicit adapter boundaries.
4. The biomimicry role model can be implemented as a disciplined architecture rather than only a metaphor.
## Sprint 1 Scope
### In Scope
- fork plan and package structure
- mission envelope schema
- task graph schema
- prompt package schema
- first colony runtime skeleton
- planner role
- prompt master role
- librarian role
- researcher role stub with provider interface
- worker execution
- aggregation and review roles
- policy gateway abstraction
- Oracle adapter
- CRM adapter
- mission and artifact persistence
- observability and audit baseline
### Out of Scope
- autonomous public posting
- full Sentinel autonomy
- long-running checkpoint resume
- multi-tenant public marketplace
- generalized self-modifying colony behavior
## Sprint 1 Deliverables
### Deliverable 1: Architecture Package
This document set plus the PRD, SRS, and first-principles design documents.
### Deliverable 2: Repo Structure Proposal
Recommended structure:
```text
services/
colony-orchestrator/
src/
core/
planner/
prompt-master/
librarian/
researcher/
workers/
aggregator/
reviewer/
policy/
adapters/
schemas/
telemetry/
backend/
services/
colony_gateway.py
api/
routes_colony.py
```
### Deliverable 3: Mission and Artifact Schemas
Versioned JSON contracts for:
- mission envelope
- task graph
- prompt package
- librarian pass
- research artifact
- worker result
- aggregation packet
- review packet
### Deliverable 4: Oracle Adapter MVP
Ability for Oracle to submit one mission class:
- structured advisory mission
- plan
- retrieve
- execute
- aggregate
- return answer and optional writeback proposal
### Deliverable 5: CRM Adapter MVP
Ability for CRM to submit one mission class:
- lead intelligence mission
- internal context retrieval
- optional external research
- worker synthesis
- review
- structured output
### Deliverable 6: Governance MVP
Policy checks for:
- allowed tools
- allowed network families
- allowed writebacks
- model routing class
### Deliverable 7: Observability MVP
Traces and audit events for:
- mission created
- tasks planned
- prompts generated
- tools called
- policy decisions
- final output approved
## Milestones
### Milestone 1: Fork and Runtime Skeleton
Expected outcome:
- create fork boundary
- isolate upstream kernel primitives
- define service packaging
### Milestone 2: Artifact Contracts
Expected outcome:
- all core mission artifacts versioned
- planners and adapters code against stable schemas
### Milestone 3: Planner, Prompt Master, Librarian
Expected outcome:
- decomposition flow exists
- prompt packages exist
- librarian passes exist
### Milestone 4: Oracle and CRM Integration
Expected outcome:
- at least two real mission classes run through the colony
### Milestone 5: Governance and Review Closure
Expected outcome:
- review pass and policy gating both function
## Workstreams
### Workstream A: Fork and Service Boundary
Tasks:
- fork Open Multi Agent into a dedicated service boundary
- preserve MIT license and attribution
- define which upstream files remain core and which become Velocity-specific extensions
Acceptance criteria:
- service starts independently
- upstream kernel compiles
- Project Velocity extension folders exist cleanly
### Workstream B: Artifact and Schema Layer
Tasks:
- define schema files
- version artifact envelopes
- add validation utilities
Acceptance criteria:
- invalid mission and artifact payloads are rejected deterministically
- every pipeline stage reads and writes explicit typed artifacts
### Workstream C: Colony Roles
Tasks:
- implement planner role
- implement prompt master role
- implement librarian role
- implement aggregator role
- implement reviewer role
Acceptance criteria:
- mission can move through all five stages with traceable artifacts
### Workstream D: Research and Tooling
Tasks:
- define external search provider interface
- define browser provider interface
- define citation artifact shape
Acceptance criteria:
- a research task can return structured evidence with provenance
### Workstream E: Governance
Tasks:
- define policy decision contract
- classify tool families
- classify model routing tiers
- validate writeback actions
Acceptance criteria:
- a blocked tool or blocked writeback is rejected before execution and logged
### Workstream F: Velocity Adapters
Tasks:
- Oracle adapter MVP
- CRM adapter MVP
- request and response mapping
- artifact persistence linkage
Acceptance criteria:
- Oracle and CRM can each run one real mission class through the colony
## Suggested Roles
Suggested engineering ownership:
- orchestration kernel engineer
- policy and safety engineer
- backend integration engineer
- prompt systems engineer
- QA and observability engineer
These can be combined if team size is small, but the role boundaries should remain explicit.
## Timeline
### Week 1
- finalize fork boundary
- define schema layer
- stand up runtime skeleton
- implement planner and prompt master contracts
### Week 2
- implement librarian and research interfaces
- implement aggregator and reviewer
- add audit traces
### Week 3
- integrate Oracle mission
- integrate CRM mission
- add governance gates
### Week 4
- run end-to-end test scenarios
- harden observability
- finalize Sprint 1 review package
## Acceptance Criteria by Sprint End
Sprint 1 is successful if the team can demonstrate:
- one Oracle mission running through the colony end to end
- one CRM mission running through the colony end to end
- task DAG decomposition with explicit artifact traces
- prompt packages created by a dedicated prompt master stage
- librarian passes routing internal knowledge
- at least one external research step with citations
- aggregation and one-pass review
- policy rejection for at least one disallowed action
## Risks and Mitigation
### Risk 1: Overbuilding the metaphor
Mitigation:
Treat each ant role as a software responsibility with a schema and interface, not as narrative decoration.
### Risk 2: Open Multi Agent becomes too invasive
Mitigation:
Keep the fork behind a dedicated service boundary. Avoid scattering upstream assumptions through the Python root.
### Risk 3: Governance slows the system too much
Mitigation:
Apply risk-tiered policy. Not every mission needs the heaviest controls.
### Risk 4: Prompt Master becomes an opaque black box
Mitigation:
Version templates, examples, and outputs. Evaluate prompt packages with golden tests.
### Risk 5: Librarian becomes a data leak vector
Mitigation:
Use explicit passes with resource family scopes and expirations.
## Architectural Decisions to Validate in Sprint 1
The sprint should explicitly validate:
- TypeScript colony runtime plus Python root is a workable split
- mission persistence is needed immediately, not later
- one-pass review is enough for normal missions
- librarian-pass routing materially improves quality and cost
- prompt-master separation improves worker quality
## Repository Placement
This Sprint 1 artifact set belongs under:
- `.Agent Context/Sprint 1/`
Implementation work should likely land under:
- `services/colony-orchestrator/`
- `backend/services/`
- `backend/api/`
## Bottom Line
Sprint 1 should not try to build the whole ant colony civilization.
Sprint 1 should prove the minimum viable colony:
- plan
- route
- execute
- review
- govern
- integrate with Oracle and CRM
If those six things work cleanly, the rest of the ecosystem can grow on top of a stable spine.

744
.Agent Context/Sprint 1/Biomimetic Agentic Orchestration Layer/TypeScript Colony Service Internal Module Spec.md Normal file
View File

@@ -0,0 +1,744 @@
# TypeScript Colony Service Internal Module Spec
**Date:** 2026-04-14
**Status:** Draft implementation artifact
**Purpose:** Define the internal module architecture, responsibilities, interfaces, sequencing, and operational behavior of the TypeScript colony service.
## 1. Purpose
This document describes how the `services/colony-orchestrator/` service should be structured internally so the team can implement it as a coherent runtime rather than a loose collection of agents.
## 2. Service Boundary
The TypeScript colony service is an internal orchestration service. It is not a second product shell and it is not the canonical persistence owner.
It receives mission requests from the Python root, executes the colony workflow, and returns reviewed outputs plus structured artifacts.
## 3. Top-Level Service Tree
Required internal structure:
```text
services/colony-orchestrator/
src/
index.ts
app.ts
config/
core/
contracts/
planner/
prompt-master/
librarian/
researcher/
workers/
aggregation/
review/
policy/
adapters/
persistence/
telemetry/
```
## 4. Core Runtime Modules
### 4.1 `src/index.ts`
Purpose:
Boot entry. Starts HTTP server and wires service configuration.
### 4.2 `src/app.ts`
Purpose:
Creates the HTTP application and mounts internal routes such as:
- `POST /missions`
- `GET /missions/:missionId/status`
- `GET /health`
### 4.3 `src/core/colony-runtime.ts`
Purpose:
Primary runtime facade. Accepts a mission request and orchestrates the full pipeline.
Responsibilities:
- invoke planner
- invoke prompt master
- invoke librarian
- invoke researcher if required
- spawn workers
- invoke aggregator
- invoke reviewer
- return final packet
### 4.4 `src/core/mission-runner.ts`
Purpose:
Mission-local runtime controller.
Responsibilities:
- mission step transitions
- task state tracking
- time budget enforcement
- token budget enforcement
- abort propagation
### 4.5 `src/core/task-graph.ts`
Purpose:
Own the executable DAG model.
Responsibilities:
- load planned tasks
- resolve dependencies
- release ready tasks
- track terminal states
### 4.6 `src/core/scheduler.ts`
Purpose:
Assign ready tasks to worker slots based on:
- role type
- concurrency
- policy constraints
### 4.7 `src/core/worker-factory.ts`
Purpose:
Construct worker execution units from:
- prompt package
- tool scope
- model route
- librarian pass
- research artifacts
## 5. Contract Modules
All contracts should live under `src/contracts/`.
Required files:
- `mission.ts`
- `task.ts`
- `prompt-package.ts`
- `research-artifact.ts`
- `librarian-pass.ts`
- `worker-result.ts`
- `aggregation-packet.ts`
- `review-packet.ts`
- `policy-decision.ts`
- `writeback-proposal.ts`
Each contract file should export:
- TypeScript types
- Zod schemas
- version constants
- validation helpers
## 6. Planner Modules
### 6.1 `src/planner/mission-normalizer.ts`
Responsibilities:
- normalize raw mission input
- infer mission subtype if needed
- attach defaults for budgets and risk class
### 6.2 `src/planner/planner-agent.ts`
Responsibilities:
- produce task DAG
- assign role types
- define dependencies
- define success criteria
### 6.3 `src/planner/decomposition-policy.ts`
Responsibilities:
- keep mission decomposition within approved complexity
- prevent unnecessary over-decomposition
- enforce mission-type-specific planning rules
## 7. Prompt Master Modules
### 7.1 `src/prompt-master/prompt-master-agent.ts`
Responsibilities:
- produce role-specific prompt packages
- choose templates
- choose exemplars
- bind constraints and tool scope
### 7.2 `src/prompt-master/template-registry.ts`
Responsibilities:
- resolve prompt template IDs
- load template metadata
- version templates
### 7.3 `src/prompt-master/exemplar-registry.ts`
Responsibilities:
- resolve prompt exemplars
- return relevant examples by mission type and role type
### 7.4 `src/prompt-master/prompt-package-builder.ts`
Responsibilities:
- assemble final package object
- bind constraints
- bind route instructions
- bind model route
## 8. Librarian Modules
### 8.1 `src/librarian/librarian-agent.ts`
Responsibilities:
- inspect mission type and tasks
- determine relevant internal sources
- create scoped passes
### 8.2 `src/librarian/source-catalog.ts`
Purpose:
Map domain resource families to root-access semantics.
Initial families:
- `crm.leads`
- `crm.chat_logs`
- `oracle.canvas`
- `oracle.actions`
- `catalyst.campaigns`
- `sentinel.events`
### 8.3 `src/librarian/route-card-builder.ts`
Purpose:
Create readable structured route cards for workers.
### 8.4 `src/librarian/cache-index.ts`
Purpose:
Return cached previews and compact route hints.
## 9. Researcher Modules
### 9.1 `src/researcher/researcher-agent.ts`
Responsibilities:
- execute external research task
- call provider abstractions
- normalize results into research artifacts
### 9.2 `src/researcher/search-provider.ts`
Purpose:
Abstract providers such as:
- Brave
- DuckDuckGo
- future commercial search tools
### 9.3 `src/researcher/browser-provider.ts`
Purpose:
Abstract:
- headless browser
- GUI browser capable provider
Sprint 1 should start with interface only or headless-first implementation.
### 9.4 `src/researcher/citation-normalizer.ts`
Purpose:
Convert raw result fragments into citation-ready structures.
## 10. Worker Modules
### 10.1 `src/workers/worker-agent.ts`
Purpose:
Execute worker tasks.
Responsibilities:
- invoke LLM with prompt package
- attach librarian pass
- attach research evidence
- call allowed tools
- emit worker result
### 10.2 `src/workers/worker-context-builder.ts`
Purpose:
Build bounded worker context window from:
- mission summary
- task objective
- prompt package
- librarian pass
- research artifacts
### 10.3 `src/workers/result-normalizer.ts`
Purpose:
Normalize raw LLM output into:
- output text
- structured output
- citations
- confidence
## 11. Aggregation Modules
### 11.1 `src/aggregation/aggregator-agent.ts`
Purpose:
Build one synthesized draft from worker results.
### 11.2 `src/aggregation/evidence-matrix.ts`
Purpose:
Map claims to supporting artifacts.
### 11.3 `src/aggregation/contradiction-detector.ts`
Purpose:
Flag output contradictions before review.
## 12. Review Modules
### 12.1 `src/review/reviewer-agent.ts`
Purpose:
Perform one bounded review pass over the aggregation packet.
### 12.2 `src/review/review-policy.ts`
Purpose:
Define what constitutes approval, approval with edits, or rejection.
### 12.3 `src/review/release-gate.ts`
Purpose:
Convert review outcome into:
- final output
- pending approval requirement
- rejected mission state
## 13. Policy Modules
### 13.1 `src/policy/risk-classifier.ts`
Purpose:
Classify mission risk based on:
- mission type
- requested tools
- external search use
- writeback potential
### 13.2 `src/policy/tool-policy.ts`
Purpose:
Filter allowed tools per task.
### 13.3 `src/policy/model-routing-policy.ts`
Purpose:
Select model tier per task.
### 13.4 `src/policy/writeback-policy.ts`
Purpose:
Ensure workers only emit proposals, not direct writes.
## 14. Adapter Modules
### 14.1 `src/adapters/python-root-client.ts`
Purpose:
Single client module for communication back to root-owned endpoints if needed.
### 14.2 `src/adapters/oracle-adapter.ts`
Purpose:
Map Oracle missions into colony task structure.
### 14.3 `src/adapters/crm-adapter.ts`
Purpose:
Map CRM missions into colony task structure.
### 14.4 `src/adapters/catalyst-adapter.ts`
Purpose:
Map Catalyst strategy missions.
### 14.5 `src/adapters/sentinel-adapter.ts`
Purpose:
Attach Sentinel evidence to eligible missions.
## 15. Persistence Modules
Even though root is canonical, the service still needs a persistence interface layer to push state outward cleanly.
### 15.1 `src/persistence/mission-store.ts`
Responsibilities:
- create mission in root
- update mission state in root
### 15.2 `src/persistence/artifact-store.ts`
Responsibilities:
- persist artifacts to root
- fetch artifact snapshots where needed
### 15.3 `src/persistence/pheromone-store.ts`
Responsibilities:
- emit compact signal writes
- read reinforcement hints for planning
## 16. Telemetry Modules
### 16.1 `src/telemetry/audit-log.ts`
Purpose:
Emit audit events for:
- planning
- tool use
- policy decisions
- review outcomes
### 16.2 `src/telemetry/mission-trace.ts`
Purpose:
Generate trace IDs and mission-level span tracking.
### 16.3 `src/telemetry/metrics.ts`
Purpose:
Emit latency, token, and failure metrics.
## 17. Required Internal Service Routes
The colony service should minimally expose:
- `POST /missions`
- `GET /missions/:missionId/status`
- `GET /health`
It should not expose every artifact directly to the outside world. Artifact inspection belongs primarily in the Python root.
## 18. Internal Mission Execution Sequence
The service should run this fixed sequence:
1. validate mission envelope
2. classify risk
3. plan task graph
4. persist task graph
5. build prompt packages
6. persist prompt packages
7. create librarian passes
8. create research artifacts if required
9. execute workers
10. persist worker results
11. aggregate
12. review
13. emit final packet
## 19. Failure Handling Rules
The service must:
- fail fast on invalid contracts
- mark mission as failed on unrecoverable planner or reviewer error
- preserve partial artifacts when possible
- return structured failure reason to root
## 20. Acceptance Criteria
This module architecture is sufficient when:
- every folder has a clear responsibility
- mission flow can be implemented without circular ownership
- Oracle and CRM adapters fit naturally into the service
- root persistence remains canonical
- governance hooks exist before autonomous behavior expands
## 21. Required Internal Interfaces
The module tree is not enough by itself. The implementation must define stable service interfaces between layers.
### 21.1 `ColonyRuntime`
Required interface:
```ts
interface ColonyRuntime {
runMission(input: MissionEnvelope): Promise<ReviewedMissionResult>;
getMissionStatus(missionId: string): Promise<MissionRuntimeStatus>;
health(): Promise<ColonyHealthStatus>;
}
```
### 21.2 `MissionStore`
Required interface:
```ts
interface MissionStore {
createMission(mission: MissionEnvelope): Promise<void>;
updateMissionStatus(missionId: string, status: MissionStatus, reviewStatus?: ReviewStatus): Promise<void>;
appendEvent(missionId: string, eventType: string, payload: Record<string, unknown>): Promise<void>;
}
```
### 21.3 `ArtifactStore`
Required interface:
```ts
interface ArtifactStore {
saveTaskGraph(missionId: string, tasks: MissionTask[]): Promise<void>;
savePromptPackages(missionId: string, packages: PromptPackage[]): Promise<void>;
saveResearchArtifacts(missionId: string, artifacts: ResearchArtifact[]): Promise<void>;
saveLibrarianPasses(missionId: string, passes: LibrarianPass[]): Promise<void>;
saveWorkerResults(missionId: string, results: WorkerResult[]): Promise<void>;
saveAggregationPacket(missionId: string, packet: AggregationPacket): Promise<void>;
saveReviewPacket(missionId: string, packet: ReviewPacket): Promise<void>;
savePolicyDecision(missionId: string, decision: PolicyDecision): Promise<void>;
saveWritebackProposal(missionId: string, proposal: WritebackProposal): Promise<void>;
}
```
### 21.4 `PythonRootClient`
Required interface:
```ts
interface PythonRootClient {
createMission(mission: MissionEnvelope): Promise<void>;
updateMissionStatus(input: RootMissionStatusUpdate): Promise<void>;
persistArtifactBatch(input: RootArtifactBatch): Promise<void>;
publishApprovalRequired(input: ApprovalRequiredSignal): Promise<void>;
resolveHealth(): Promise<RootHealthSnapshot>;
}
```
## 22. Module Dependency Rules
The service must not become a circular tangle.
Allowed dependency direction:
- `config` may be imported by any module
- `contracts` may be imported by any module
- `core` may import every functional module
- `planner`, `prompt-master`, `librarian`, `researcher`, `workers`, `aggregation`, `review`, `policy`, and `adapters` may import `contracts`, `config`, and selected shared utilities only
- `telemetry` and `persistence` may be imported by `core` and domain modules, but they must not import business modules
Forbidden dependency direction:
- `planner` must not import `review`
- `review` must not import `planner`
- `workers` must not call `python-root-client` directly
- `adapters` must not own scheduler logic
- `telemetry` must not become a hidden business rules layer
## 23. Mission State and Runtime Control
The TypeScript service needs its own runtime state view even though root remains canonical.
Runtime control responsibilities:
- `mission-runner.ts` owns in-process mission execution state
- `task-graph.ts` owns dependency resolution
- `scheduler.ts` owns ready queue release and concurrency slots
- `worker-factory.ts` owns worker construction but not task planning
Required runtime safeguards:
- per-mission timeout
- per-task timeout
- maximum concurrent workers per mission
- maximum retries per worker task
- cancellation propagation when mission is rejected or root disconnects permanently
## 24. Provider Abstraction Rules
The service should not bake provider-specific logic into role modules.
Required provider seams:
- `model-provider.ts` abstraction beneath `workers` and `prompt-master`
- `search-provider.ts` abstraction beneath `researcher`
- `browser-provider.ts` abstraction for GUI and headless research execution
- `cache-provider.ts` abstraction for short-lived artifact caching if added later
Provider rule:
All provider implementations must return normalized contract objects, not raw vendor payloads.
## 25. Prompt Master, Librarian, and Researcher Handshake
The most failure-prone part of the colony is the handoff between prompt quality, internal routing, and external research.
Required sequence:
1. planner emits task graph
2. prompt master emits prompt packages
3. librarian emits passes
4. researcher emits external evidence when required
5. worker execution starts only after the required packets exist
Rule:
No worker should be launched with only a raw prompt and no scoped context packet.
## 26. Error Taxonomy
The service should emit typed failures, not generic exceptions.
Minimum runtime error classes:
- `ContractValidationError`
- `MissionAdmissionError`
- `PlanningError`
- `PromptPackagingError`
- `LibrarianRoutingError`
- `ResearchProviderError`
- `WorkerExecutionError`
- `AggregationError`
- `ReviewError`
- `PolicyViolationError`
- `RootPersistenceError`
Each must include:
- `mission_id`
- `task_id` when applicable
- `stage`
- `retryable: true|false`
- `safe_message`
- `internal_details`
## 27. Testing Matrix
Minimum service-side test suites:
- contract validation unit tests
- planner decomposition tests by mission type
- prompt package generation tests
- librarian pass generation tests
- researcher provider normalization tests
- worker result normalization tests
- aggregation contradiction tests
- reviewer approval gate tests
- policy block tests
- root client retry and failure tests
- full mission integration tests for Oracle and CRM
## 28. Implementation Order Inside the Service
The TypeScript service should be built in this order:
1. `contracts`
2. `config`
3. `persistence`
4. `core` shell and `app.ts`
5. `planner`
6. `prompt-master`
7. `librarian`
8. `researcher`
9. `workers`
10. `aggregation`
11. `review`
12. `policy`
13. `adapters`
14. `telemetry`
Do not implement domain adapters before the core execution path exists.
## 29. Ticket-Grade Deliverables
Minimum service implementation tickets:
1. bootstrap `services/colony-orchestrator`
2. add Zod-backed contract layer
3. add `python-root-client` and persistence facades
4. implement planner and task graph
5. implement prompt master registries
6. implement librarian and route-card builder
7. implement researcher provider abstraction
8. implement worker runner and scheduler
9. implement aggregator and contradiction detector
10. implement reviewer and release gate
11. implement policy layer
12. implement Oracle adapter
13. implement CRM adapter
14. add telemetry and mission traces
## 30. Bottom Line
This service should be built as a disciplined internal runtime with clear module boundaries, typed interfaces, and explicit failure behavior, not as a loose swarm of scripts. The module map above is the minimum structure needed to keep the colony understandable as it grows.