sagnik/Project_Velocity
2
1
Fork 2
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Overlay the mathematical Sun Path over the live camera feed or 3D model view (#8)

Browse Source 
#7 Task completed.

Co-authored-by: Sayan Datta <sayan@Sayans-MacBook-Air.local>
Reviewed-on: #8
This commit was merged in pull request #8.
This commit is contained in:
sayan
2026-03-21 17:01:06 +05:30
parent 6c98affe53
commit 023ba48da2
16 changed files with 3344 additions and 2939 deletions
Download Patch File Download Diff File Expand all files Collapse all files

709
.Agent Context/Bibels/dreamweaver_ios_integration_guide.md
View File

@@ -1,512 +1,197 @@
# Dream Weaver — iOS ↔ Backend Integration Guide
**Version:** 2.0 | **Updated:** 2026-03-09 | **Server:** `54.172.172.2` | **Port:** `8080`
> This document is for **Sayan** (iOS / Swift) and **Sourik** (backend review).
> It describes exactly how the iPad app should talk to the Dream Weaver AI and how keywords from a user tap become a full ComfyUI generation.
---
## 1. Architecture Overview
```
┌────────────────────┐ HTTP/S ┌──────────────────────────────┐
│ │ ─── POST image ───► │ Dream Weaver Gateway │
│ iPad App (Swift) │ │ FastAPI port 8080 │
│ │ ◄── PNG result ─── │ dw_gateway.py │
└────────────────────┘ └─────────────┬────────────────┘
│ internal HTTP
▼
┌─────────────────────────┐
│ ComfyUI Engine │
│ port 8188 │
│ RealVisXL V5.0 Ltng │
│ 4× NVIDIA L4 (96 GB) │
└─────────────────────────┘
```
**Key rule:** The iPad app **never** talks to ComfyUI directly. It only talks to the Gateway on `:8080`.
---
## 2. How Keywords Become Prompts
### 2.1 The Prompt Expansion System
Each interior style in the app is backed by a **prompt template** in `comfy_engine/prompts/`. When the user taps a style card (or types keywords), those keywords get **merged into the template** to build the final ComfyUI prompt injected into node `9` (positive CLIPTextEncode).
**Prompt template structure** (from [scandinavian_minimalist.txt](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/prompts/scandinavian_minimalist.txt)):
```
POSITIVE PROMPT:
scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles,
abundant natural light streaming through large windows, clean white walls, ...
Style Weight: <lora:Interior_Style_Scandi:0.8>
NEGATIVE PROMPT:
(worst quality, low quality, illustration, 3d render...), heavy ornamentation,...
TECHNICAL PARAMETERS:
- Denoising Strength: 0.70
- CFG Scale: 7.0
- Recommended Sampler: dpmpp_2m_karras
- Steps: 30-40
```
### 2.2 Keyword Expansion Flow
```
User taps: ["marble", "gold", "luxury"]
+
Style selected: "art_deco"
│
▼
Backend expands:
base_prompt = art_deco_luxe.txt (POSITIVE PROMPT section)
user_keywords_str = "marble, gold, luxury"
final_prompt = base_prompt + ", " + user_keywords_str
│
▼
Injected into ComfyUI workflow:
node "9" → CLIPTextEncode → text: [final_prompt]
node "10" → CLIPTextEncode → text: [negative_prompt from template]
node "1" → LoadImage → image: [uploaded filename]
node "13" → KSampler → denoise: 0.72, cfg: 7.5, steps: 35
```
### 2.3 Available Styles and Their Keywords (for the Style Picker UI)
| Style ID | Display Name | Suggested Keywords Palette |
|---|---|---|
| `scandinavian` | Scandinavian Minimalist | oak, linen, white, hygge, cozy, birch, natural |
| `art_deco` | Art Deco Luxe | gold, marble, velvet, geometric, 1920s, brass, crystal |
| `biophilic` | Biophilic Organic | green wall, stone, rattan, terracotta, botanical, moss |
| `cyberpunk` | Cyberpunk Neon | neon, chrome, holographic, dark, LED, futuristic, blade runner |
| `japandi` | Japandi Fusion | wabi-sabi, ash wood, ceramic, zen, minimal, shoji, serene |
---
## 3. API Reference — What Sayan Needs to Call
### BASE URL
```
http://54.172.172.2:8080
```
> [!NOTE]
> Once we attach an Elastic IP or domain, swap this in `AppConfig.swift`.
---
### 3.1 `GET /health` — Liveness Check
Call this on app launch to confirm the server is up before showing the Generate button.
**Request:**
```http
GET http://54.172.172.2:8080/health
```
**Response:**
```json
{
"status": "ok",
"comfyui": true,
"gpu": "4x NVIDIA L4 (96GB VRAM)",
"model": "RealVisXL V5.0 Lightning"
}
```
**Swift:**
```swift
func checkServerHealth() async throws -> Bool {
let url = URL(string: "\(AppConfig.baseURL)/health")!
let (data, _) = try await URLSession.shared.data(from: url)
let json = try JSONDecoder().decode(HealthResponse.self, from: data)
return json.status == "ok"
}
```
---
### 3.2 `POST /dream-weaver` — Submit Generation Job (Async)
Use this for the main generation flow. Returns a `job_id` immediately; poll for result.
**Request:** `multipart/form-data`
| Field | Type | Required | Description |
|---|---|---|---|
| [image](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/a100_deployment_executor.py#306-322) | File (JPEG/PNG) | ✅ | The room photo from camera or library |
| [style](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/dreamweaver_batch_processor.py#394-414) | String | ✅ | One of: `scandinavian`, `art_deco`, `biophilic`, `cyberpunk`, `japandi` |
| `keywords` | String | ➕ | Comma-separated user keywords e.g. `"gold, marble, luxury"` |
| `denoise` | Float | ➕ | 0.5–0.85 (default `0.72`). Higher = more creative |
**Response:**
```json
{
"job_id": "a1b2c3d4-...",
"status": "processing",
"poll_url": "/dream-weaver/status/a1b2c3d4-...",
"result_url": "/dream-weaver/result/a1b2c3d4-..."
}
```
**Swift example:**
```swift
func submitGeneration(image: UIImage, style: String, keywords: [String]) async throws -> GenerationJob {
let url = URL(string: "\(AppConfig.baseURL)/dream-weaver")!
var request = URLRequest(url: url)
request.httpMethod = "POST"
let boundary = UUID().uuidString
request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")
var body = Data()
// Image field
let imageData = image.jpegData(compressionQuality: 0.85)!
body.append("--\(boundary)\r\n".data(using: .utf8)!)
body.append("Content-Disposition: form-data; name=\"image\"; filename=\"room.jpg\"\r\n".data(using: .utf8)!)
body.append("Content-Type: image/jpeg\r\n\r\n".data(using: .utf8)!)
body.append(imageData)
body.append("\r\n".data(using: .utf8)!)
// Style field
body.append("--\(boundary)\r\n".data(using: .utf8)!)
body.append("Content-Disposition: form-data; name=\"style\"\r\n\r\n".data(using: .utf8)!)
body.append("\(style)\r\n".data(using: .utf8)!)
// Keywords field (user tapped keywords)
if !keywords.isEmpty {
let kwString = keywords.joined(separator: ", ")
body.append("--\(boundary)\r\n".data(using: .utf8)!)
body.append("Content-Disposition: form-data; name=\"keywords\"\r\n\r\n".data(using: .utf8)!)
body.append("\(kwString)\r\n".data(using: .utf8)!)
}
body.append("--\(boundary)--\r\n".data(using: .utf8)!)
request.httpBody = body
let (data, _) = try await URLSession.shared.data(for: request)
return try JSONDecoder().decode(GenerationJob.self, from: data)
}
```
---
### 3.3 `GET /dream-weaver/status/{job_id}` — Poll Job Status
Poll every **2 seconds** until `ready == true`.
**Response while processing:**
```json
{ "status": "processing", "ready": false, "style": "art_deco" }
```
**Response when done:**
```json
{
"status": "done",
"ready": true,
"result_url": "/dream-weaver/result/a1b2c3d4-...",
"style": "art_deco"
}
```
**Swift polling loop:**
```swift
func pollForResult(jobId: String) async throws -> URL {
let statusURL = URL(string: "\(AppConfig.baseURL)/dream-weaver/status/\(jobId)")!
for _ in 0..<150 { // max 5 min (150 × 2s)
try await Task.sleep(nanoseconds: 2_000_000_000) // 2 seconds
let (data, _) = try await URLSession.shared.data(from: statusURL)
let status = try JSONDecoder().decode(JobStatus.self, from: data)
if status.ready {
return URL(string: "\(AppConfig.baseURL)/dream-weaver/result/\(jobId)")!
}
if status.status == "error" {
throw DreamWeaverError.generationFailed(status.error ?? "Unknown")
}
}
throw DreamWeaverError.timeout
}
```
---
### 3.4 `GET /dream-weaver/result/{job_id}` — Download Result Image
Returns a PNG image stream directly. Download and display to user.
```swift
func downloadResult(resultURL: URL) async throws -> UIImage {
let (data, _) = try await URLSession.shared.data(from: resultURL)
guard let image = UIImage(data: data) else {
throw DreamWeaverError.invalidImageData
}
return image
}
```
---
### 3.5 `POST /dream-weaver/sync` — One-Shot Blocking Call
For **testing only** or fast network connections. Waits up to 120 seconds and returns the image directly.
```swift
// Same multipart form as /dream-weaver but returns PNG bytes directly
// Not recommended for production — use async flow above
```
---
## 4. Keyword → Prompt Expansion (Backend Change Required)
> [!IMPORTANT]
> The current [dw_gateway.py](file:///C:/Windows/Temp/dw_gateway.py) does NOT yet accept `keywords` from the app.
> Sagnik needs to add keyword expansion to the gateway. Here is the exact code change:
**In [dw_gateway.py](file:///C:/Windows/Temp/dw_gateway.py), update [build_workflow()](file:///C:/Windows/Temp/dw_gateway.py#33-65):**
```python
# Prompt library — maps style ID to (positive_base, negative, cfg, denoise, steps)
STYLE_LIBRARY = {
"scandinavian": {
"pos": "scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles, abundant natural light, clean white walls, simple functional furniture, cozy hygge atmosphere, architectural photography, 8k resolution, photorealistic",
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, extra windows, unrealistic lighting, structural changes",
"cfg": 7.0, "denoise": 0.70, "steps": 30,
},
"art_deco": {
"pos": "art deco luxury interior design, geometric chevron patterns, gold brass accents, rich velvet upholstery emerald and sapphire, sunburst mirrors, polished marble flooring, crystal chandeliers, 1920s glamour, 8k resolution, photorealistic",
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, rustic, minimalism, cheap materials",
"cfg": 7.5, "denoise": 0.72, "steps": 30,
},
"biophilic": {
"pos": "biophilic organic interior design, living green walls with ferns and moss, natural stone accent walls, rattan and bamboo furniture, abundant houseplants, earth tone sage green and terracotta, 8k resolution, photorealistic, dappled sunlight",
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, synthetic materials, plastic plants",
"cfg": 7.0, "denoise": 0.68, "steps": 30,
},
"cyberpunk": {
"pos": "cyberpunk neon interior design, high contrast LED strip lighting electric blue and hot pink, reflective chrome surfaces, dark matte walls, futuristic furniture, glowing circuit patterns, tech-noir blade runner aesthetic, 8k resolution, photorealistic, volumetric fog",
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, natural daylight, rustic elements",
"cfg": 8.0, "denoise": 0.75, "steps": 30,
},
"japandi": {
"pos": "japandi fusion interior design, wabi-sabi textures, low-profile furniture, muted earth tones warm grays soft browns, handmade ceramic accents, light ash wood, shoji screen elements, minimal decoration, zen garden elements, 8k resolution, photorealistic, serene",
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, bright colors, ornate decoration, cluttered",
"cfg": 6.5, "denoise": 0.70, "steps": 30,
},
}
def build_workflow(img_name: str, style: str = "scandinavian",
keywords: str = "", denoise_override: float = None) -> dict:
s = STYLE_LIBRARY.get(style, STYLE_LIBRARY["scandinavian"])
# Merge user keywords into base positive prompt
pos = s["pos"]
if keywords.strip():
pos = pos + ", " + keywords.strip()
cfg = s["cfg"]
denoise = denoise_override if denoise_override else s["denoise"]
steps = s["steps"]
neg = s["neg"]
return {
"1": {"class_type": "CheckpointLoaderSimple",
"inputs": {"ckpt_name": "realvisxlV50_v50LightningBakedvae.safetensors"}},
"2": {"class_type": "LoadImage",
"inputs": {"image": img_name, "upload": "image"}},
"3": {"class_type": "CLIPTextEncode",
"inputs": {"text": pos, "clip": ["1", 1]}}, # ← POSITIVE PROMPT
"4": {"class_type": "CLIPTextEncode",
"inputs": {"text": neg, "clip": ["1", 1]}}, # ← NEGATIVE PROMPT
"5": {"class_type": "VAEEncode",
"inputs": {"pixels": ["2", 0], "vae": ["1", 2]}},
"6": {"class_type": "KSampler",
"inputs": {"model": ["1", 0], "positive": ["3", 0], "negative": ["4", 0],
"latent_image": ["5", 0],
"seed": int(time.time()) % 999983,
"steps": steps, "cfg": cfg,
"sampler_name": "dpmpp_2m", "scheduler": "karras",
"denoise": denoise}},
"7": {"class_type": "VAEDecode",
"inputs": {"samples": ["6", 0], "vae": ["1", 2]}},
"8": {"class_type": "SaveImage",
"inputs": {"images": ["7", 0], "filename_prefix": f"dreamweaver_{style}"}}
}
```
**Also update the endpoint signatures** to accept `keywords`:
```python
@app.post("/dream-weaver")
async def dream_weaver(
image: UploadFile = File(...),
style: str = Form(default="scandinavian"),
keywords: str = Form(default=""), # ← ADD THIS
denoise: float = Form(default=0.0), # 0.0 = use style default
):
...
wf = build_workflow(comfy_name, style=style, keywords=keywords,
denoise_override=denoise if denoise > 0 else None)
```
---
## 5. Swift Data Models
```swift
// AppConfig.swift
struct AppConfig {
static let baseURL = "http://54.172.172.2:8080"
// Change this to HTTPS domain once SSL is set up
}
// Models
struct GenerationJob: Codable {
let jobId: String
let status: String
let pollUrl: String
let resultUrl: String
enum CodingKeys: String, CodingKey {
case jobId = "job_id"
case status, pollUrl = "poll_url", resultUrl = "result_url"
}
}
struct JobStatus: Codable {
let status: String
let ready: Bool
let resultUrl: String?
let error: String?
enum CodingKeys: String, CodingKey {
case status, ready, resultUrl = "result_url", error
}
}
struct HealthResponse: Codable {
let status: String
let comfyui: Bool
}
enum DreamWeaverError: Error {
case generationFailed(String)
case timeout
case invalidImageData
}
// Style model for the style picker
struct InteriorStyle: Identifiable {
let id: String // used as the `style` form field value
let displayName: String
let keywords: [String] // shown as tappable chips in UI
let previewImage: String // local asset name
}
let availableStyles: [InteriorStyle] = [
InteriorStyle(id: "scandinavian", displayName: "Scandinavian", keywords: ["oak","linen","white","hygge","cozy","birch"], previewImage: "style_scandi"),
InteriorStyle(id: "art_deco", displayName: "Art Deco Luxe", keywords: ["gold","marble","velvet","geometric","brass","crystal"], previewImage: "style_artdeco"),
InteriorStyle(id: "biophilic", displayName: "Biophilic", keywords: ["green wall","stone","rattan","terracotta","botanical"], previewImage: "style_biophilic"),
InteriorStyle(id: "cyberpunk", displayName: "Cyberpunk Neon", keywords: ["neon","chrome","LED","futuristic","dark","holographic"], previewImage: "style_cyberpunk"),
InteriorStyle(id: "japandi", displayName: "Japandi Fusion", keywords: ["wabi-sabi","ceramic","ash wood","zen","minimal"], previewImage: "style_japandi"),
]
```
---
## 6. Complete Generation Flow (UI → Server → Result)
```
1. User opens camera / library → picks a room photo
2. User selects a style card → style ID captured
3. User optionally taps keyword chips → keywords[] array built
4. User taps "Generate" →
POST /dream-weaver (multipart)
image: <jpeg data>
style: "art_deco"
keywords: "gold, marble, luxury hotel"
denoise: 0.72
5. Server returns { job_id: "abc123", status: "processing" }
6. App shows loading/progress UI
7. App polls GET /dream-weaver/status/abc123 every 2 seconds
8. When ready == true →
GET /dream-weaver/result/abc123 → returns PNG bytes
9. App displays result full-screen with save/share options
```
**Expected latency on 4× L4 GPU server:** `~15–20 seconds` end-to-end.
---
## 7. WebSocket Progress (Optional Advanced Feature)
If Sayan wants a real-time progress bar (e.g. "Step 12/30"), connect directly to ComfyUI's WebSocket **only if port 8188 is opened**. Otherwise, polling `/status` is sufficient.
```swift
// WebSocket — only if 8188 is exposed externally
class ComfyProgressWebSocket: NSObject, URLSessionWebSocketDelegate {
var onProgress: ((Int, Int) -> Void)?
var task: URLSessionWebSocketTask?
func connect(clientId: String) {
let url = URL(string: "ws://54.172.172.2:8188/ws?clientId=\(clientId)")!
task = URLSession.shared.webSocketTask(with: url)
task?.resume()
listen()
}
private func listen() {
task?.receive { [weak self] result in
if case .success(let message) = result,
case .string(let text) = message,
let data = text.data(using: .utf8),
let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
let type = json["type"] as? String, type == "progress",
let inner = json["data"] as? [String: Int] {
self?.onProgress?(inner["value"] ?? 0, inner["max"] ?? 30)
}
self?.listen() // recurse
}
}
}
```
> [!NOTE]
> Port 8188 is currently **not open externally** in the security group. Only port 8080 is. To use WebSocket progress, Sagnik needs to add an inbound rule for 8188. Until then, using `/status` polling every 2s gives good enough UX.
---
## 8. Error Handling
| HTTP Status | Meaning | UI Action |
|---|---|---|
| `200` | Success | Show result or job_id |
| `404` on `/status` | Job expired (> 30 min) | "Session expired. Please retry." |
| `500` | Generation failed (OOM, model error) | "Generation failed. Try a simpler image." |
| Connection error | Server down or no internet | "Checking server…" + retry logic |
The job [status](file:///C:/Windows/Temp/dw_gateway.py#154-164) field can also be `"error"` with an `error` field explaining what failed.
---
## 9. Quick Checklist for Sayan
- [ ] Update `AppConfig.swift` with `baseURL = "http://54.172.172.2:8080"`
- [ ] Implement `POST /dream-weaver` multipart with `image + style + keywords`
- [ ] Implement polling loop on `GET /dream-weaver/status/{job_id}`
- [ ] Implement image download from `GET /dream-weaver/result/{job_id}`
- [ ] Add `GET /health` check on app launch
- [ ] Build keyword chips UI with the 5 style palettes from Section 2.3
- [ ] Test with the 20 sample images in `comfy_engine/test_inputs/`
## 10. Quick Checklist for Sagnik (backend)
- [ ] Update [dw_gateway.py](file:///C:/Windows/Temp/dw_gateway.py) with the full `STYLE_LIBRARY` dict (Section 4)
- [ ] Add `keywords: str = Form(default="")` to both POST endpoints
- [ ] Pass keywords into [build_workflow()](file:///C:/Windows/Temp/dw_gateway.py#33-65) for prompt expansion
- [ ] Redeploy gateway on port 8080 (`nohup python3 dw_gateway.py &`)
- [ ] (Optional) Open port 8188 in security group for WebSocket progress
# Dream Weaver API v2 — iOS Integration Guide (Dynamic Keywords)
**Version:** 2.0-FINAL | **Updated:** 2026-03-09 | **Server:** `54.172.172.2` | **Port:** `8082`
> This document is for **Sayan** (iOS / Swift).
> Dream Weaver API v2 introduces a **Dynamic Keyword to Local LLM Prompt Expansion** system.
> The app no longer relies on 5 hardcoded styles. Users can pick ANY keywords, and a local LLM (Qwen 3.5 27B via Ollama) will generate a photorealistic interior design prompt based on the room type without sending data to the cloud.
> [!CAUTION]
> **PORT 8080 IS DEAD.** Do not use port 8080 anymore. The old gateway process has been completely killed. If you try to send `POST /dream-weaver` or `/docs` to port 8080 you will get a 404. You MUST change your `AppConfig.baseURL` parameter to use port **`8082`**.
---
## 1. Architecture Overview (API v2)
```
┌────────────────────┐ HTTP/S ┌──────────────────────────────┐
│ │ ── keywords ────► │ Dream Weaver Gateway v2 │
│ iPad App (Swift) │ │ FastAPI port 8082 │
│ │ ◄── PNG result ── │ dw_gateway_v2.py │
└────────────────────┘ └─────────────┬────────────────┘
│ LLM Prompt Expansion
│ (Local Ollama: Qwen 3.5 27B)
▼
┌─────────────────────────┐
│ ComfyUI Engine │
│ port 8188 │
│ RealVisXL V5.0 Ltng │
└─────────────────────────┘
```
**Key changes in v2:**
1. The API now runs on port **`8082`** to avoid conflicts.
2. The [style](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/dreamweaver_batch_processor.py#394-414) parameter is deprecated in favor of `keywords` (array of strings) and [room_type](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/dw_gateway_v2.py#171-186).
---
## 2. Dynamic Keyword Expansion Flow
Instead of injecting keywords into a rigid template, the new backend reads the `keywords` and [room_type](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/dw_gateway_v2.py#171-186), and asks a local LLM (Qwen 3.5 27B) to act as an interior designer:
1. **User input:** `keywords: ["blue marble", "gold veins", "renaissance"]`, `room_type: "bathroom"`
2. **Backend LLM Expansion:** The LLM knows that a "bathroom" cannot have beds and needs wet-area materials. It creates a rich positive prompt: *"renaissance revival luxury interior design, blue veined marble flooring, gold brass fixtures..."*
3. **ComfyUI Generation:** The expanded prompt is sent to ComfyUI for generation.
**Supported Room Types:**
`bedroom`, `living_room`, `bathroom`, `kitchen`, `dining_room`, `home_office`, `hallway`, `balcony`.
---
## 3. API Reference — New v2 Endpoints
### BASE URL
```
http://54.172.172.2:8082
```
### 3.1 `GET /health` — Liveness Check
Call this on app launch to confirm the v2 server is up.
**Response:**
```json
{
"status": "ok",
"comfyui": true,
"gpu": "4x NVIDIA L4 (96GB VRAM)",
"model": "RealVisXL V5.0 Lightning",
"llm_expansion": true,
"version": "2.0.0"
}
```
### 3.2 `GET /room-types`
Returns all supported room types and their required design context (useful if you want to build UI tooltips).
```json
{
"room_types": {
"bedroom": {
"description": "a private sleeping space",
"key_elements": ["bed", "bedside tables", "wardrobe", "soft lighting", "textiles", "headboard"]
},
...
}
}
```
### 3.3 `POST /dream-weaver/expand` (Preview Prompt)
Use this if you want the user to **preview** the LLM's generated prompt before committing to a generation.
**Request (JSON):**
```json
{
"keywords": ["blue marble", "gold veins", "renaissance"],
"room_type": "bathroom"
}
```
**Response:**
```json
{
"style_name": "Renaissance Luxury",
"positive_prompt": "renaissance revival luxury interior design, blue veined marble flooring...",
"negative_prompt": "(worst quality, low quality...), extra windows...",
"cfg": 7.5,
"denoise": 0.72,
"steps": 30,
"source": "ollama_local"
}
```
### 3.4 `POST /dream-weaver` (Submit Generation)
Use this for the main generation flow.
**Request:** `multipart/form-data`
| Field | Type | Required | Description |
|---|---|---|---|
| [image](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/a100_deployment_executor.py#306-322) | File | ✅ | The room photo (JPEG/PNG) |
| `keywords` | String | ✅ | Comma-separated user keywords e.g. `"gold, marble, luxury"` |
| [room_type](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/dw_gateway_v2.py#171-186) | String | ✅ | e.g. `"living_room"`, `"bedroom"` |
| `additional_notes` | String | ➕ | (Optional) e.g. `"make it feel like a luxury hotel"` |
| `denoise` | Float | ➕ | (Optional) 0.5–0.85. If omitted, LLM decides. |
**Response:**
```json
{
"job_id": "a1b2c3d4-...",
"status": "processing",
"prompt_preview": "renaissance revival luxury interior design...",
"poll_url": "/dream-weaver/status/a1b2c3d4-...",
"result_url": "/dream-weaver/result/a1b2c3d4-..."
}
```
---
## 4. Polling & Downloading (Unchanged from v1)
**Poll Job Status:**
`GET /dream-weaver/status/{job_id}` every 2 seconds until `ready == true`.
**Download Result:**
`GET /dream-weaver/result/{job_id}` returns the raw PNG stream.
---
## 5. Updated Swift Example (v2)
```swift
func submitGenerationV2(image: UIImage, roomType: String, keywords: [String]) async throws -> GenerationJob {
let url = URL(string: "\(AppConfig.baseURL)/dream-weaver")!
var request = URLRequest(url: url)
request.httpMethod = "POST"
let boundary = UUID().uuidString
request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")
var body = Data()
// 1. Image
let imageData = image.jpegData(compressionQuality: 0.85)!
body.appendMultipartForm(boundary: boundary, name: "image", filename: "room.jpg", contentType: "image/jpeg", data: imageData)
// 2. Room Type
body.appendMultipartForm(boundary: boundary, name: "room_type", value: roomType)
// 3. Keywords
let kwString = keywords.joined(separator: ", ")
body.appendMultipartForm(boundary: boundary, name: "keywords", value: kwString)
body.append("--\(boundary)--\r\n".data(using: .utf8)!)
request.httpBody = body
let (data, _) = try await URLSession.shared.data(for: request)
return try JSONDecoder().decode(GenerationJob.self, from: data)
}
// Helper extension for building multipart forms cleanly
extension Data {
mutating func appendMultipartForm(boundary: String, name: String, value: String) {
self.append("--\(boundary)\r\n".data(using: .utf8)!)
self.append("Content-Disposition: form-data; name=\"\(name)\"\r\n\r\n".data(using: .utf8)!)
self.append("\(value)\r\n".data(using: .utf8)!)
}
mutating func appendMultipartForm(boundary: String, name: String, filename: String, contentType: String, data: Data) {
self.append("--\(boundary)\r\n".data(using: .utf8)!)
self.append("Content-Disposition: form-data; name=\"\(name)\"; filename=\"\(filename)\"\r\n".data(using: .utf8)!)
self.append("Content-Type: \(contentType)\r\n\r\n".data(using: .utf8)!)
self.append(data)
self.append("\r\n".data(using: .utf8)!)
}
}
```
---
## 6. Sayan's Action Checklist (v2)
- [ ] Change `AppConfig.baseURL` port to `8082` (e.g., `http://54.172.172.2:8082`).
- [ ] Add a UI element for the user to select the **Room Type** (`bedroom`, `living_room`, `bathroom`, etc.).
- [ ] Change the `POST /dream-weaver` payload from `{style}` to `{keywords, room_type}`.
- [ ] (Optional) Use the new `GET /dream-weaver/expand` endpoint to let the user preview and edit the AI-generated prompt before generating.

72
.Agent Context/Bibels/infrastructure_deployment_manifest.md Normal file
View File

@@ -0,0 +1,72 @@
# Dream Weaver — Infrastructure & Connectivity Manifest
**Environment:** Production AWS Node
**Last Verified:** 2026-03-14
**Status:** ✅ HEALTHY
---
## 🖥️ Server Instance
> [!IMPORTANT]
> **Active Public IP: `54.91.19.60`**
> The previous Elastic IP (`54.172.172.2`) is currently detached and will time out. Ensure all your connection strings use the active IP.
| Component | Value |
|---|---|
| **Instance ID** | `i-0e4eab5fe67cf9abe` |
| **Instance Name** | Desineuron AWS Node 4x L4 (96GB VRAM) Spot |
| **Public IP (Active)** | **`54.91.19.60`** |
| **Private IP** | `172.31.46.190` |
| **VPC ID** | `vpc-081d2397920aad268` |
---
## 🛡️ Security Group Settings
**Security Group ID:** `sg-0b144c17b1b89f4c6` (Synapse-Ops)
The following Inbound rules are explicitly confirmed and open:
| Protocol | Port | Source | Description |
|---|---|---|---|
| TCP | **22** | `0.0.0.0/0` | SSH Access |
| TCP | **8082** | `0.0.0.0/0` | **Dream Weaver API v2 (Current)** |
| TCP | **8188** | `0.0.0.0/0` | ComfyUI Internal API |
| TCP | **8000** | `0.0.0.0/0` | ComfyUI Web UI (Alternate) |
---
## 🚀 Services & Endpoints
### 1. Dream Weaver Gateway v2
* **Port:** `8082`
* **Status:** ✅ Active
* **Health Check:** `http://54.91.19.60:8082/health`
* **Main Endpoint:** `POST http://54.91.19.60:8082/dream-weaver`
### 2. ComfyUI Engine
* **Port:** `8188`
* **Status:** ✅ Active
* **Prompt Endpoint:** `POST http://54.91.19.60:8188/prompt`
---
## 🔑 SSH Configuration
**Local Key File Path:** `f:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\desineuron-l4-node.pem`
### Quick Connect Command
```bash
ssh -i "path/to/desineuron-l4-node.pem" ubuntu@54.91.19.60
```
---
## 📝 Operator Checklist (Troubleshooting)
1. **Verify API Process:**
`ps aux | grep dw_gateway_v2`
2. **Check Logs:**
`tail -f /home/ubuntu/gw_v2.log`
3. **Check Port Listeners:**
`sudo netstat -tulpn | grep 8082`
4. **No Zombie Processes:**
Port 8080 has been cleared. Only 8082 is serving the gateway.

342
comfy_engine/scripts/README.md
View File

@@ -1,171 +1,171 @@
# Dream Weaver Automation Scripts
This directory contains Python automation scripts for the Dream Weaver interior restyling workflow.
## Scripts Overview
### 1. dreamweaver_batch_processor.py
Main batch processing controller for automated image restyling.
**Features:**
- Directory monitoring for automatic job queueing
- Automatic mask caching for improved performance
- Queue management with status tracking
- Support for all three processing phases
- WebSocket integration with ComfyUI for real-time status
**Usage:**
```bash
# Process single image
python dreamweaver_batch_processor.py --input image.jpg --style scandinavian_minimalist --phase 1
# Process all images in directory
python dreamweaver_batch_processor.py --batch --style art_deco_luxe --phase 2
# Start directory monitoring mode
python dreamweaver_batch_processor.py --monitor
```
### 2. mask_preprocessor.py
Utility for preprocessing and caching segmentation masks.
**Features:**
- Offline mask generation and caching
- Mask refinement (grow, feather, invert)
- Multi-region mask support (walls, floor, ceiling)
- Batch preprocessing for entire directories
- Cache management and statistics
**Usage:**
```bash
# Preprocess single image
python mask_preprocessor.py --image image.jpg
# Preprocess entire directory
python mask_preprocessor.py --directory ../test_inputs/
# Show cache statistics
python mask_preprocessor.py --stats
# Clear all cached masks
python mask_preprocessor.py --clear-cache
# Custom mask parameters
python mask_preprocessor.py --image image.jpg --grow 5 --feather 8
```
## Configuration
Scripts use configuration from `CONFIG` dictionary in each file. Key settings:
- `comfyui_server`: ComfyUI HTTP endpoint (default: http://localhost:8188)
- `comfyui_ws`: ComfyUI WebSocket endpoint (default: ws://localhost:8188/ws)
- `input_directory`: Default input images directory
- `output_directory`: Generated images output directory
- `cache_directory`: Mask cache storage location
- `batch_size`: Number of images to process in batch (Phase 3)
## Integration with ComfyUI
These scripts require ComfyUI to be running with the Dream Weaver workflows loaded.
**Starting ComfyUI:**
```bash
cd Project_Velocity/comfy_engine
python main.py --fp16 --lowvram
```
**For Production (Dual RTX PRO 6000):**
```bash
python main.py --bf16 --highvram --xformers --gpu-batch-size 8
```
## Workflow Files
Scripts reference these workflow JSON files:
- `workflows/dreamweaver_phase1_depth.json` - Single ControlNet (RTX 3080Ti)
- `workflows/dreamweaver_phase2_multicontrol.json` - Multi-ControlNet (RTX 3080Ti)
- `workflows/dreamweaver_phase3_batch.json` - Batch processing (Dual RTX PRO 6000)
## Style Templates
Available style templates (located in `../prompts/`):
- `scandinavian_minimalist` - Light, airy Nordic design
- `art_deco_luxe` - Glamorous 1920s aesthetic
- `cyberpunk_neon` - High-tech futuristic
- `biophilic_organic` - Nature-connected sustainable
- `japandi_fusion` - Japanese-Scandinavian blend
## Dependencies
Install required packages:
```bash
pip install -r ../requirements.txt
```
## Logging
Scripts output logs to:
- Console (real-time)
- `dreamweaver_batch.log` (file)
Log level can be adjusted in script `logging.basicConfig()` calls.
## Hardware Requirements
**Phase 1 & 2 (Development):**
- NVIDIA RTX 3080Ti (12GB VRAM)
- 32GB System RAM
- SSD Storage
**Phase 3 (Production):**
- Dual NVIDIA RTX PRO 6000 Blackwell (96GB VRAM each)
- 128GB System RAM
- NVMe SSD Storage
- NVLink enabled for GPU memory pooling
## API Reference
### ComfyUI Endpoints Used
- `POST /prompt` - Submit workflow to queue
- `GET /queue` - Get queue status
- `WS /ws` - WebSocket for real-time updates
### Job Status Values
- `pending` - Waiting in queue
- `processing` - Currently generating
- `completed` - Successfully finished
- `failed` - Error occurred
## Troubleshooting
**Connection Refused Error:**
- Ensure ComfyUI is running
- Check server URL in configuration
- Verify firewall settings
**Out of Memory:**
- Reduce batch size
- Lower resolution
- Enable tiled VAE decoding
**Mask Cache Issues:**
- Clear cache: `python mask_preprocessor.py --clear-cache`
- Check cache directory permissions
- Verify available disk space
## Development
To extend functionality:
1. Modify `BatchProcessor` class for new processing logic
2. Add new style templates in `../prompts/`
3. Update workflow JSON files for new ControlNet configurations
## Support
For issues related to:
- **Scripts**: Check logs in `dreamweaver_batch.log`
- **ComfyUI**: Refer to ComfyUI documentation
- **Workflows**: See technical specification in `../docs/DREAMWEAVER_TECHNICAL_SPEC.md`
# Dream Weaver Automation Scripts
This directory contains Python automation scripts for the Dream Weaver interior restyling workflow.
## Scripts Overview
### 1. dreamweaver_batch_processor.py
Main batch processing controller for automated image restyling.
**Features:**
- Directory monitoring for automatic job queueing
- Automatic mask caching for improved performance
- Queue management with status tracking
- Support for all three processing phases
- WebSocket integration with ComfyUI for real-time status
**Usage:**
```bash
# Process single image
python dreamweaver_batch_processor.py --input image.jpg --style scandinavian_minimalist --phase 1
# Process all images in directory
python dreamweaver_batch_processor.py --batch --style art_deco_luxe --phase 2
# Start directory monitoring mode
python dreamweaver_batch_processor.py --monitor
```
### 2. mask_preprocessor.py
Utility for preprocessing and caching segmentation masks.
**Features:**
- Offline mask generation and caching
- Mask refinement (grow, feather, invert)
- Multi-region mask support (walls, floor, ceiling)
- Batch preprocessing for entire directories
- Cache management and statistics
**Usage:**
```bash
# Preprocess single image
python mask_preprocessor.py --image image.jpg
# Preprocess entire directory
python mask_preprocessor.py --directory ../test_inputs/
# Show cache statistics
python mask_preprocessor.py --stats
# Clear all cached masks
python mask_preprocessor.py --clear-cache
# Custom mask parameters
python mask_preprocessor.py --image image.jpg --grow 5 --feather 8
```
## Configuration
Scripts use configuration from `CONFIG` dictionary in each file. Key settings:
- `comfyui_server`: ComfyUI HTTP endpoint (default: http://localhost:8188)
- `comfyui_ws`: ComfyUI WebSocket endpoint (default: ws://localhost:8188/ws)
- `input_directory`: Default input images directory
- `output_directory`: Generated images output directory
- `cache_directory`: Mask cache storage location
- `batch_size`: Number of images to process in batch (Phase 3)
## Integration with ComfyUI
These scripts require ComfyUI to be running with the Dream Weaver workflows loaded.
**Starting ComfyUI:**
```bash
cd Project_Velocity/comfy_engine
python main.py --fp16 --lowvram
```
**For Production (Dual RTX PRO 6000):**
```bash
python main.py --bf16 --highvram --xformers --gpu-batch-size 8
```
## Workflow Files
Scripts reference these workflow JSON files:
- `workflows/dreamweaver_phase1_depth.json` - Single ControlNet (RTX 3080Ti)
- `workflows/dreamweaver_phase2_multicontrol.json` - Multi-ControlNet (RTX 3080Ti)
- `workflows/dreamweaver_phase3_batch.json` - Batch processing (Dual RTX PRO 6000)
## Style Templates
Available style templates (located in `../prompts/`):
- `scandinavian_minimalist` - Light, airy Nordic design
- `art_deco_luxe` - Glamorous 1920s aesthetic
- `cyberpunk_neon` - High-tech futuristic
- `biophilic_organic` - Nature-connected sustainable
- `japandi_fusion` - Japanese-Scandinavian blend
## Dependencies
Install required packages:
```bash
pip install -r ../requirements.txt
```
## Logging
Scripts output logs to:
- Console (real-time)
- `dreamweaver_batch.log` (file)
Log level can be adjusted in script `logging.basicConfig()` calls.
## Hardware Requirements
**Phase 1 & 2 (Development):**
- NVIDIA RTX 3080Ti (12GB VRAM)
- 32GB System RAM
- SSD Storage
**Phase 3 (Production):**
- Dual NVIDIA RTX PRO 6000 Blackwell (96GB VRAM each)
- 128GB System RAM
- NVMe SSD Storage
- NVLink enabled for GPU memory pooling
## API Reference
### ComfyUI Endpoints Used
- `POST /prompt` - Submit workflow to queue
- `GET /queue` - Get queue status
- `WS /ws` - WebSocket for real-time updates
### Job Status Values
- `pending` - Waiting in queue
- `processing` - Currently generating
- `completed` - Successfully finished
- `failed` - Error occurred
## Troubleshooting
**Connection Refused Error:**
- Ensure ComfyUI is running
- Check server URL in configuration
- Verify firewall settings
**Out of Memory:**
- Reduce batch size
- Lower resolution
- Enable tiled VAE decoding
**Mask Cache Issues:**
- Clear cache: `python mask_preprocessor.py --clear-cache`
- Check cache directory permissions
- Verify available disk space
## Development
To extend functionality:
1. Modify `BatchProcessor` class for new processing logic
2. Add new style templates in `../prompts/`
3. Update workflow JSON files for new ControlNet configurations
## Support
For issues related to:
- **Scripts**: Check logs in `dreamweaver_batch.log`
- **ComfyUI**: Refer to ComfyUI documentation
- **Workflows**: See technical specification in `../docs/DREAMWEAVER_TECHNICAL_SPEC.md`

1244
comfy_engine/scripts/a100_deployment_executor.py
View File

File diff suppressed because it is too large Load Diff

996
comfy_engine/scripts/dreamweaver_batch_processor.py
View File

@@ -1,498 +1,498 @@
#!/usr/bin/env python3
"""
Dream Weaver Batch Processor
============================
Automated batch processing script for Dream Weaver interior restyling workflow.
Handles directory monitoring, automatic mask caching, and queue management.
Target Hardware: Dual NVIDIA RTX PRO 6000 Blackwell (96GB GDDR7 each)
Author: Project Velocity Team
Version: 1.0.0
"""
import os
import sys
import json
import time
import hashlib
import asyncio
import argparse
import logging
from pathlib import Path
from datetime import datetime
from typing import Dict, List, Optional, Tuple
from dataclasses import dataclass, asdict
import requests
import websockets
import aiofiles
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
# Configuration
CONFIG = {
"comfyui_server": "http://localhost:8188",
"comfyui_ws": "ws://localhost:8188/ws",
"input_directory": "Project_Velocity/comfy_engine/test_inputs/",
"output_directory": "Project_Velocity/comfy_engine/test_outputs/",
"cache_directory": "Project_Velocity/comfy_engine/cache/masks/",
"workflow_phase1": "Project_Velocity/comfy_engine/workflows/dreamweaver_phase1_depth.json",
"workflow_phase2": "Project_Velocity/comfy_engine/workflows/dreamweaver_phase2_multicontrol.json",
"workflow_phase3": "Project_Velocity/comfy_engine/workflows/dreamweaver_phase3_batch.json",
"batch_size": 8,
"target_resolution": (1024, 1024),
"enable_mask_cache": True,
"gpu_sharding": True,
"dual_gpu": True,
}
# Setup logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
handlers=[
logging.FileHandler('dreamweaver_batch.log'),
logging.StreamHandler()
]
)
logger = logging.getLogger('DreamWeaver')
@dataclass
class ProcessingJob:
"""Represents a single image processing job."""
job_id: str
input_path: str
output_path: str
style_template: str
phase: int
status: str = "pending"
created_at: datetime = None
started_at: datetime = None
completed_at: datetime = None
error_message: str = None
mask_cached: bool = False
def __post_init__(self):
if self.created_at is None:
self.created_at = datetime.now()
def to_dict(self) -> Dict:
return {
"job_id": self.job_id,
"input_path": self.input_path,
"output_path": self.output_path,
"style_template": self.style_template,
"phase": self.phase,
"status": self.status,
"created_at": self.created_at.isoformat() if self.created_at else None,
"started_at": self.started_at.isoformat() if self.started_at else None,
"completed_at": self.completed_at.isoformat() if self.completed_at else None,
"error_message": self.error_message,
"mask_cached": self.mask_cached
}
class MaskCacheManager:
"""Manages caching of segmentation masks for improved performance."""
def __init__(self, cache_dir: str):
self.cache_dir = Path(cache_dir)
self.cache_dir.mkdir(parents=True, exist_ok=True)
logger.info(f"Mask cache initialized at: {self.cache_dir}")
def _get_cache_key(self, image_path: str) -> str:
"""Generate cache key from image content hash."""
hasher = hashlib.md5()
with open(image_path, 'rb') as f:
hasher.update(f.read())
return hasher.hexdigest()
def get_cached_mask(self, image_path: str) -> Optional[str]:
"""Retrieve cached mask path if it exists."""
cache_key = self._get_cache_key(image_path)
cached_path = self.cache_dir / f"{cache_key}.png"
if cached_path.exists():
logger.info(f"Cache hit for {image_path}")
return str(cached_path)
return None
def cache_mask(self, image_path: str, mask_path: str) -> str:
"""Cache a mask file for future use."""
cache_key = self._get_cache_key(image_path)
cached_path = self.cache_dir / f"{cache_key}.png"
import shutil
shutil.copy2(mask_path, cached_path)
logger.info(f"Cached mask for {image_path} at {cached_path}")
return str(cached_path)
class ComfyUIClient:
"""Client for communicating with ComfyUI server."""
def __init__(self, server_url: str, ws_url: str):
self.server_url = server_url
self.ws_url = ws_url
self.client_id = self._generate_client_id()
logger.info(f"ComfyUI client initialized with ID: {self.client_id}")
def _generate_client_id(self) -> str:
"""Generate unique client ID."""
return f"dreamweaver_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{os.urandom(4).hex()}"
async def submit_workflow(self, workflow: Dict, input_image: str) -> str:
"""Submit a workflow to ComfyUI queue."""
# Update workflow with input image
for node_id, node in workflow.items():
if node.get("class_type") == "LoadImage":
node["inputs"]["image"] = input_image
if node.get("class_type") == "LoadImageBatch":
node["inputs"]["directory"] = os.path.dirname(input_image)
payload = {
"prompt": workflow,
"client_id": self.client_id
}
response = requests.post(
f"{self.server_url}/prompt",
json=payload
)
response.raise_for_status()
result = response.json()
prompt_id = result.get("prompt_id")
logger.info(f"Submitted workflow with prompt_id: {prompt_id}")
return prompt_id
async def get_queue_status(self) -> Dict:
"""Get current queue status."""
response = requests.get(f"{self.server_url}/queue")
return response.json()
async def wait_for_completion(self, prompt_id: str, timeout: int = 300) -> bool:
"""Wait for workflow completion via WebSocket."""
start_time = time.time()
async with websockets.connect(
f"{self.ws_url}?clientId={self.client_id}"
) as websocket:
while time.time() - start_time < timeout:
try:
message = await asyncio.wait_for(
websocket.recv(),
timeout=5.0
)
data = json.loads(message)
if data.get("type") == "executing":
if data["data"].get("prompt_id") == prompt_id:
node_id = data["data"].get("node")
logger.debug(f"Executing node: {node_id}")
elif data.get("type") == "completed":
if data["data"].get("prompt_id") == prompt_id:
logger.info(f"Workflow {prompt_id} completed")
return True
elif data.get("type") == "error":
logger.error(f"Workflow error: {data}")
return False
except asyncio.TimeoutError:
continue
logger.warning(f"Workflow {prompt_id} timed out")
return False
class BatchProcessor:
"""Main batch processing controller."""
def __init__(self, config: Dict):
self.config = config
self.queue: List[ProcessingJob] = []
self.processing = False
self.cache_manager = MaskCacheManager(config["cache_directory"])
self.comfy_client = ComfyUIClient(
config["comfyui_server"],
config["comfyui_ws"]
)
# Load workflow templates
self.workflows = self._load_workflows()
# Ensure output directory exists
Path(config["output_directory"]).mkdir(parents=True, exist_ok=True)
def _load_workflows(self) -> Dict[int, Dict]:
"""Load workflow JSON files."""
workflows = {}
workflow_paths = {
1: self.config["workflow_phase1"],
2: self.config["workflow_phase2"],
3: self.config["workflow_phase3"]
}
for phase, path in workflow_paths.items():
try:
with open(path, 'r') as f:
workflows[phase] = json.load(f)
logger.info(f"Loaded Phase {phase} workflow")
except Exception as e:
logger.error(f"Failed to load Phase {phase} workflow: {e}")
return workflows
def add_job(self, input_path: str, style_template: str = "scandinavian_minimalist", phase: int = 1) -> str:
"""Add a new processing job to the queue."""
job_id = hashlib.md5(f"{input_path}_{time.time()}".encode()).hexdigest()[:12]
output_filename = f"{Path(input_path).stem}_restyled_{job_id}.png"
output_path = os.path.join(self.config["output_directory"], output_filename)
job = ProcessingJob(
job_id=job_id,
input_path=input_path,
output_path=output_path,
style_template=style_template,
phase=phase
)
# Check if mask is cached
if self.config["enable_mask_cache"]:
cached_mask = self.cache_manager.get_cached_mask(input_path)
job.mask_cached = cached_mask is not None
self.queue.append(job)
logger.info(f"Added job {job_id} to queue. Queue size: {len(self.queue)}")
return job_id
async def process_single(self, job: ProcessingJob) -> bool:
"""Process a single job."""
job.status = "processing"
job.started_at = datetime.now()
try:
logger.info(f"Processing job {job.job_id}: {job.input_path}")
# Get workflow for phase
workflow = self.workflows.get(job.phase)
if not workflow:
raise ValueError(f"Workflow for phase {job.phase} not found")
# Submit to ComfyUI
prompt_id = await self.comfy_client.submit_workflow(
workflow,
job.input_path
)
# Wait for completion
success = await self.comfy_client.wait_for_completion(prompt_id)
if success:
job.status = "completed"
job.completed_at = datetime.now()
logger.info(f"Job {job.job_id} completed successfully")
return True
else:
job.status = "failed"
job.error_message = "Workflow execution failed or timed out"
logger.error(f"Job {job.job_id} failed")
return False
except Exception as e:
job.status = "failed"
job.error_message = str(e)
logger.error(f"Error processing job {job.job_id}: {e}")
return False
async def process_batch(self, jobs: List[ProcessingJob]) -> List[bool]:
"""Process multiple jobs in batch (Phase 3)."""
if not jobs:
return []
logger.info(f"Processing batch of {len(jobs)} jobs")
results = []
# For batch processing, use Phase 3 workflow
workflow = self.workflows.get(3)
if not workflow:
logger.warning("Phase 3 workflow not available, processing sequentially")
for job in jobs:
result = await self.process_single(job)
results.append(result)
return results
# TODO: Implement true batch processing with Phase 3 workflow
# This would require grouping images and processing together
for job in jobs:
result = await self.process_single(job)
results.append(result)
return results
async def run(self):
"""Main processing loop."""
logger.info("Starting batch processor")
self.processing = True
while self.processing:
# Get pending jobs
pending_jobs = [j for j in self.queue if j.status == "pending"]
if not pending_jobs:
await asyncio.sleep(1)
continue
# Check if batch processing is appropriate
if len(pending_jobs) >= self.config["batch_size"] and self.config.get("dual_gpu"):
# Process in batches for Phase 3
batch = pending_jobs[:self.config["batch_size"]]
await self.process_batch(batch)
else:
# Process single job with appropriate phase
job = pending_jobs[0]
await self.process_single(job)
def stop(self):
"""Stop the processing loop."""
logger.info("Stopping batch processor")
self.processing = False
def get_status(self) -> Dict:
"""Get current processing status."""
total = len(self.queue)
pending = len([j for j in self.queue if j.status == "pending"])
processing = len([j for j in self.queue if j.status == "processing"])
completed = len([j for j in self.queue if j.status == "completed"])
failed = len([j for j in self.queue if j.status == "failed"])
return {
"total_jobs": total,
"pending": pending,
"processing": processing,
"completed": completed,
"failed": failed,
"is_running": self.processing
}
class InputDirectoryHandler(FileSystemEventHandler):
"""Handles new file events in input directory."""
def __init__(self, processor: BatchProcessor):
self.processor = processor
def on_created(self, event):
if not event.is_directory:
file_path = event.src_path
if file_path.lower().endswith(('.jpg', '.jpeg', '.png', '.webp')):
logger.info(f"New image detected: {file_path}")
self.processor.add_job(file_path)
def load_style_template(template_name: str) -> str:
"""Load a style template from prompts directory."""
template_path = Path("Project_Velocity/comfy_engine/prompts/") / f"{template_name}.txt"
if template_path.exists():
with open(template_path, 'r') as f:
content = f.read()
# Extract positive prompt
lines = content.split('\n')
positive_lines = []
in_positive = False
for line in lines:
if 'POSITIVE PROMPT:' in line:
in_positive = True
continue
if in_positive and line.startswith('Style Weight:'):
break
if in_positive and line.strip() and not line.startswith('-'):
positive_lines.append(line.strip())
return ' '.join(positive_lines)
return ""
async def main():
"""Main entry point."""
parser = argparse.ArgumentParser(
description="Dream Weaver Batch Processor"
)
parser.add_argument(
"--monitor",
action="store_true",
help="Enable directory monitoring mode"
)
parser.add_argument(
"--input",
type=str,
help="Single input image to process"
)
parser.add_argument(
"--style",
type=str,
default="scandinavian_minimalist",
choices=["scandinavian_minimalist", "art_deco_luxe", "cyberpunk_neon", "biophilic_organic", "japandi_fusion"],
help="Style template to apply"
)
parser.add_argument(
"--phase",
type=int,
default=1,
choices=[1, 2, 3],
help="Processing phase to use"
)
parser.add_argument(
"--batch",
action="store_true",
help="Process all images in input directory"
)
args = parser.parse_args()
# Initialize processor
processor = BatchProcessor(CONFIG)
if args.input:
# Process single image
job_id = processor.add_job(args.input, args.style, args.phase)
await processor.process_single(processor.queue[-1])
print(f"Processed image: {args.input}")
print(f"Job ID: {job_id}")
elif args.batch:
# Process all images in directory
input_dir = Path(CONFIG["input_directory"])
image_files = list(input_dir.glob("*.jpg")) + list(input_dir.glob("*.png"))
for img_file in image_files:
processor.add_job(str(img_file), args.style, args.phase)
await processor.run()
elif args.monitor:
# Start directory monitoring
event_handler = InputDirectoryHandler(processor)
observer = Observer()
observer.schedule(
event_handler,
CONFIG["input_directory"],
recursive=False
)
observer.start()
logger.info(f"Started monitoring: {CONFIG['input_directory']}")
try:
# Run processor
await processor.run()
except KeyboardInterrupt:
processor.stop()
observer.stop()
observer.join()
else:
print("No action specified. Use --help for usage information.")
if __name__ == "__main__":
asyncio.run(main())
#!/usr/bin/env python3
"""
Dream Weaver Batch Processor
============================
Automated batch processing script for Dream Weaver interior restyling workflow.
Handles directory monitoring, automatic mask caching, and queue management.
Target Hardware: Dual NVIDIA RTX PRO 6000 Blackwell (96GB GDDR7 each)
Author: Project Velocity Team
Version: 1.0.0
"""
import os
import sys
import json
import time
import hashlib
import asyncio
import argparse
import logging
from pathlib import Path
from datetime import datetime
from typing import Dict, List, Optional, Tuple
from dataclasses import dataclass, asdict
import requests
import websockets
import aiofiles
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
# Configuration
CONFIG = {
"comfyui_server": "http://localhost:8188",
"comfyui_ws": "ws://localhost:8188/ws",
"input_directory": "Project_Velocity/comfy_engine/test_inputs/",
"output_directory": "Project_Velocity/comfy_engine/test_outputs/",
"cache_directory": "Project_Velocity/comfy_engine/cache/masks/",
"workflow_phase1": "Project_Velocity/comfy_engine/workflows/dreamweaver_phase1_depth.json",
"workflow_phase2": "Project_Velocity/comfy_engine/workflows/dreamweaver_phase2_multicontrol.json",
"workflow_phase3": "Project_Velocity/comfy_engine/workflows/dreamweaver_phase3_batch.json",
"batch_size": 8,
"target_resolution": (1024, 1024),
"enable_mask_cache": True,
"gpu_sharding": True,
"dual_gpu": True,
}
# Setup logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
handlers=[
logging.FileHandler('dreamweaver_batch.log'),
logging.StreamHandler()
]
)
logger = logging.getLogger('DreamWeaver')
@dataclass
class ProcessingJob:
"""Represents a single image processing job."""
job_id: str
input_path: str
output_path: str
style_template: str
phase: int
status: str = "pending"
created_at: datetime = None
started_at: datetime = None
completed_at: datetime = None
error_message: str = None
mask_cached: bool = False
def __post_init__(self):
if self.created_at is None:
self.created_at = datetime.now()
def to_dict(self) -> Dict:
return {
"job_id": self.job_id,
"input_path": self.input_path,
"output_path": self.output_path,
"style_template": self.style_template,
"phase": self.phase,
"status": self.status,
"created_at": self.created_at.isoformat() if self.created_at else None,
"started_at": self.started_at.isoformat() if self.started_at else None,
"completed_at": self.completed_at.isoformat() if self.completed_at else None,
"error_message": self.error_message,
"mask_cached": self.mask_cached
}
class MaskCacheManager:
"""Manages caching of segmentation masks for improved performance."""
def __init__(self, cache_dir: str):
self.cache_dir = Path(cache_dir)
self.cache_dir.mkdir(parents=True, exist_ok=True)
logger.info(f"Mask cache initialized at: {self.cache_dir}")
def _get_cache_key(self, image_path: str) -> str:
"""Generate cache key from image content hash."""
hasher = hashlib.md5()
with open(image_path, 'rb') as f:
hasher.update(f.read())
return hasher.hexdigest()
def get_cached_mask(self, image_path: str) -> Optional[str]:
"""Retrieve cached mask path if it exists."""
cache_key = self._get_cache_key(image_path)
cached_path = self.cache_dir / f"{cache_key}.png"
if cached_path.exists():
logger.info(f"Cache hit for {image_path}")
return str(cached_path)
return None
def cache_mask(self, image_path: str, mask_path: str) -> str:
"""Cache a mask file for future use."""
cache_key = self._get_cache_key(image_path)
cached_path = self.cache_dir / f"{cache_key}.png"
import shutil
shutil.copy2(mask_path, cached_path)
logger.info(f"Cached mask for {image_path} at {cached_path}")
return str(cached_path)
class ComfyUIClient:
"""Client for communicating with ComfyUI server."""
def __init__(self, server_url: str, ws_url: str):
self.server_url = server_url
self.ws_url = ws_url
self.client_id = self._generate_client_id()
logger.info(f"ComfyUI client initialized with ID: {self.client_id}")
def _generate_client_id(self) -> str:
"""Generate unique client ID."""
return f"dreamweaver_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{os.urandom(4).hex()}"
async def submit_workflow(self, workflow: Dict, input_image: str) -> str:
"""Submit a workflow to ComfyUI queue."""
# Update workflow with input image
for node_id, node in workflow.items():
if node.get("class_type") == "LoadImage":
node["inputs"]["image"] = input_image
if node.get("class_type") == "LoadImageBatch":
node["inputs"]["directory"] = os.path.dirname(input_image)
payload = {
"prompt": workflow,
"client_id": self.client_id
}
response = requests.post(
f"{self.server_url}/prompt",
json=payload
)
response.raise_for_status()
result = response.json()
prompt_id = result.get("prompt_id")
logger.info(f"Submitted workflow with prompt_id: {prompt_id}")
return prompt_id
async def get_queue_status(self) -> Dict:
"""Get current queue status."""
response = requests.get(f"{self.server_url}/queue")
return response.json()
async def wait_for_completion(self, prompt_id: str, timeout: int = 300) -> bool:
"""Wait for workflow completion via WebSocket."""
start_time = time.time()
async with websockets.connect(
f"{self.ws_url}?clientId={self.client_id}"
) as websocket:
while time.time() - start_time < timeout:
try:
message = await asyncio.wait_for(
websocket.recv(),
timeout=5.0
)
data = json.loads(message)
if data.get("type") == "executing":
if data["data"].get("prompt_id") == prompt_id:
node_id = data["data"].get("node")
logger.debug(f"Executing node: {node_id}")
elif data.get("type") == "completed":
if data["data"].get("prompt_id") == prompt_id:
logger.info(f"Workflow {prompt_id} completed")
return True
elif data.get("type") == "error":
logger.error(f"Workflow error: {data}")
return False
except asyncio.TimeoutError:
continue
logger.warning(f"Workflow {prompt_id} timed out")
return False
class BatchProcessor:
"""Main batch processing controller."""
def __init__(self, config: Dict):
self.config = config
self.queue: List[ProcessingJob] = []
self.processing = False
self.cache_manager = MaskCacheManager(config["cache_directory"])
self.comfy_client = ComfyUIClient(
config["comfyui_server"],
config["comfyui_ws"]
)
# Load workflow templates
self.workflows = self._load_workflows()
# Ensure output directory exists
Path(config["output_directory"]).mkdir(parents=True, exist_ok=True)
def _load_workflows(self) -> Dict[int, Dict]:
"""Load workflow JSON files."""
workflows = {}
workflow_paths = {
1: self.config["workflow_phase1"],
2: self.config["workflow_phase2"],
3: self.config["workflow_phase3"]
}
for phase, path in workflow_paths.items():
try:
with open(path, 'r') as f:
workflows[phase] = json.load(f)
logger.info(f"Loaded Phase {phase} workflow")
except Exception as e:
logger.error(f"Failed to load Phase {phase} workflow: {e}")
return workflows
def add_job(self, input_path: str, style_template: str = "scandinavian_minimalist", phase: int = 1) -> str:
"""Add a new processing job to the queue."""
job_id = hashlib.md5(f"{input_path}_{time.time()}".encode()).hexdigest()[:12]
output_filename = f"{Path(input_path).stem}_restyled_{job_id}.png"
output_path = os.path.join(self.config["output_directory"], output_filename)
job = ProcessingJob(
job_id=job_id,
input_path=input_path,
output_path=output_path,
style_template=style_template,
phase=phase
)
# Check if mask is cached
if self.config["enable_mask_cache"]:
cached_mask = self.cache_manager.get_cached_mask(input_path)
job.mask_cached = cached_mask is not None
self.queue.append(job)
logger.info(f"Added job {job_id} to queue. Queue size: {len(self.queue)}")
return job_id
async def process_single(self, job: ProcessingJob) -> bool:
"""Process a single job."""
job.status = "processing"
job.started_at = datetime.now()
try:
logger.info(f"Processing job {job.job_id}: {job.input_path}")
# Get workflow for phase
workflow = self.workflows.get(job.phase)
if not workflow:
raise ValueError(f"Workflow for phase {job.phase} not found")
# Submit to ComfyUI
prompt_id = await self.comfy_client.submit_workflow(
workflow,
job.input_path
)
# Wait for completion
success = await self.comfy_client.wait_for_completion(prompt_id)
if success:
job.status = "completed"
job.completed_at = datetime.now()
logger.info(f"Job {job.job_id} completed successfully")
return True
else:
job.status = "failed"
job.error_message = "Workflow execution failed or timed out"
logger.error(f"Job {job.job_id} failed")
return False
except Exception as e:
job.status = "failed"
job.error_message = str(e)
logger.error(f"Error processing job {job.job_id}: {e}")
return False
async def process_batch(self, jobs: List[ProcessingJob]) -> List[bool]:
"""Process multiple jobs in batch (Phase 3)."""
if not jobs:
return []
logger.info(f"Processing batch of {len(jobs)} jobs")
results = []
# For batch processing, use Phase 3 workflow
workflow = self.workflows.get(3)
if not workflow:
logger.warning("Phase 3 workflow not available, processing sequentially")
for job in jobs:
result = await self.process_single(job)
results.append(result)
return results
# TODO: Implement true batch processing with Phase 3 workflow
# This would require grouping images and processing together
for job in jobs:
result = await self.process_single(job)
results.append(result)
return results
async def run(self):
"""Main processing loop."""
logger.info("Starting batch processor")
self.processing = True
while self.processing:
# Get pending jobs
pending_jobs = [j for j in self.queue if j.status == "pending"]
if not pending_jobs:
await asyncio.sleep(1)
continue
# Check if batch processing is appropriate
if len(pending_jobs) >= self.config["batch_size"] and self.config.get("dual_gpu"):
# Process in batches for Phase 3
batch = pending_jobs[:self.config["batch_size"]]
await self.process_batch(batch)
else:
# Process single job with appropriate phase
job = pending_jobs[0]
await self.process_single(job)
def stop(self):
"""Stop the processing loop."""
logger.info("Stopping batch processor")
self.processing = False
def get_status(self) -> Dict:
"""Get current processing status."""
total = len(self.queue)
pending = len([j for j in self.queue if j.status == "pending"])
processing = len([j for j in self.queue if j.status == "processing"])
completed = len([j for j in self.queue if j.status == "completed"])
failed = len([j for j in self.queue if j.status == "failed"])
return {
"total_jobs": total,
"pending": pending,
"processing": processing,
"completed": completed,
"failed": failed,
"is_running": self.processing
}
class InputDirectoryHandler(FileSystemEventHandler):
"""Handles new file events in input directory."""
def __init__(self, processor: BatchProcessor):
self.processor = processor
def on_created(self, event):
if not event.is_directory:
file_path = event.src_path
if file_path.lower().endswith(('.jpg', '.jpeg', '.png', '.webp')):
logger.info(f"New image detected: {file_path}")
self.processor.add_job(file_path)
def load_style_template(template_name: str) -> str:
"""Load a style template from prompts directory."""
template_path = Path("Project_Velocity/comfy_engine/prompts/") / f"{template_name}.txt"
if template_path.exists():
with open(template_path, 'r') as f:
content = f.read()
# Extract positive prompt
lines = content.split('\n')
positive_lines = []
in_positive = False
for line in lines:
if 'POSITIVE PROMPT:' in line:
in_positive = True
continue
if in_positive and line.startswith('Style Weight:'):
break
if in_positive and line.strip() and not line.startswith('-'):
positive_lines.append(line.strip())
return ' '.join(positive_lines)
return ""
async def main():
"""Main entry point."""
parser = argparse.ArgumentParser(
description="Dream Weaver Batch Processor"
)
parser.add_argument(
"--monitor",
action="store_true",
help="Enable directory monitoring mode"
)
parser.add_argument(
"--input",
type=str,
help="Single input image to process"
)
parser.add_argument(
"--style",
type=str,
default="scandinavian_minimalist",
choices=["scandinavian_minimalist", "art_deco_luxe", "cyberpunk_neon", "biophilic_organic", "japandi_fusion"],
help="Style template to apply"
)
parser.add_argument(
"--phase",
type=int,
default=1,
choices=[1, 2, 3],
help="Processing phase to use"
)
parser.add_argument(
"--batch",
action="store_true",
help="Process all images in input directory"
)
args = parser.parse_args()
# Initialize processor
processor = BatchProcessor(CONFIG)
if args.input:
# Process single image
job_id = processor.add_job(args.input, args.style, args.phase)
await processor.process_single(processor.queue[-1])
print(f"Processed image: {args.input}")
print(f"Job ID: {job_id}")
elif args.batch:
# Process all images in directory
input_dir = Path(CONFIG["input_directory"])
image_files = list(input_dir.glob("*.jpg")) + list(input_dir.glob("*.png"))
for img_file in image_files:
processor.add_job(str(img_file), args.style, args.phase)
await processor.run()
elif args.monitor:
# Start directory monitoring
event_handler = InputDirectoryHandler(processor)
observer = Observer()
observer.schedule(
event_handler,
CONFIG["input_directory"],
recursive=False
)
observer.start()
logger.info(f"Started monitoring: {CONFIG['input_directory']}")
try:
# Run processor
await processor.run()
except KeyboardInterrupt:
processor.stop()
observer.stop()
observer.join()
else:
print("No action specified. Use --help for usage information.")
if __name__ == "__main__":
asyncio.run(main())

840
comfy_engine/scripts/dw_gateway_v2.py
View File

@@ -1,420 +1,420 @@
#!/usr/bin/env python3
"""
Dream Weaver API Gateway v2 — Dynamic Keyword → Local LLM → ComfyUI Pipeline
========================================================================
Port: 8080 (public-facing)
ComfyUI: localhost:8188 (internal)
NEW IN v2:
- POST /dream-weaver now accepts keywords[] + room_type for LLM-based prompt generation
- POST /dream-weaver/expand — expand keywords to prompt WITHOUT generating (preview)
- GET /room-types — list available room types
- Uses local Ollama model (qwen3.5:27b) for prompt expansion (no cloud API dependencies)
Environment variables:
OLLAMA_URL — Ollama server (default: http://localhost:11434)
OLLAMA_MODEL — Model name (default: qwen3.5:27b)
"""
import asyncio, json, time, uuid, io, sys, os, logging
from pathlib import Path
from typing import Optional, List
import httpx
import uvicorn
from fastapi import FastAPI, UploadFile, File, HTTPException, Form, BackgroundTasks
from fastapi.responses import JSONResponse, StreamingResponse
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
# Add scripts dir to path so we can import prompt_expander
SCRIPTS_DIR = Path(__file__).parent / "scripts"
sys.path.insert(0, str(SCRIPTS_DIR))
try:
from prompt_expander import expand_prompt, expand_prompt_simple, ROOM_CONTEXTS, ExpandedPrompt
LLM_AVAILABLE = True
except ImportError:
LLM_AVAILABLE = False
logging.warning("prompt_expander not found — LLM expansion disabled")
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
logger = logging.getLogger("DreamWeaverGateway")
COMFY = "http://127.0.0.1:8188"
COMFY_ROOT = "/opt/dlami/nvme/ComfyUI"
app = FastAPI(
title="Dream Weaver API v2",
version="2.0.0",
description="Dynamic keyword-to-interior-design generation powered by LLM + ComfyUI"
)
app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"])
# In-memory job store (swap for Redis in production)
jobs: dict = {}
# ─── Models ──────────────────────────────────────────────────────────────────
class ExpandRequest(BaseModel):
keywords: List[str]
room_type: str = "living_room"
additional_notes: str = ""
class ExpandResponse(BaseModel):
style_name: str
positive_prompt: str
negative_prompt: str
cfg: float
denoise: float
steps: int
reasoning: str
source: str
# ─── ComfyUI helpers ──────────────────────────────────────────────────────────
async def upload_to_comfy(data: bytes, filename: str) -> str:
async with httpx.AsyncClient(timeout=30) as client:
r = await client.post(f"{COMFY}/upload/image",
files={"image": (filename, data, "image/jpeg")},
data={"overwrite": "true"})
r.raise_for_status()
return r.json()["name"]
def build_workflow(img_name: str, expanded: "ExpandedPrompt") -> dict:
"""Build ComfyUI API workflow from an ExpandedPrompt result."""
return {
"1": {"class_type": "CheckpointLoaderSimple",
"inputs": {"ckpt_name": "realvisxlV50_v50LightningBakedvae.safetensors"}},
"2": {"class_type": "LoadImage",
"inputs": {"image": img_name, "upload": "image"}},
"3": {"class_type": "CLIPTextEncode", # Positive prompt
"inputs": {"text": expanded.positive_prompt, "clip": ["1", 1]}},
"4": {"class_type": "CLIPTextEncode", # Negative prompt
"inputs": {"text": expanded.negative_prompt, "clip": ["1", 1]}},
"5": {"class_type": "VAEEncode",
"inputs": {"pixels": ["2", 0], "vae": ["1", 2]}},
"6": {"class_type": "KSampler",
"inputs": {"model": ["1", 0],
"positive": ["3", 0],
"negative": ["4", 0],
"latent_image": ["5", 0],
"seed": int(time.time()) % 999983,
"steps": expanded.steps,
"cfg": expanded.cfg,
"sampler_name": "dpmpp_2m",
"scheduler": "karras",
"denoise": expanded.denoise}},
"7": {"class_type": "VAEDecode",
"inputs": {"samples": ["6", 0], "vae": ["1", 2]}},
"8": {"class_type": "SaveImage",
"inputs": {"images": ["7", 0],
"filename_prefix": f"dw_{expanded.style_name.replace(' ', '_')[:30]}"}},
}
async def queue_prompt(workflow: dict) -> str:
async with httpx.AsyncClient(timeout=30) as client:
r = await client.post(f"{COMFY}/prompt",
json={"prompt": workflow, "client_id": str(uuid.uuid4())})
r.raise_for_status()
return r.json()["prompt_id"]
async def poll_result(prompt_id: str, timeout: int = 300):
start = time.time()
async with httpx.AsyncClient(timeout=10) as client:
while time.time() - start < timeout:
r = await client.get(f"{COMFY}/history/{prompt_id}")
if r.status_code == 200:
h = r.json().get(prompt_id, {})
if h.get("status", {}).get("status_str") == "error":
return None, h.get("status", {}).get("messages", ["unknown"])
imgs = [img for nd in h.get("outputs", {}).values()
for img in nd.get("images", [])]
if imgs:
return imgs[0], None
await asyncio.sleep(2)
return None, "timeout"
async def background_poll(job_id: str, prompt_id: str):
img, err = await poll_result(prompt_id)
if img:
jobs[job_id].update({"status": "done", "output": img, "completed": time.time()})
else:
jobs[job_id].update({"status": "error", "error": str(err)})
# ─── Endpoints ───────────────────────────────────────────────────────────────
@app.get("/health")
async def health():
comfy_ok = False
try:
async with httpx.AsyncClient(timeout=5) as c:
r = await c.get(f"{COMFY}/system_stats")
comfy_ok = r.status_code == 200
except Exception:
pass
return {
"status": "ok",
"comfyui": comfy_ok,
"gpu": "4x NVIDIA L4 (96GB VRAM)",
"model": "RealVisXL V5.0 Lightning",
"llm_expansion": LLM_AVAILABLE,
"version": "2.0.0"
}
@app.get("/room-types")
async def room_types():
"""List all supported room types with their context."""
if not LLM_AVAILABLE:
return {"room_types": ["bedroom", "living_room", "bathroom", "kitchen",
"dining_room", "home_office", "hallway", "balcony"]}
return {
"room_types": {
k: {
"description": v["description"],
"key_elements": v["key_elements"]
}
for k, v in ROOM_CONTEXTS.items()
}
}
@app.post("/dream-weaver/expand", response_model=ExpandResponse)
async def expand_endpoint(req: ExpandRequest):
"""
Preview the LLM-generated prompt WITHOUT submitting to ComfyUI.
Use this to let the user review/edit the prompt before generating.
Request body:
{
"keywords": ["blue marble", "gold veins", "renaissance", "sharp contours"],
"room_type": "bedroom",
"additional_notes": "luxury hotel feel"
}
"""
if not req.keywords:
raise HTTPException(status_code=400, detail="keywords list cannot be empty")
try:
if LLM_AVAILABLE:
result = await asyncio.to_thread(
expand_prompt,
keywords=req.keywords,
room_type=req.room_type,
additional_notes=req.additional_notes
)
else:
result = expand_prompt_simple(req.keywords, req.room_type)
except Exception as e:
logger.error(f"Prompt expansion failed: {e}")
raise HTTPException(status_code=500, detail=f"LLM expansion failed: {str(e)}")
return ExpandResponse(
style_name=result.style_name,
positive_prompt=result.positive_prompt,
negative_prompt=result.negative_prompt,
cfg=result.cfg,
denoise=result.denoise,
steps=result.steps,
reasoning=result.reasoning,
source=result.source
)
@app.post("/dream-weaver")
async def dream_weaver(
image: UploadFile = File(...),
# ── Dynamic keyword mode (new) ──
keywords: str = Form(default=""), # comma-separated: "blue marble, gold, renaissance"
room_type: str = Form(default="living_room"),
additional_notes: str = Form(default=""),
# ── Optional overrides ──
custom_positive: str = Form(default=""), # skip LLM, use this prompt directly
custom_negative: str = Form(default=""),
denoise: float = Form(default=0.0), # 0.0 = use LLM recommendation
cfg_scale: float = Form(default=0.0), # 0.0 = use LLM recommendation
):
"""
Submit a room photo for AI redesign using dynamic keyword → LLM → ComfyUI pipeline.
Two modes:
1. KEYWORD MODE (recommended): Provide keywords + room_type, LLM generates prompt
2. DIRECT MODE: Provide custom_positive + custom_negative to bypass LLM
Returns job_id for async polling.
"""
job_id = str(uuid.uuid4())
jobs[job_id] = {"status": "uploading", "created": time.time()}
try:
# Upload image to ComfyUI
data = await image.read()
filename = f"dw_{job_id[:8]}_{image.filename or 'room.jpg'}"
comfy_name = await upload_to_comfy(data, filename)
jobs[job_id]["status"] = "expanding_prompt"
# ── Determine prompt ──────────────────────────────────────────────
if custom_positive:
# Direct mode — user provided prompts explicitly
from dataclasses import dataclass
@dataclass
class DirectPrompt:
style_name: str = "custom"
positive_prompt: str = custom_positive
negative_prompt: str = custom_negative or (
"(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), "
"blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes"
)
cfg: float = cfg_scale or 7.5
denoise: float = denoise or 0.72
steps: int = 30
reasoning: str = "Direct user input"
source: str = "direct"
expanded = DirectPrompt()
elif keywords:
# Keyword mode — expand via LLM
kw_list = [k.strip() for k in keywords.split(",") if k.strip()]
if LLM_AVAILABLE:
expanded = await asyncio.to_thread(
expand_prompt,
keywords=kw_list,
room_type=room_type,
additional_notes=additional_notes
)
else:
expanded = expand_prompt_simple(kw_list, room_type)
# Apply manual overrides if provided
if denoise > 0:
expanded.denoise = denoise
if cfg_scale > 0:
expanded.cfg = cfg_scale
else:
raise HTTPException(status_code=400,
detail="Provide either 'keywords' or 'custom_positive'")
jobs[job_id].update({
"status": "queued",
"style": expanded.style_name,
"prompt_source": expanded.source,
"positive_prompt": expanded.positive_prompt,
"negative_prompt": expanded.negative_prompt,
"room_type": room_type,
})
# Submit workflow
wf = build_workflow(comfy_name, expanded)
prompt_id = await queue_prompt(wf)
jobs[job_id].update({"status": "processing", "prompt_id": prompt_id})
# Start background polling
asyncio.create_task(background_poll(job_id, prompt_id))
return {
"job_id": job_id,
"status": "processing",
"style": expanded.style_name,
"prompt_preview": expanded.positive_prompt[:120] + "...",
"reasoning": expanded.reasoning,
"poll_url": f"/dream-weaver/status/{job_id}",
"result_url": f"/dream-weaver/result/{job_id}"
}
except HTTPException:
raise
except Exception as e:
jobs[job_id] = {"status": "error", "error": str(e)}
logger.error(f"Generation failed: {e}")
raise HTTPException(status_code=500, detail=str(e))
@app.get("/dream-weaver/status/{job_id}")
async def status(job_id: str):
job = jobs.get(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
result = {k: v for k, v in job.items() if k != "output"}
result["ready"] = job.get("status") == "done"
if result["ready"]:
result["result_url"] = f"/dream-weaver/result/{job_id}"
return result
@app.get("/dream-weaver/result/{job_id}")
async def result(job_id: str):
job = jobs.get(job_id)
if not job or job.get("status") != "done":
raise HTTPException(status_code=404, detail="Result not ready")
img = job["output"]
url = (f"{COMFY}/view?filename={img['filename']}"
f"&subfolder={img.get('subfolder','')}&type={img.get('type','output')}")
async with httpx.AsyncClient(timeout=30) as c:
r = await c.get(url)
return StreamingResponse(
io.BytesIO(r.content),
media_type="image/png",
headers={"Content-Disposition": f"attachment; filename=dreamweaver_{job_id[:8]}.png"}
)
@app.post("/dream-weaver/sync")
async def dream_weaver_sync(
image: UploadFile = File(...),
keywords: str = Form(default=""),
room_type: str = Form(default="living_room"),
additional_notes: str = Form(default=""),
custom_positive: str = Form(default=""),
custom_negative: str = Form(default=""),
):
"""
Blocking version — waits up to 120s and returns image bytes directly.
Use for testing. Prefer async /dream-weaver for production.
"""
data = await image.read()
filename = f"sync_{uuid.uuid4().hex[:8]}_{image.filename or 'room.jpg'}"
comfy_name = await upload_to_comfy(data, filename)
if custom_positive:
from dataclasses import dataclass
@dataclass
class _P:
style_name = "custom"
positive_prompt = custom_positive
negative_prompt = custom_negative or "(worst quality, low quality), blurry, structural changes"
cfg = 7.5; denoise = 0.72; steps = 30
reasoning = ""; source = "direct"
expanded = _P()
elif keywords:
kw_list = [k.strip() for k in keywords.split(",") if k.strip()]
expanded = (expand_prompt(kw_list, room_type, additional_notes)
if LLM_AVAILABLE else expand_prompt_simple(kw_list, room_type))
else:
raise HTTPException(status_code=400, detail="Provide keywords or custom_positive")
wf = build_workflow(comfy_name, expanded)
prompt_id = await queue_prompt(wf)
img, err = await poll_result(prompt_id, timeout=120)
if err:
raise HTTPException(status_code=500, detail=str(err))
url = (f"{COMFY}/view?filename={img['filename']}"
f"&subfolder={img.get('subfolder','')}&type={img.get('type','output')}")
async with httpx.AsyncClient(timeout=30) as c:
r = await c.get(url)
return StreamingResponse(io.BytesIO(r.content), media_type="image/png",
headers={"X-Style": expanded.style_name,
"X-Prompt-Source": expanded.source})
if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=int(os.environ.get("PORT", "8082")), log_level="info")
#!/usr/bin/env python3
"""
Dream Weaver API Gateway v2 — Dynamic Keyword → Local LLM → ComfyUI Pipeline
========================================================================
Port: 8080 (public-facing)
ComfyUI: localhost:8188 (internal)
NEW IN v2:
- POST /dream-weaver now accepts keywords[] + room_type for LLM-based prompt generation
- POST /dream-weaver/expand — expand keywords to prompt WITHOUT generating (preview)
- GET /room-types — list available room types
- Uses local Ollama model (qwen3.5:27b) for prompt expansion (no cloud API dependencies)
Environment variables:
OLLAMA_URL — Ollama server (default: http://localhost:11434)
OLLAMA_MODEL — Model name (default: qwen3.5:27b)
"""
import asyncio, json, time, uuid, io, sys, os, logging
from pathlib import Path
from typing import Optional, List
import httpx
import uvicorn
from fastapi import FastAPI, UploadFile, File, HTTPException, Form, BackgroundTasks
from fastapi.responses import JSONResponse, StreamingResponse
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
# Add scripts dir to path so we can import prompt_expander
SCRIPTS_DIR = Path(__file__).parent / "scripts"
sys.path.insert(0, str(SCRIPTS_DIR))
try:
from prompt_expander import expand_prompt, expand_prompt_simple, ROOM_CONTEXTS, ExpandedPrompt
LLM_AVAILABLE = True
except ImportError:
LLM_AVAILABLE = False
logging.warning("prompt_expander not found — LLM expansion disabled")
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
logger = logging.getLogger("DreamWeaverGateway")
COMFY = "http://127.0.0.1:8188"
COMFY_ROOT = "/opt/dlami/nvme/ComfyUI"
app = FastAPI(
title="Dream Weaver API v2",
version="2.0.0",
description="Dynamic keyword-to-interior-design generation powered by LLM + ComfyUI"
)
app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"])
# In-memory job store (swap for Redis in production)
jobs: dict = {}
# ─── Models ──────────────────────────────────────────────────────────────────
class ExpandRequest(BaseModel):
keywords: List[str]
room_type: str = "living_room"
additional_notes: str = ""
class ExpandResponse(BaseModel):
style_name: str
positive_prompt: str
negative_prompt: str
cfg: float
denoise: float
steps: int
reasoning: str
source: str
# ─── ComfyUI helpers ──────────────────────────────────────────────────────────
async def upload_to_comfy(data: bytes, filename: str) -> str:
async with httpx.AsyncClient(timeout=30) as client:
r = await client.post(f"{COMFY}/upload/image",
files={"image": (filename, data, "image/jpeg")},
data={"overwrite": "true"})
r.raise_for_status()
return r.json()["name"]
def build_workflow(img_name: str, expanded: "ExpandedPrompt") -> dict:
"""Build ComfyUI API workflow from an ExpandedPrompt result."""
return {
"1": {"class_type": "CheckpointLoaderSimple",
"inputs": {"ckpt_name": "realvisxlV50_v50LightningBakedvae.safetensors"}},
"2": {"class_type": "LoadImage",
"inputs": {"image": img_name, "upload": "image"}},
"3": {"class_type": "CLIPTextEncode", # Positive prompt
"inputs": {"text": expanded.positive_prompt, "clip": ["1", 1]}},
"4": {"class_type": "CLIPTextEncode", # Negative prompt
"inputs": {"text": expanded.negative_prompt, "clip": ["1", 1]}},
"5": {"class_type": "VAEEncode",
"inputs": {"pixels": ["2", 0], "vae": ["1", 2]}},
"6": {"class_type": "KSampler",
"inputs": {"model": ["1", 0],
"positive": ["3", 0],
"negative": ["4", 0],
"latent_image": ["5", 0],
"seed": int(time.time()) % 999983,
"steps": expanded.steps,
"cfg": expanded.cfg,
"sampler_name": "dpmpp_2m",
"scheduler": "karras",
"denoise": expanded.denoise}},
"7": {"class_type": "VAEDecode",
"inputs": {"samples": ["6", 0], "vae": ["1", 2]}},
"8": {"class_type": "SaveImage",
"inputs": {"images": ["7", 0],
"filename_prefix": f"dw_{expanded.style_name.replace(' ', '_')[:30]}"}},
}
async def queue_prompt(workflow: dict) -> str:
async with httpx.AsyncClient(timeout=30) as client:
r = await client.post(f"{COMFY}/prompt",
json={"prompt": workflow, "client_id": str(uuid.uuid4())})
r.raise_for_status()
return r.json()["prompt_id"]
async def poll_result(prompt_id: str, timeout: int = 300):
start = time.time()
async with httpx.AsyncClient(timeout=10) as client:
while time.time() - start < timeout:
r = await client.get(f"{COMFY}/history/{prompt_id}")
if r.status_code == 200:
h = r.json().get(prompt_id, {})
if h.get("status", {}).get("status_str") == "error":
return None, h.get("status", {}).get("messages", ["unknown"])
imgs = [img for nd in h.get("outputs", {}).values()
for img in nd.get("images", [])]
if imgs:
return imgs[0], None
await asyncio.sleep(2)
return None, "timeout"
async def background_poll(job_id: str, prompt_id: str):
img, err = await poll_result(prompt_id)
if img:
jobs[job_id].update({"status": "done", "output": img, "completed": time.time()})
else:
jobs[job_id].update({"status": "error", "error": str(err)})
# ─── Endpoints ───────────────────────────────────────────────────────────────
@app.get("/health")
async def health():
comfy_ok = False
try:
async with httpx.AsyncClient(timeout=5) as c:
r = await c.get(f"{COMFY}/system_stats")
comfy_ok = r.status_code == 200
except Exception:
pass
return {
"status": "ok",
"comfyui": comfy_ok,
"gpu": "4x NVIDIA L4 (96GB VRAM)",
"model": "RealVisXL V5.0 Lightning",
"llm_expansion": LLM_AVAILABLE,
"version": "2.0.0"
}
@app.get("/room-types")
async def room_types():
"""List all supported room types with their context."""
if not LLM_AVAILABLE:
return {"room_types": ["bedroom", "living_room", "bathroom", "kitchen",
"dining_room", "home_office", "hallway", "balcony"]}
return {
"room_types": {
k: {
"description": v["description"],
"key_elements": v["key_elements"]
}
for k, v in ROOM_CONTEXTS.items()
}
}
@app.post("/dream-weaver/expand", response_model=ExpandResponse)
async def expand_endpoint(req: ExpandRequest):
"""
Preview the LLM-generated prompt WITHOUT submitting to ComfyUI.
Use this to let the user review/edit the prompt before generating.
Request body:
{
"keywords": ["blue marble", "gold veins", "renaissance", "sharp contours"],
"room_type": "bedroom",
"additional_notes": "luxury hotel feel"
}
"""
if not req.keywords:
raise HTTPException(status_code=400, detail="keywords list cannot be empty")
try:
if LLM_AVAILABLE:
result = await asyncio.to_thread(
expand_prompt,
keywords=req.keywords,
room_type=req.room_type,
additional_notes=req.additional_notes
)
else:
result = expand_prompt_simple(req.keywords, req.room_type)
except Exception as e:
logger.error(f"Prompt expansion failed: {e}")
raise HTTPException(status_code=500, detail=f"LLM expansion failed: {str(e)}")
return ExpandResponse(
style_name=result.style_name,
positive_prompt=result.positive_prompt,
negative_prompt=result.negative_prompt,
cfg=result.cfg,
denoise=result.denoise,
steps=result.steps,
reasoning=result.reasoning,
source=result.source
)
@app.post("/dream-weaver")
async def dream_weaver(
image: UploadFile = File(...),
# ── Dynamic keyword mode (new) ──
keywords: str = Form(default=""), # comma-separated: "blue marble, gold, renaissance"
room_type: str = Form(default="living_room"),
additional_notes: str = Form(default=""),
# ── Optional overrides ──
custom_positive: str = Form(default=""), # skip LLM, use this prompt directly
custom_negative: str = Form(default=""),
denoise: float = Form(default=0.0), # 0.0 = use LLM recommendation
cfg_scale: float = Form(default=0.0), # 0.0 = use LLM recommendation
):
"""
Submit a room photo for AI redesign using dynamic keyword → LLM → ComfyUI pipeline.
Two modes:
1. KEYWORD MODE (recommended): Provide keywords + room_type, LLM generates prompt
2. DIRECT MODE: Provide custom_positive + custom_negative to bypass LLM
Returns job_id for async polling.
"""
job_id = str(uuid.uuid4())
jobs[job_id] = {"status": "uploading", "created": time.time()}
try:
# Upload image to ComfyUI
data = await image.read()
filename = f"dw_{job_id[:8]}_{image.filename or 'room.jpg'}"
comfy_name = await upload_to_comfy(data, filename)
jobs[job_id]["status"] = "expanding_prompt"
# ── Determine prompt ──────────────────────────────────────────────
if custom_positive:
# Direct mode — user provided prompts explicitly
from dataclasses import dataclass
@dataclass
class DirectPrompt:
style_name: str = "custom"
positive_prompt: str = custom_positive
negative_prompt: str = custom_negative or (
"(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), "
"blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes"
)
cfg: float = cfg_scale or 7.5
denoise: float = denoise or 0.72
steps: int = 30
reasoning: str = "Direct user input"
source: str = "direct"
expanded = DirectPrompt()
elif keywords:
# Keyword mode — expand via LLM
kw_list = [k.strip() for k in keywords.split(",") if k.strip()]
if LLM_AVAILABLE:
expanded = await asyncio.to_thread(
expand_prompt,
keywords=kw_list,
room_type=room_type,
additional_notes=additional_notes
)
else:
expanded = expand_prompt_simple(kw_list, room_type)
# Apply manual overrides if provided
if denoise > 0:
expanded.denoise = denoise
if cfg_scale > 0:
expanded.cfg = cfg_scale
else:
raise HTTPException(status_code=400,
detail="Provide either 'keywords' or 'custom_positive'")
jobs[job_id].update({
"status": "queued",
"style": expanded.style_name,
"prompt_source": expanded.source,
"positive_prompt": expanded.positive_prompt,
"negative_prompt": expanded.negative_prompt,
"room_type": room_type,
})
# Submit workflow
wf = build_workflow(comfy_name, expanded)
prompt_id = await queue_prompt(wf)
jobs[job_id].update({"status": "processing", "prompt_id": prompt_id})
# Start background polling
asyncio.create_task(background_poll(job_id, prompt_id))
return {
"job_id": job_id,
"status": "processing",
"style": expanded.style_name,
"prompt_preview": expanded.positive_prompt[:120] + "...",
"reasoning": expanded.reasoning,
"poll_url": f"/dream-weaver/status/{job_id}",
"result_url": f"/dream-weaver/result/{job_id}"
}
except HTTPException:
raise
except Exception as e:
jobs[job_id] = {"status": "error", "error": str(e)}
logger.error(f"Generation failed: {e}")
raise HTTPException(status_code=500, detail=str(e))
@app.get("/dream-weaver/status/{job_id}")
async def status(job_id: str):
job = jobs.get(job_id)
if not job:
raise HTTPException(status_code=404, detail="Job not found")
result = {k: v for k, v in job.items() if k != "output"}
result["ready"] = job.get("status") == "done"
if result["ready"]:
result["result_url"] = f"/dream-weaver/result/{job_id}"
return result
@app.get("/dream-weaver/result/{job_id}")
async def result(job_id: str):
job = jobs.get(job_id)
if not job or job.get("status") != "done":
raise HTTPException(status_code=404, detail="Result not ready")
img = job["output"]
url = (f"{COMFY}/view?filename={img['filename']}"
f"&subfolder={img.get('subfolder','')}&type={img.get('type','output')}")
async with httpx.AsyncClient(timeout=30) as c:
r = await c.get(url)
return StreamingResponse(
io.BytesIO(r.content),
media_type="image/png",
headers={"Content-Disposition": f"attachment; filename=dreamweaver_{job_id[:8]}.png"}
)
@app.post("/dream-weaver/sync")
async def dream_weaver_sync(
image: UploadFile = File(...),
keywords: str = Form(default=""),
room_type: str = Form(default="living_room"),
additional_notes: str = Form(default=""),
custom_positive: str = Form(default=""),
custom_negative: str = Form(default=""),
):
"""
Blocking version — waits up to 120s and returns image bytes directly.
Use for testing. Prefer async /dream-weaver for production.
"""
data = await image.read()
filename = f"sync_{uuid.uuid4().hex[:8]}_{image.filename or 'room.jpg'}"
comfy_name = await upload_to_comfy(data, filename)
if custom_positive:
from dataclasses import dataclass
@dataclass
class _P:
style_name = "custom"
positive_prompt = custom_positive
negative_prompt = custom_negative or "(worst quality, low quality), blurry, structural changes"
cfg = 7.5; denoise = 0.72; steps = 30
reasoning = ""; source = "direct"
expanded = _P()
elif keywords:
kw_list = [k.strip() for k in keywords.split(",") if k.strip()]
expanded = (expand_prompt(kw_list, room_type, additional_notes)
if LLM_AVAILABLE else expand_prompt_simple(kw_list, room_type))
else:
raise HTTPException(status_code=400, detail="Provide keywords or custom_positive")
wf = build_workflow(comfy_name, expanded)
prompt_id = await queue_prompt(wf)
img, err = await poll_result(prompt_id, timeout=120)
if err:
raise HTTPException(status_code=500, detail=str(err))
url = (f"{COMFY}/view?filename={img['filename']}"
f"&subfolder={img.get('subfolder','')}&type={img.get('type','output')}")
async with httpx.AsyncClient(timeout=30) as c:
r = await c.get(url)
return StreamingResponse(io.BytesIO(r.content), media_type="image/png",
headers={"X-Style": expanded.style_name,
"X-Prompt-Source": expanded.source})
if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=int(os.environ.get("PORT", "8082")), log_level="info")

776
comfy_engine/scripts/mask_preprocessor.py
View File

@@ -1,388 +1,388 @@
#!/usr/bin/env python3
"""
Dream Weaver Mask Preprocessor
==============================
Utility script for preprocessing and caching segmentation masks.
Enables offline mask generation to speed up production workflows.
Target Hardware: Dual NVIDIA RTX PRO 6000 Blackwell
Author: Project Velocity Team
Version: 1.0.0
"""
import os
import sys
import json
import hashlib
import argparse
import logging
from pathlib import Path
from typing import List, Optional, Tuple, Dict
from dataclasses import dataclass
import numpy as np
from PIL import Image
import cv2
# Configure logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger('MaskPreprocessor')
@dataclass
class MaskConfig:
"""Configuration for mask generation."""
grow_pixels: int = 3
feather_pixels: int = 5
threshold: float = 0.3
target_classes: List[str] = None
def __post_init__(self):
if self.target_classes is None:
self.target_classes = ["wall", "floor", "ceiling"]
class MaskPreprocessor:
"""Preprocesses and caches segmentation masks for Dream Weaver."""
def __init__(self, cache_dir: str = "Project_Velocity/comfy_engine/cache/masks/"):
self.cache_dir = Path(cache_dir)
self.cache_dir.mkdir(parents=True, exist_ok=True)
self.config = MaskConfig()
logger.info(f"MaskPreprocessor initialized. Cache directory: {self.cache_dir}")
def _get_image_hash(self, image_path: str) -> str:
"""Generate MD5 hash of image for caching."""
hasher = hashlib.md5()
with open(image_path, 'rb') as f:
hasher.update(f.read())
return hasher.hexdigest()
def _get_cache_path(self, image_path: str, suffix: str = "") -> Path:
"""Generate cache file path for an image."""
image_hash = self._get_image_hash(image_path)
filename = f"{image_hash}{suffix}.png"
return self.cache_dir / filename
def is_cached(self, image_path: str, suffix: str = "") -> bool:
"""Check if a mask is already cached."""
cache_path = self._get_cache_path(image_path, suffix)
return cache_path.exists()
def load_from_cache(self, image_path: str, suffix: str = "") -> Optional[np.ndarray]:
"""Load mask from cache if available."""
cache_path = self._get_cache_path(image_path, suffix)
if cache_path.exists():
logger.info(f"Loading cached mask from {cache_path}")
mask = cv2.imread(str(cache_path), cv2.IMREAD_GRAYSCALE)
return mask
return None
def save_to_cache(self, image_path: str, mask: np.ndarray, suffix: str = "") -> str:
"""Save mask to cache."""
cache_path = self._get_cache_path(image_path, suffix)
cv2.imwrite(str(cache_path), mask)
logger.info(f"Saved mask to cache: {cache_path}")
return str(cache_path)
def create_structural_mask(self, image_path: str, mask_data: np.ndarray) -> np.ndarray:
"""
Create a structural preservation mask from segmentation data.
This mask identifies walls, floors, ceilings that must be preserved.
"""
# Ensure binary mask
if len(mask_data.shape) == 3:
mask_data = cv2.cvtColor(mask_data, cv2.COLOR_BGR2GRAY)
_, binary_mask = cv2.threshold(
mask_data,
int(255 * self.config.threshold),
255,
cv2.THRESH_BINARY
)
return binary_mask.astype(np.uint8)
def grow_mask(self, mask: np.ndarray, pixels: int = None) -> np.ndarray:
"""
Grow (dilate) the mask by specified pixels.
This prevents edge bleeding by expanding the mask slightly.
"""
if pixels is None:
pixels = self.config.grow_pixels
kernel = np.ones((pixels * 2 + 1, pixels * 2 + 1), np.uint8)
grown_mask = cv2.dilate(mask, kernel, iterations=1)
return grown_mask
def feather_mask(self, mask: np.ndarray, pixels: int = None) -> np.ndarray:
"""
Apply Gaussian blur to feather mask edges.
Creates smooth transitions at boundaries.
"""
if pixels is None:
pixels = self.config.feather_pixels
# Ensure odd kernel size
kernel_size = pixels * 2 + 1
feathered = cv2.GaussianBlur(mask, (kernel_size, kernel_size), 0)
return feathered
def invert_mask(self, mask: np.ndarray) -> np.ndarray:
"""Invert mask (structural -> stylable or vice versa)."""
return cv2.bitwise_not(mask)
def combine_masks(self, masks: List[np.ndarray], operation: str = "union") -> np.ndarray:
"""
Combine multiple masks.
operation: 'union' (OR), 'intersection' (AND), 'difference'
"""
if not masks:
return None
result = masks[0].copy()
for mask in masks[1:]:
if operation == "union":
result = cv2.bitwise_or(result, mask)
elif operation == "intersection":
result = cv2.bitwise_and(result, mask)
elif operation == "difference":
result = cv2.bitwise_and(result, cv2.bitwise_not(mask))
return result
def create_multi_region_mask(
self,
image_path: str,
regions: Dict[str, np.ndarray]
) -> Dict[str, np.ndarray]:
"""
Create masks for multiple regions (walls, floor, ceiling, etc.)
Returns dictionary of processed masks.
"""
processed_masks = {}
for region_name, mask_data in regions.items():
logger.info(f"Processing mask for region: {region_name}")
# Create base mask
base_mask = self.create_structural_mask(image_path, mask_data)
# Grow mask to prevent edge bleeding
grown_mask = self.grow_mask(base_mask)
# Feather edges
feathered_mask = self.feather_mask(grown_mask)
# Cache the processed mask
cache_path = self.save_to_cache(
image_path,
feathered_mask,
suffix=f"_{region_name}"
)
processed_masks[region_name] = {
"mask": feathered_mask,
"cache_path": cache_path
}
# Create combined structural mask
all_structural = [m["mask"] for m in processed_masks.values()]
combined_structural = self.combine_masks(all_structural, operation="union")
# Create stylable mask (inverse of structural)
stylable_mask = self.invert_mask(combined_structural)
# Save combined masks
structural_cache = self.save_to_cache(
image_path,
combined_structural,
suffix="_structural"
)
stylable_cache = self.save_to_cache(
image_path,
stylable_mask,
suffix="_stylable"
)
processed_masks["combined_structural"] = {
"mask": combined_structural,
"cache_path": structural_cache
}
processed_masks["stylable"] = {
"mask": stylable_mask,
"cache_path": stylable_cache
}
return processed_masks
def preprocess_image(self, image_path: str) -> Dict:
"""
Complete preprocessing pipeline for a single image.
Returns metadata about generated masks.
"""
logger.info(f"Preprocessing image: {image_path}")
# Check if already cached
if self.is_cached(image_path, "_structural"):
logger.info(f"Image already preprocessed: {image_path}")
return {
"image_path": image_path,
"cached": True,
"masks": {
"structural": str(self._get_cache_path(image_path, "_structural")),
"stylable": str(self._get_cache_path(image_path, "_stylable"))
}
}
# Load image for reference
img = cv2.imread(image_path)
if img is None:
raise ValueError(f"Could not load image: {image_path}")
height, width = img.shape[:2]
# Create placeholder masks (in production, these would come from SAM)
# This simulates wall, floor, ceiling segmentation
regions = {}
# Wall mask (upper portion)
wall_mask = np.zeros((height, width), dtype=np.uint8)
wall_mask[0:int(height*0.6), :] = 255
regions["wall"] = wall_mask
# Floor mask (lower portion)
floor_mask = np.zeros((height, width), dtype=np.uint8)
floor_mask[int(height*0.6):, :] = 255
regions["floor"] = floor_mask
# Ceiling mask (top portion)
ceiling_mask = np.zeros((height, width), dtype=np.uint8)
ceiling_mask[0:int(height*0.15), :] = 255
regions["ceiling"] = ceiling_mask
# Process all regions
processed = self.create_multi_region_mask(image_path, regions)
return {
"image_path": image_path,
"cached": False,
"dimensions": (width, height),
"masks": {
name: data["cache_path"]
for name, data in processed.items()
}
}
def batch_preprocess(self, directory: str, pattern: str = "*.jpg") -> List[Dict]:
"""Preprocess all images in a directory."""
input_dir = Path(directory)
image_files = list(input_dir.glob(pattern))
image_files.extend(list(input_dir.glob("*.png")))
results = []
for img_file in image_files:
try:
result = self.preprocess_image(str(img_file))
results.append(result)
except Exception as e:
logger.error(f"Failed to preprocess {img_file}: {e}")
return results
def clear_cache(self):
"""Clear all cached masks."""
for cache_file in self.cache_dir.glob("*.png"):
cache_file.unlink()
logger.info("Cache cleared")
def get_cache_stats(self) -> Dict:
"""Get cache statistics."""
cache_files = list(self.cache_dir.glob("*.png"))
total_size = sum(f.stat().st_size for f in cache_files)
return {
"cached_files": len(cache_files),
"total_size_mb": total_size / (1024 * 1024),
"cache_directory": str(self.cache_dir)
}
def main():
"""Main entry point for command-line usage."""
parser = argparse.ArgumentParser(
description="Dream Weaver Mask Preprocessor"
)
parser.add_argument(
"--image",
type=str,
help="Single image to preprocess"
)
parser.add_argument(
"--directory",
type=str,
help="Directory of images to preprocess"
)
parser.add_argument(
"--cache-dir",
type=str,
default="Project_Velocity/comfy_engine/cache/masks/",
help="Cache directory for masks"
)
parser.add_argument(
"--grow",
type=int,
default=3,
help="Pixels to grow mask (dilation)"
)
parser.add_argument(
"--feather",
type=int,
default=5,
help="Pixels to feather mask edges"
)
parser.add_argument(
"--clear-cache",
action="store_true",
help="Clear all cached masks"
)
parser.add_argument(
"--stats",
action="store_true",
help="Show cache statistics"
)
args = parser.parse_args()
# Initialize preprocessor
preprocessor = MaskPreprocessor(cache_dir=args.cache_dir)
preprocessor.config.grow_pixels = args.grow
preprocessor.config.feather_pixels = args.feather
if args.clear_cache:
preprocessor.clear_cache()
return
if args.stats:
stats = preprocessor.get_cache_stats()
print(json.dumps(stats, indent=2))
return
if args.image:
result = preprocessor.preprocess_image(args.image)
print(json.dumps(result, indent=2))
elif args.directory:
results = preprocessor.batch_preprocess(args.directory)
print(json.dumps(results, indent=2))
print(f"\nProcessed {len(results)} images")
else:
print("No action specified. Use --help for usage information.")
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""
Dream Weaver Mask Preprocessor
==============================
Utility script for preprocessing and caching segmentation masks.
Enables offline mask generation to speed up production workflows.
Target Hardware: Dual NVIDIA RTX PRO 6000 Blackwell
Author: Project Velocity Team
Version: 1.0.0
"""
import os
import sys
import json
import hashlib
import argparse
import logging
from pathlib import Path
from typing import List, Optional, Tuple, Dict
from dataclasses import dataclass
import numpy as np
from PIL import Image
import cv2
# Configure logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger('MaskPreprocessor')
@dataclass
class MaskConfig:
"""Configuration for mask generation."""
grow_pixels: int = 3
feather_pixels: int = 5
threshold: float = 0.3
target_classes: List[str] = None
def __post_init__(self):
if self.target_classes is None:
self.target_classes = ["wall", "floor", "ceiling"]
class MaskPreprocessor:
"""Preprocesses and caches segmentation masks for Dream Weaver."""
def __init__(self, cache_dir: str = "Project_Velocity/comfy_engine/cache/masks/"):
self.cache_dir = Path(cache_dir)
self.cache_dir.mkdir(parents=True, exist_ok=True)
self.config = MaskConfig()
logger.info(f"MaskPreprocessor initialized. Cache directory: {self.cache_dir}")
def _get_image_hash(self, image_path: str) -> str:
"""Generate MD5 hash of image for caching."""
hasher = hashlib.md5()
with open(image_path, 'rb') as f:
hasher.update(f.read())
return hasher.hexdigest()
def _get_cache_path(self, image_path: str, suffix: str = "") -> Path:
"""Generate cache file path for an image."""
image_hash = self._get_image_hash(image_path)
filename = f"{image_hash}{suffix}.png"
return self.cache_dir / filename
def is_cached(self, image_path: str, suffix: str = "") -> bool:
"""Check if a mask is already cached."""
cache_path = self._get_cache_path(image_path, suffix)
return cache_path.exists()
def load_from_cache(self, image_path: str, suffix: str = "") -> Optional[np.ndarray]:
"""Load mask from cache if available."""
cache_path = self._get_cache_path(image_path, suffix)
if cache_path.exists():
logger.info(f"Loading cached mask from {cache_path}")
mask = cv2.imread(str(cache_path), cv2.IMREAD_GRAYSCALE)
return mask
return None
def save_to_cache(self, image_path: str, mask: np.ndarray, suffix: str = "") -> str:
"""Save mask to cache."""
cache_path = self._get_cache_path(image_path, suffix)
cv2.imwrite(str(cache_path), mask)
logger.info(f"Saved mask to cache: {cache_path}")
return str(cache_path)
def create_structural_mask(self, image_path: str, mask_data: np.ndarray) -> np.ndarray:
"""
Create a structural preservation mask from segmentation data.
This mask identifies walls, floors, ceilings that must be preserved.
"""
# Ensure binary mask
if len(mask_data.shape) == 3:
mask_data = cv2.cvtColor(mask_data, cv2.COLOR_BGR2GRAY)
_, binary_mask = cv2.threshold(
mask_data,
int(255 * self.config.threshold),
255,
cv2.THRESH_BINARY
)
return binary_mask.astype(np.uint8)
def grow_mask(self, mask: np.ndarray, pixels: int = None) -> np.ndarray:
"""
Grow (dilate) the mask by specified pixels.
This prevents edge bleeding by expanding the mask slightly.
"""
if pixels is None:
pixels = self.config.grow_pixels
kernel = np.ones((pixels * 2 + 1, pixels * 2 + 1), np.uint8)
grown_mask = cv2.dilate(mask, kernel, iterations=1)
return grown_mask
def feather_mask(self, mask: np.ndarray, pixels: int = None) -> np.ndarray:
"""
Apply Gaussian blur to feather mask edges.
Creates smooth transitions at boundaries.
"""
if pixels is None:
pixels = self.config.feather_pixels
# Ensure odd kernel size
kernel_size = pixels * 2 + 1
feathered = cv2.GaussianBlur(mask, (kernel_size, kernel_size), 0)
return feathered
def invert_mask(self, mask: np.ndarray) -> np.ndarray:
"""Invert mask (structural -> stylable or vice versa)."""
return cv2.bitwise_not(mask)
def combine_masks(self, masks: List[np.ndarray], operation: str = "union") -> np.ndarray:
"""
Combine multiple masks.
operation: 'union' (OR), 'intersection' (AND), 'difference'
"""
if not masks:
return None
result = masks[0].copy()
for mask in masks[1:]:
if operation == "union":
result = cv2.bitwise_or(result, mask)
elif operation == "intersection":
result = cv2.bitwise_and(result, mask)
elif operation == "difference":
result = cv2.bitwise_and(result, cv2.bitwise_not(mask))
return result
def create_multi_region_mask(
self,
image_path: str,
regions: Dict[str, np.ndarray]
) -> Dict[str, np.ndarray]:
"""
Create masks for multiple regions (walls, floor, ceiling, etc.)
Returns dictionary of processed masks.
"""
processed_masks = {}
for region_name, mask_data in regions.items():
logger.info(f"Processing mask for region: {region_name}")
# Create base mask
base_mask = self.create_structural_mask(image_path, mask_data)
# Grow mask to prevent edge bleeding
grown_mask = self.grow_mask(base_mask)
# Feather edges
feathered_mask = self.feather_mask(grown_mask)
# Cache the processed mask
cache_path = self.save_to_cache(
image_path,
feathered_mask,
suffix=f"_{region_name}"
)
processed_masks[region_name] = {
"mask": feathered_mask,
"cache_path": cache_path
}
# Create combined structural mask
all_structural = [m["mask"] for m in processed_masks.values()]
combined_structural = self.combine_masks(all_structural, operation="union")
# Create stylable mask (inverse of structural)
stylable_mask = self.invert_mask(combined_structural)
# Save combined masks
structural_cache = self.save_to_cache(
image_path,
combined_structural,
suffix="_structural"
)
stylable_cache = self.save_to_cache(
image_path,
stylable_mask,
suffix="_stylable"
)
processed_masks["combined_structural"] = {
"mask": combined_structural,
"cache_path": structural_cache
}
processed_masks["stylable"] = {
"mask": stylable_mask,
"cache_path": stylable_cache
}
return processed_masks
def preprocess_image(self, image_path: str) -> Dict:
"""
Complete preprocessing pipeline for a single image.
Returns metadata about generated masks.
"""
logger.info(f"Preprocessing image: {image_path}")
# Check if already cached
if self.is_cached(image_path, "_structural"):
logger.info(f"Image already preprocessed: {image_path}")
return {
"image_path": image_path,
"cached": True,
"masks": {
"structural": str(self._get_cache_path(image_path, "_structural")),
"stylable": str(self._get_cache_path(image_path, "_stylable"))
}
}
# Load image for reference
img = cv2.imread(image_path)
if img is None:
raise ValueError(f"Could not load image: {image_path}")
height, width = img.shape[:2]
# Create placeholder masks (in production, these would come from SAM)
# This simulates wall, floor, ceiling segmentation
regions = {}
# Wall mask (upper portion)
wall_mask = np.zeros((height, width), dtype=np.uint8)
wall_mask[0:int(height*0.6), :] = 255
regions["wall"] = wall_mask
# Floor mask (lower portion)
floor_mask = np.zeros((height, width), dtype=np.uint8)
floor_mask[int(height*0.6):, :] = 255
regions["floor"] = floor_mask
# Ceiling mask (top portion)
ceiling_mask = np.zeros((height, width), dtype=np.uint8)
ceiling_mask[0:int(height*0.15), :] = 255
regions["ceiling"] = ceiling_mask
# Process all regions
processed = self.create_multi_region_mask(image_path, regions)
return {
"image_path": image_path,
"cached": False,
"dimensions": (width, height),
"masks": {
name: data["cache_path"]
for name, data in processed.items()
}
}
def batch_preprocess(self, directory: str, pattern: str = "*.jpg") -> List[Dict]:
"""Preprocess all images in a directory."""
input_dir = Path(directory)
image_files = list(input_dir.glob(pattern))
image_files.extend(list(input_dir.glob("*.png")))
results = []
for img_file in image_files:
try:
result = self.preprocess_image(str(img_file))
results.append(result)
except Exception as e:
logger.error(f"Failed to preprocess {img_file}: {e}")
return results
def clear_cache(self):
"""Clear all cached masks."""
for cache_file in self.cache_dir.glob("*.png"):
cache_file.unlink()
logger.info("Cache cleared")
def get_cache_stats(self) -> Dict:
"""Get cache statistics."""
cache_files = list(self.cache_dir.glob("*.png"))
total_size = sum(f.stat().st_size for f in cache_files)
return {
"cached_files": len(cache_files),
"total_size_mb": total_size / (1024 * 1024),
"cache_directory": str(self.cache_dir)
}
def main():
"""Main entry point for command-line usage."""
parser = argparse.ArgumentParser(
description="Dream Weaver Mask Preprocessor"
)
parser.add_argument(
"--image",
type=str,
help="Single image to preprocess"
)
parser.add_argument(
"--directory",
type=str,
help="Directory of images to preprocess"
)
parser.add_argument(
"--cache-dir",
type=str,
default="Project_Velocity/comfy_engine/cache/masks/",
help="Cache directory for masks"
)
parser.add_argument(
"--grow",
type=int,
default=3,
help="Pixels to grow mask (dilation)"
)
parser.add_argument(
"--feather",
type=int,
default=5,
help="Pixels to feather mask edges"
)
parser.add_argument(
"--clear-cache",
action="store_true",
help="Clear all cached masks"
)
parser.add_argument(
"--stats",
action="store_true",
help="Show cache statistics"
)
args = parser.parse_args()
# Initialize preprocessor
preprocessor = MaskPreprocessor(cache_dir=args.cache_dir)
preprocessor.config.grow_pixels = args.grow
preprocessor.config.feather_pixels = args.feather
if args.clear_cache:
preprocessor.clear_cache()
return
if args.stats:
stats = preprocessor.get_cache_stats()
print(json.dumps(stats, indent=2))
return
if args.image:
result = preprocessor.preprocess_image(args.image)
print(json.dumps(result, indent=2))
elif args.directory:
results = preprocessor.batch_preprocess(args.directory)
print(json.dumps(results, indent=2))
print(f"\nProcessed {len(results)} images")
else:
print("No action specified. Use --help for usage information.")
if __name__ == "__main__":
main()

412
comfy_engine/scripts/prompt_expander.py
View File

@@ -1,206 +1,206 @@
#!/usr/bin/env python3
"""
Dream Weaver — Local LLM Prompt Expander
========================================
Converts user keywords + room type into a photorealistic interior design prompt
using a local Ollama model (default: qwen3.5:27b).
Cloud API calls (Gemini, OpenAI) have been completely removed for data privacy
and local inference requirements.
Usage:
from prompt_expander import expand_prompt
result = expand_prompt(
keywords=["blue marble", "gold veins", "renaissance", "sharp contours"],
room_type="bedroom"
)
"""
import os
import json
import logging
import requests
import re
logger = logging.getLogger(__name__)
# ── Room-type context injected into every LLM call ───────────────────────────
ROOM_CONTEXTS = {
"bedroom": {
"description": "a private sleeping space",
"key_elements": ["bed", "bedside tables", "wardrobe", "soft lighting", "textiles", "headboard"],
"must_haves": "bed linen, pillows, bedside lighting",
"avoid": "office furniture, dining elements, cooking equipment"
},
"living_room": {
"description": "a social gathering and relaxation space",
"key_elements": ["sofa", "coffee table", "TV unit", "accent chairs", "rugs"],
"must_haves": "seating arrangement, focal point",
"avoid": "beds, cooking equipment, clinical elements"
},
"bathroom": {
"description": "a private hygiene and wellness space",
"key_elements": ["vanity", "bathtub", "shower", "tiles", "mirrors"],
"must_haves": "wet-area materials, luxury fixtures",
"avoid": "soft furnishings, carpet, beds"
},
"kitchen": {
"description": "a functional cooking space",
"key_elements": ["cabinetry", "countertops", "backsplash", "appliances", "island"],
"must_haves": "work surfaces, storage",
"avoid": "beds, lounge furniture"
},
"dining_room": {
"description": "an eating and entertaining space",
"key_elements": ["dining table", "chairs", "sideboard", "pendant lighting"],
"must_haves": "central dining table, seating",
"avoid": "beds, cooking appliances"
},
"home_office": {
"description": "a workspace within a home",
"key_elements": ["desk", "ergonomic chair", "shelving", "task lighting"],
"must_haves": "functional desk setup",
"avoid": "beds in foreground, dining furniture"
},
"hallway": {
"description": "an entrance or transitional corridor",
"key_elements": ["console table", "mirror", "coat storage", "lighting"],
"must_haves": "welcoming entrance elements",
"avoid": "beds, large seating"
},
"balcony": {
"description": "an outdoor living extension",
"key_elements": ["outdoor furniture", "planters", "lighting", "railings"],
"must_haves": "weather-resistant materials",
"avoid": "indoor bedding, non-weather-resistant elements"
},
}
FEW_SHOT_EXAMPLES = """
EXAMPLE 1:
Keywords: ["light oak", "white walls", "hygge", "natural light", "minimalist"]
Room type: bedroom
Positive prompt: scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles, abundant natural light streaming through large windows, clean white walls, simple functional furniture, cozy hygge atmosphere, soft cream and warm gray tones, organic cotton fabrics, potted green plants, minimalist pendant lighting, decluttered space, architectural photography, 8k resolution, photorealistic, global illumination, soft shadows, natural materials, sustainable design
Negative prompt: (worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, heavy ornamentation, dark colors, cluttered space
EXAMPLE 2:
Keywords: ["gold brass", "marble", "velvet", "emerald green", "1920s", "geometric"]
Room type: living_room
Positive prompt: art deco luxury interior design, geometric chevron patterns, gold brass accents, rich velvet upholstery in emerald green and sapphire blue, sunburst mirrors, polished marble flooring with brass inlay, crystal chandeliers, lacquered wood furniture, bold symmetrical arrangements, 1920s glamour, warm ambient lighting, architectural photography, 8k resolution, photorealistic, global illumination, elegant reflections, geometric motifs, stepped forms
Negative prompt: (worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, rustic elements, farmhouse style, minimalism, cheap materials
"""
SYSTEM_PROMPT = """You are Dream Weaver's interior design prompt engineer. Convert user-provided keywords and a room type into a high-quality prompt for image generation.
TASK:
Generate JSON containing:
1. "positive_prompt" (rich, photorealistic, 80-120 words)
2. "negative_prompt" (preventing artifacts, 30-50 words)
3. "cfg" (float 6.0-9.0)
4. "denoise" (float 0.5-0.85)
5. "steps" (int 25-40)
RULES FOR POSITIVE PROMPT:
- Focus on the core aesthetic derived from keywords
- Include architecture, furniture, and lighting suitable for the room type
- End with: "architectural photography, 8k resolution, photorealistic"
RULES FOR NEGATIVE PROMPT:
- Start with: (worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes
OUTPUT FORMAT:
Provide valid JSON only, with keys: "style_name", "positive_prompt", "negative_prompt", "cfg", "denoise", "steps", "reasoning".
FEW-SHOT EXAMPLES:
""" + FEW_SHOT_EXAMPLES
class ExpandedPrompt:
def __init__(self, style_name, positive_prompt, negative_prompt, cfg, denoise, steps, reasoning, source):
self.style_name = style_name
self.positive_prompt = positive_prompt
self.negative_prompt = negative_prompt
self.cfg = cfg
self.denoise = denoise
self.steps = steps
self.reasoning = reasoning
self.source = source
def _call_ollama(user_message: str) -> str:
ollama_url = os.environ.get("OLLAMA_URL", "http://localhost:11434")
# Using Qwen 3.5 27B as requested
model = os.environ.get("OLLAMA_MODEL", "qwen3.5:27b")
full_prompt = f"{SYSTEM_PROMPT}\n\nUSER REQUEST:\n{user_message}\n\nReturn JSON ONLY. No markdown wrapping."
r = requests.post(
f"{ollama_url}/api/generate",
json={
"model": model,
"prompt": full_prompt,
"stream": False,
"format": "json",
"options": {"temperature": 0.5}
},
timeout=180 # Large models take time
)
r.raise_for_status()
return r.json()["response"]
def expand_prompt(keywords: list[str], room_type: str = "living_room", additional_notes: str = "") -> ExpandedPrompt:
if not keywords:
raise ValueError("Keywords required")
room_type = room_type.lower().replace(" ", "_")
if room_type not in ROOM_CONTEXTS:
room_type = "living_room"
ctx = ROOM_CONTEXTS[room_type]
user_message = f"""KEYWORDS: {', '.join(keywords)}
ROOM TYPE: {room_type} ({ctx['description']})
MUST HAVE: {ctx['must_haves']}
AVOID: {ctx['avoid']}
{f'NOTES: {additional_notes}' if additional_notes else ''}"""
try:
logger.info("Calling local Ollama LLM...")
raw = _call_ollama(user_message).strip()
json_match = re.search(r'\{[\s\S]*\}', raw)
if json_match:
raw_json = json_match.group(0)
else:
raw_json = raw
data = json.loads(raw_json)
return ExpandedPrompt(
style_name=data.get("style_name", "custom_local"),
positive_prompt=data["positive_prompt"],
negative_prompt=data["negative_prompt"],
cfg=float(data.get("cfg", 7.5)),
denoise=float(data.get("denoise", 0.72)),
steps=int(data.get("steps", 30)),
reasoning=data.get("reasoning", ""),
source="ollama_local"
)
except Exception as e:
logger.warning(f"Ollama failed, using sync fallback: {e}")
return expand_prompt_simple(keywords, room_type)
def expand_prompt_simple(keywords: list[str], room_type: str = "living_room") -> ExpandedPrompt:
ctx = ROOM_CONTEXTS.get(room_type.replace(" ", "_"), ROOM_CONTEXTS["living_room"])
kw_str = ", ".join(keywords)
positive = f"{kw_str} interior design, {', '.join(ctx['key_elements'][:4])}, photorealistic {room_type.replace('_', ' ')} interior, architectural photography, 8k resolution, photorealistic"
negative = "(worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, extra windows, unrealistic lighting, structural changes"
return ExpandedPrompt(
style_name="fallback", positive_prompt=positive, negative_prompt=negative,
cfg=7.5, denoise=0.72, steps=30, reasoning="No LLM", source="fallback"
)
if __name__ == "__main__":
import sys
logging.basicConfig(level=logging.INFO)
ans = expand_prompt(["blue marble", "gold"], "bathroom")
print(ans.positive_prompt)
#!/usr/bin/env python3
"""
Dream Weaver — Local LLM Prompt Expander
========================================
Converts user keywords + room type into a photorealistic interior design prompt
using a local Ollama model (default: qwen3.5:27b).
Cloud API calls (Gemini, OpenAI) have been completely removed for data privacy
and local inference requirements.
Usage:
from prompt_expander import expand_prompt
result = expand_prompt(
keywords=["blue marble", "gold veins", "renaissance", "sharp contours"],
room_type="bedroom"
)
"""
import os
import json
import logging
import requests
import re
logger = logging.getLogger(__name__)
# ── Room-type context injected into every LLM call ───────────────────────────
ROOM_CONTEXTS = {
"bedroom": {
"description": "a private sleeping space",
"key_elements": ["bed", "bedside tables", "wardrobe", "soft lighting", "textiles", "headboard"],
"must_haves": "bed linen, pillows, bedside lighting",
"avoid": "office furniture, dining elements, cooking equipment"
},
"living_room": {
"description": "a social gathering and relaxation space",
"key_elements": ["sofa", "coffee table", "TV unit", "accent chairs", "rugs"],
"must_haves": "seating arrangement, focal point",
"avoid": "beds, cooking equipment, clinical elements"
},
"bathroom": {
"description": "a private hygiene and wellness space",
"key_elements": ["vanity", "bathtub", "shower", "tiles", "mirrors"],
"must_haves": "wet-area materials, luxury fixtures",
"avoid": "soft furnishings, carpet, beds"
},
"kitchen": {
"description": "a functional cooking space",
"key_elements": ["cabinetry", "countertops", "backsplash", "appliances", "island"],
"must_haves": "work surfaces, storage",
"avoid": "beds, lounge furniture"
},
"dining_room": {
"description": "an eating and entertaining space",
"key_elements": ["dining table", "chairs", "sideboard", "pendant lighting"],
"must_haves": "central dining table, seating",
"avoid": "beds, cooking appliances"
},
"home_office": {
"description": "a workspace within a home",
"key_elements": ["desk", "ergonomic chair", "shelving", "task lighting"],
"must_haves": "functional desk setup",
"avoid": "beds in foreground, dining furniture"
},
"hallway": {
"description": "an entrance or transitional corridor",
"key_elements": ["console table", "mirror", "coat storage", "lighting"],
"must_haves": "welcoming entrance elements",
"avoid": "beds, large seating"
},
"balcony": {
"description": "an outdoor living extension",
"key_elements": ["outdoor furniture", "planters", "lighting", "railings"],
"must_haves": "weather-resistant materials",
"avoid": "indoor bedding, non-weather-resistant elements"
},
}
FEW_SHOT_EXAMPLES = """
EXAMPLE 1:
Keywords: ["light oak", "white walls", "hygge", "natural light", "minimalist"]
Room type: bedroom
Positive prompt: scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles, abundant natural light streaming through large windows, clean white walls, simple functional furniture, cozy hygge atmosphere, soft cream and warm gray tones, organic cotton fabrics, potted green plants, minimalist pendant lighting, decluttered space, architectural photography, 8k resolution, photorealistic, global illumination, soft shadows, natural materials, sustainable design
Negative prompt: (worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, heavy ornamentation, dark colors, cluttered space
EXAMPLE 2:
Keywords: ["gold brass", "marble", "velvet", "emerald green", "1920s", "geometric"]
Room type: living_room
Positive prompt: art deco luxury interior design, geometric chevron patterns, gold brass accents, rich velvet upholstery in emerald green and sapphire blue, sunburst mirrors, polished marble flooring with brass inlay, crystal chandeliers, lacquered wood furniture, bold symmetrical arrangements, 1920s glamour, warm ambient lighting, architectural photography, 8k resolution, photorealistic, global illumination, elegant reflections, geometric motifs, stepped forms
Negative prompt: (worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, rustic elements, farmhouse style, minimalism, cheap materials
"""
SYSTEM_PROMPT = """You are Dream Weaver's interior design prompt engineer. Convert user-provided keywords and a room type into a high-quality prompt for image generation.
TASK:
Generate JSON containing:
1. "positive_prompt" (rich, photorealistic, 80-120 words)
2. "negative_prompt" (preventing artifacts, 30-50 words)
3. "cfg" (float 6.0-9.0)
4. "denoise" (float 0.5-0.85)
5. "steps" (int 25-40)
RULES FOR POSITIVE PROMPT:
- Focus on the core aesthetic derived from keywords
- Include architecture, furniture, and lighting suitable for the room type
- End with: "architectural photography, 8k resolution, photorealistic"
RULES FOR NEGATIVE PROMPT:
- Start with: (worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes
OUTPUT FORMAT:
Provide valid JSON only, with keys: "style_name", "positive_prompt", "negative_prompt", "cfg", "denoise", "steps", "reasoning".
FEW-SHOT EXAMPLES:
""" + FEW_SHOT_EXAMPLES
class ExpandedPrompt:
def __init__(self, style_name, positive_prompt, negative_prompt, cfg, denoise, steps, reasoning, source):
self.style_name = style_name
self.positive_prompt = positive_prompt
self.negative_prompt = negative_prompt
self.cfg = cfg
self.denoise = denoise
self.steps = steps
self.reasoning = reasoning
self.source = source
def _call_ollama(user_message: str) -> str:
ollama_url = os.environ.get("OLLAMA_URL", "http://localhost:11434")
# Using Qwen 3.5 27B as requested
model = os.environ.get("OLLAMA_MODEL", "qwen3.5:27b")
full_prompt = f"{SYSTEM_PROMPT}\n\nUSER REQUEST:\n{user_message}\n\nReturn JSON ONLY. No markdown wrapping."
r = requests.post(
f"{ollama_url}/api/generate",
json={
"model": model,
"prompt": full_prompt,
"stream": False,
"format": "json",
"options": {"temperature": 0.5}
},
timeout=180 # Large models take time
)
r.raise_for_status()
return r.json()["response"]
def expand_prompt(keywords: list[str], room_type: str = "living_room", additional_notes: str = "") -> ExpandedPrompt:
if not keywords:
raise ValueError("Keywords required")
room_type = room_type.lower().replace(" ", "_")
if room_type not in ROOM_CONTEXTS:
room_type = "living_room"
ctx = ROOM_CONTEXTS[room_type]
user_message = f"""KEYWORDS: {', '.join(keywords)}
ROOM TYPE: {room_type} ({ctx['description']})
MUST HAVE: {ctx['must_haves']}
AVOID: {ctx['avoid']}
{f'NOTES: {additional_notes}' if additional_notes else ''}"""
try:
logger.info("Calling local Ollama LLM...")
raw = _call_ollama(user_message).strip()
json_match = re.search(r'\{[\s\S]*\}', raw)
if json_match:
raw_json = json_match.group(0)
else:
raw_json = raw
data = json.loads(raw_json)
return ExpandedPrompt(
style_name=data.get("style_name", "custom_local"),
positive_prompt=data["positive_prompt"],
negative_prompt=data["negative_prompt"],
cfg=float(data.get("cfg", 7.5)),
denoise=float(data.get("denoise", 0.72)),
steps=int(data.get("steps", 30)),
reasoning=data.get("reasoning", ""),
source="ollama_local"
)
except Exception as e:
logger.warning(f"Ollama failed, using sync fallback: {e}")
return expand_prompt_simple(keywords, room_type)
def expand_prompt_simple(keywords: list[str], room_type: str = "living_room") -> ExpandedPrompt:
ctx = ROOM_CONTEXTS.get(room_type.replace(" ", "_"), ROOM_CONTEXTS["living_room"])
kw_str = ", ".join(keywords)
positive = f"{kw_str} interior design, {', '.join(ctx['key_elements'][:4])}, photorealistic {room_type.replace('_', ' ')} interior, architectural photography, 8k resolution, photorealistic"
negative = "(worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, extra windows, unrealistic lighting, structural changes"
return ExpandedPrompt(
style_name="fallback", positive_prompt=positive, negative_prompt=negative,
cfg=7.5, denoise=0.72, steps=30, reasoning="No LLM", source="fallback"
)
if __name__ == "__main__":
import sys
logging.basicConfig(level=logging.INFO)
ans = expand_prompt(["blue marble", "gold"], "bathroom")
print(ans.positive_prompt)

2
iOS/velocity/velocity/Core/Config/AppConfig.swift
View File

@@ -11,6 +11,6 @@ enum AppConfig {
!override.isEmpty, override != "$(BASE_URL)" {
return override
}
return "http://54.172.172.2:8080"
return "http://54.91.19.60:8082"
}()
}

66
iOS/velocity/velocity/Core/Math/SunMath.swift
View File

@@ -2,11 +2,14 @@ import CoreLocation
import Foundation
struct SunPosition {
let azimuth: Double // 0...360, degrees clockwise from true north
let azimuth: Double // 0...360, degrees clockwise from true north
let elevation: Double // -90...90 degrees above horizon
}
enum SunMath {
// MARK: - Single Position
static func calculateSunPosition(date: Date, coordinate: CLLocationCoordinate2D) -> SunPosition {
let timezone = TimeZone.current
let localOffsetHours = Double(timezone.secondsFromGMT(for: date)) / 3600.0
@@ -14,7 +17,7 @@ enum SunMath {
let n = julianDay - 2_451_545.0
let meanLongitude = normalizeDegrees(280.46 + 0.985_647_4 * n)
let meanAnomaly = normalizeDegrees(357.528 + 0.985_600_3 * n)
let meanAnomaly = normalizeDegrees(357.528 + 0.985_600_3 * n)
let lambda = meanLongitude
+ 1.915 * sin(meanAnomaly.radians)
@@ -32,9 +35,9 @@ enum SunMath {
let hourAngle = normalizeDegrees(lst - rightAscension)
let signedHourAngle = hourAngle > 180.0 ? hourAngle - 360.0 : hourAngle
let latitude = coordinate.latitude.radians
let latitude = coordinate.latitude.radians
let declinationRad = declination.radians
let hourAngleRad = signedHourAngle.radians
let hourAngleRad = signedHourAngle.radians
let elevation = asin(
sin(latitude) * sin(declinationRad)
@@ -51,11 +54,13 @@ enum SunMath {
return SunPosition(azimuth: azimuth, elevation: elevation)
}
// MARK: - Hourly Arc (used by legacy code & DashedSunLine)
/// 5-sample dictionary kept for backward compat with the Dollhouse slider.
static func sunPathSamples(for date: Date, coordinate: CLLocationCoordinate2D) -> [Date: SunPosition] {
let calendar = Calendar.current
let sampleHours = [8, 10, 12, 14, 16]
var output: [Date: SunPosition] = [:]
for hour in sampleHours {
if let sampleDate = calendar.date(bySettingHour: hour, minute: 0, second: 0, of: date) {
output[sampleDate] = calculateSunPosition(date: sampleDate, coordinate: coordinate)
@@ -64,25 +69,62 @@ enum SunMath {
return output
}
private static func normalizeDegrees(_ value: Double) -> Double {
/// Dense arc for the AR overlay — one sample per hour from 4 AM to 8 PM.
/// Filters out below-horizon positions (elevation < -5°).
static func sunPathArc(for date: Date, coordinate: CLLocationCoordinate2D) -> [(date: Date, position: SunPosition)] {
let calendar = Calendar.current
var result: [(Date, SunPosition)] = []
for hour in 4...20 {
guard let sampleDate = calendar.date(bySettingHour: hour, minute: 0, second: 0, of: date) else { continue }
let pos = calculateSunPosition(date: sampleDate, coordinate: coordinate)
// include a small below-horizon buffer so arc starts/ends smoothly
if pos.elevation > -5 {
result.append((sampleDate, pos))
}
}
return result
}
/// Approximate sunrise and sunset by scanning for elevation sign changes.
static func sunRiseSet(for date: Date, coordinate: CLLocationCoordinate2D) -> (rise: Date?, set: Date?) {
let calendar = Calendar.current
var rise: Date? = nil
var set: Date? = nil
var prevElevation: Double? = nil
var prevDate: Date? = nil
for minuteOffset in stride(from: 0, through: 24 * 60, by: 10) {
guard let sampleDate = calendar.date(byAdding: .minute, value: minuteOffset, to: calendar.startOfDay(for: date)) else { continue }
let pos = calculateSunPosition(date: sampleDate, coordinate: coordinate)
if let prev = prevElevation, let prevD = prevDate {
if prev < 0 && pos.elevation >= 0 { rise = prevD }
if prev >= 0 && pos.elevation < 0 { set = prevD }
}
prevElevation = pos.elevation
prevDate = sampleDate
}
return (rise, set)
}
// MARK: - Helpers
static func normalizeDegrees(_ value: Double) -> Double {
let reduced = value.truncatingRemainder(dividingBy: 360.0)
return reduced >= 0 ? reduced : reduced + 360.0
}
}
// MARK: - Date helpers
private extension Date {
var utcHours: Double {
let calendar = Calendar(identifier: .gregorian)
let comps = calendar.dateComponents(in: TimeZone(secondsFromGMT: 0)!, from: self)
let hours = Double(comps.hour ?? 0)
let minutes = Double(comps.minute ?? 0)
let seconds = Double(comps.second ?? 0)
return hours + minutes / 60.0 + seconds / 3600.0
return Double(comps.hour ?? 0) + Double(comps.minute ?? 0) / 60.0 + Double(comps.second ?? 0) / 3600.0
}
var julianDay: Double {
let interval = timeIntervalSince1970
return (interval / 86_400.0) + 2_440_587.5
timeIntervalSince1970 / 86_400.0 + 2_440_587.5
}
}

23
iOS/velocity/velocity/Core/Networking/ComfyClient.swift
View File

@@ -17,7 +17,9 @@ final class ComfyClient {
/// Returns `true` if `{ "status": "ok" }`.
func checkHealth() async -> Bool {
guard let url = URL(string: "\(baseURL)/health") else { return false }
guard let (data, _) = try? await URLSession.shared.data(from: url),
var request = URLRequest(url: url)
request.timeoutInterval = 30.0
guard let (data, _) = try? await URLSession.shared.data(for: request),
let json = try? JSONDecoder().decode(HealthResponse.self, from: data) else {
return false
}
@@ -30,9 +32,9 @@ final class ComfyClient {
/// Full pipeline: upload → queue → poll → download.
/// - Parameters:
/// - source: Room photo from camera or library.
/// - style: One of `scandinavian`, `art_deco`, `biophilic`, `cyberpunk`, `japandi`.
/// - roomType: The type of room being designed (e.g. "bedroom", "living_room").
/// - keywords: Comma-separated user keywords appended to the style prompt (can be empty).
func generateImage(source: UIImage, style: String, keywords: String) async throws -> UIImage {
func generateImage(source: UIImage, roomType: String, keywords: String) async throws -> UIImage {
let normalised = source.fixedOrientation()
let resized = normalised.resizedSquare(to: 1024)
guard let imageData = resized.jpegData(compressionQuality: 0.85) else {
@@ -40,7 +42,7 @@ final class ComfyClient {
}
// 1. Submit job → get job_id
let job = try await submitJob(imageData: imageData, style: style, keywords: keywords)
let job = try await submitJob(imageData: imageData, roomType: roomType, keywords: keywords)
// 2. Poll status every 2s until ready (max 5 min per integration guide §3.3)
let resultURL = try await pollUntilReady(jobId: job.jobId)
@@ -51,7 +53,7 @@ final class ComfyClient {
// MARK: - Step 1: POST /dream-weaver
private func submitJob(imageData: Data, style: String, keywords: String) async throws -> GenerationJob {
private func submitJob(imageData: Data, roomType: String, keywords: String) async throws -> GenerationJob {
guard let url = URL(string: "\(baseURL)/dream-weaver") else {
throw DreamWeaverError.generationFailed("Invalid gateway URL")
}
@@ -60,9 +62,10 @@ final class ComfyClient {
var request = URLRequest(url: url)
request.httpMethod = "POST"
request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")
request.timeoutInterval = 180.0
request.httpBody = buildMultipart(
imageData: imageData,
style: style,
roomType: roomType,
keywords: keywords,
boundary: boundary
)
@@ -110,7 +113,7 @@ final class ComfyClient {
// MARK: - Multipart Builder
private func buildMultipart(imageData: Data, style: String, keywords: String, boundary: String) -> Data {
private func buildMultipart(imageData: Data, roomType: String, keywords: String, boundary: String) -> Data {
var body = Data()
let crlf = "\r\n"
@@ -121,10 +124,10 @@ final class ComfyClient {
body += imageData
body += crlf
// style field — must be one of the 5 preset IDs
// roomType field
body += "--\(boundary)\(crlf)"
body += "Content-Disposition: form-data; name=\"style\"\(crlf)\(crlf)"
body += style
body += "Content-Disposition: form-data; name=\"room_type\"\(crlf)\(crlf)"
body += roomType
body += crlf
// keywords field — user's optional comma-separated additions

278
iOS/velocity/velocity/Features/Inventory/ARSunOverlayView.swift
View File

@@ -4,11 +4,14 @@ import CoreMotion
import SceneKit
import SwiftUI
// MARK: - ARSunOverlayView
struct ARSunOverlayView: UIViewRepresentable {
@Binding var sunNodesReady: Bool
let vm: SunseekerViewModel
func makeCoordinator() -> Coordinator {
Coordinator(sunNodesReady: $sunNodesReady)
Coordinator(sunNodesReady: $sunNodesReady, vm: vm)
}
func makeUIView(context: Context) -> ARSCNView {
@@ -18,7 +21,7 @@ struct ARSunOverlayView: UIViewRepresentable {
view.automaticallyUpdatesLighting = true
let config = ARWorldTrackingConfiguration()
config.worldAlignment = .gravityAndHeading
config.worldAlignment = .gravityAndHeading // north = -Z axis
view.session.run(config)
context.coordinator.attach(to: view)
@@ -32,87 +35,242 @@ struct ARSunOverlayView: UIViewRepresentable {
coordinator.stop()
}
// MARK: - Coordinator
final class Coordinator: NSObject, ARSCNViewDelegate, CLLocationManagerDelegate {
private let locationManager = CLLocationManager()
private let motionManager = CMMotionManager()
private weak var sceneView: ARSCNView?
private var heading: CLLocationDirection = 0
private var coordinate: CLLocationCoordinate2D?
private let vm: SunseekerViewModel
@Binding private var sunNodesReady: Bool
init(sunNodesReady: Binding<Bool>) {
// Scene node containers (replaced on each rebuild)
private var arcRootNode = SCNNode()
private var currentSunNode = SCNNode()
private var isSceneBuilt = false
// Fallback timer for CoreMotion-only mode
private var fallbackTimer: Timer?
private var limitedTrackingStart: Date?
init(sunNodesReady: Binding<Bool>, vm: SunseekerViewModel) {
_sunNodesReady = sunNodesReady
super.init()
locationManager.delegate = self
locationManager.desiredAccuracy = kCLLocationAccuracyBest
locationManager.headingFilter = 1
locationManager.requestWhenInUseAuthorization()
locationManager.startUpdatingLocation()
locationManager.startUpdatingHeading()
startMotion()
self.vm = vm
}
func attach(to sceneView: ARSCNView) {
self.sceneView = sceneView
addSunPathNodesIfPossible()
sceneView.scene.rootNode.addChildNode(arcRootNode)
sceneView.scene.rootNode.addChildNode(currentSunNode)
}
func stop() {
motionManager.stopDeviceMotionUpdates()
locationManager.stopUpdatingHeading()
locationManager.stopUpdatingLocation()
vm.stop()
fallbackTimer?.invalidate()
}
func locationManager(_ manager: CLLocationManager, didUpdateLocations locations: [CLLocation]) {
guard coordinate == nil, let location = locations.last else { return }
coordinate = location.coordinate
addSunPathNodesIfPossible()
}
// MARK: - ARSCNViewDelegate — per-frame update
func locationManager(_ manager: CLLocationManager, didUpdateHeading newHeading: CLHeading) {
heading = newHeading.trueHeading > 0 ? newHeading.trueHeading : newHeading.magneticHeading
addSunPathNodesIfPossible()
}
func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
guard vm.isReady else { return }
private func startMotion() {
guard motionManager.isDeviceMotionAvailable else { return }
motionManager.deviceMotionUpdateInterval = 0.1
motionManager.startDeviceMotionUpdates()
}
private func addSunPathNodesIfPossible() {
guard
let sceneView,
let coordinate,
!sunNodesReady
else { return }
let samples = SunMath.sunPathSamples(for: Date(), coordinate: coordinate)
let sorted = samples.sorted { $0.key < $1.key }
let root = SCNNode()
let northOffset = (heading).radians
let radius: Float = 1.8
for (_, pos) in sorted {
let elevation = Float(pos.elevation.radians)
let azimuth = Float((pos.azimuth).radians) - Float(northOffset)
let x = radius * cos(elevation) * sin(azimuth)
let y = radius * sin(elevation)
let z = -radius * cos(elevation) * cos(azimuth)
let sphere = SCNSphere(radius: 0.03)
sphere.firstMaterial?.diffuse.contents = UIColor.systemYellow
let node = SCNNode(geometry: sphere)
node.position = SCNVector3(x, y, z)
root.addChildNode(node)
// Build arc once
if !isSceneBuilt {
DispatchQueue.main.async { self.buildScene() }
}
sceneView.scene.rootNode.addChildNode(root)
// Update current sun orb every frame
if let cur = vm.currentPosition {
let pos = vm.worldPosition(for: cur, radius: 1.8)
currentSunNode.position = pos
}
}
func session(_ session: ARSession, cameraDidChangeTrackingState camera: ARCamera) {
switch camera.trackingState {
case .limited(let reason):
print("[Sunseeker] Tracking limited: \(reason)")
if limitedTrackingStart == nil {
limitedTrackingStart = Date()
// After 5s of limited tracking, switch to CoreMotion attitude fallback
fallbackTimer = Timer.scheduledTimer(withTimeInterval: 5, repeats: false) { [weak self] _ in
self?.activateCoreMotionFallback()
}
}
case .normal:
limitedTrackingStart = nil
fallbackTimer?.invalidate()
fallbackTimer = nil
case .notAvailable:
break
@unknown default:
break
}
}
// MARK: - Scene Building
private func buildScene() {
guard let sceneView else { return }
// Remove old nodes
arcRootNode.childNodes.forEach { $0.removeFromParentNode() }
currentSunNode.childNodes.forEach { $0.removeFromParentNode() }
let arc = vm.arc
let radius: Float = 1.8
var positions: [SCNVector3] = []
// Hourly marker spheres + time labels
for (date, pos) in arc {
guard pos.elevation > -5 else { continue }
let worldPos = vm.worldPosition(for: pos, radius: radius)
positions.append(worldPos)
let sphere = SCNSphere(radius: 0.018)
sphere.firstMaterial?.diffuse.contents = UIColor.systemYellow.withAlphaComponent(0.85)
sphere.firstMaterial?.lightingModel = .constant
let markerNode = SCNNode(geometry: sphere)
markerNode.position = worldPos
arcRootNode.addChildNode(markerNode)
// Time label (only on even hours to avoid clutter)
let calendar = Calendar.current
let hour = calendar.component(.hour, from: date)
if hour % 2 == 0 {
let labelNode = makeTextNode(text: hourLabel(from: date), color: .white, fontSize: 0.04)
labelNode.position = SCNVector3(worldPos.x, worldPos.y + 0.06, worldPos.z)
arcRootNode.addChildNode(labelNode)
}
}
// Continuous arc line
if positions.count >= 2 {
let lineNode = makeLineNode(through: positions, color: UIColor.systemYellow.withAlphaComponent(0.55))
arcRootNode.addChildNode(lineNode)
}
// Sunrise marker
if let riseDate = vm.riseSet.rise {
let risePos = SunMath.calculateSunPosition(date: riseDate, coordinate: vm.coordinate!)
let wPos = vm.worldPosition(for: risePos, radius: radius)
arcRootNode.addChildNode(makeSpecialMarker(at: wPos, color: .systemOrange, label: "Sunrise \(hourLabel(from: riseDate))"))
}
// Sunset marker
if let setDate = vm.riseSet.set {
let setPos = SunMath.calculateSunPosition(date: setDate, coordinate: vm.coordinate!)
let wPos = vm.worldPosition(for: setPos, radius: radius)
arcRootNode.addChildNode(makeSpecialMarker(at: wPos, color: .systemRed, label: "Sunset \(hourLabel(from: setDate))"))
}
// Current sun orb (large, animated glow)
if let cur = vm.currentPosition {
let orb = SCNSphere(radius: 0.055)
orb.firstMaterial?.diffuse.contents = UIColor.systemOrange
orb.firstMaterial?.emission.contents = UIColor.systemYellow
orb.firstMaterial?.lightingModel = .constant
let orbNode = SCNNode(geometry: orb)
orbNode.position = vm.worldPosition(for: cur, radius: radius)
// Pulse animation
let pulse = CABasicAnimation(keyPath: "scale")
pulse.fromValue = SCNVector3(1, 1, 1)
pulse.toValue = SCNVector3(1.3, 1.3, 1.3)
pulse.duration = 1.2
pulse.autoreverses = true
pulse.repeatCount = .infinity
orbNode.addAnimation(pulse, forKey: "pulse")
let label = makeTextNode(text: "Now", color: .systemYellow, fontSize: 0.05)
label.position = SCNVector3(0, 0.09, 0)
orbNode.addChildNode(label)
currentSunNode.addChildNode(orbNode)
}
isSceneBuilt = true
sunNodesReady = true
}
// MARK: - Node Factories
private func makeSpecialMarker(at pos: SCNVector3, color: UIColor, label: String) -> SCNNode {
let root = SCNNode()
let sphere = SCNSphere(radius: 0.035)
sphere.firstMaterial?.diffuse.contents = color
sphere.firstMaterial?.emission.contents = color.withAlphaComponent(0.5)
sphere.firstMaterial?.lightingModel = .constant
let markerNode = SCNNode(geometry: sphere)
markerNode.position = pos
root.addChildNode(markerNode)
let labelNode = makeTextNode(text: label, color: .white, fontSize: 0.04)
labelNode.position = SCNVector3(pos.x, pos.y + 0.07, pos.z)
root.addChildNode(labelNode)
return root
}
/// Creates a billboard SCNText node that always faces the camera.
private func makeTextNode(text: String, color: UIColor, fontSize: CGFloat) -> SCNNode {
let scnText = SCNText(string: text, extrusionDepth: 0)
scnText.font = UIFont.systemFont(ofSize: fontSize * 100, weight: .medium)
scnText.firstMaterial?.diffuse.contents = color
scnText.firstMaterial?.lightingModel = .constant
scnText.isWrapped = false
let textNode = SCNNode(geometry: scnText)
textNode.scale = SCNVector3(fontSize / 100, fontSize / 100, fontSize / 100)
// Billboard constraint — always face camera
let constraint = SCNBillboardConstraint()
constraint.freeAxes = .Y
textNode.constraints = [constraint]
// Centre text
let (min, max) = textNode.boundingBox
textNode.pivot = SCNMatrix4MakeTranslation((max.x - min.x) / 2, 0, 0)
return textNode
}
/// Builds a line strip SCNNode connecting all positions.
private func makeLineNode(through positions: [SCNVector3], color: UIColor) -> SCNNode {
guard positions.count >= 2 else { return SCNNode() }
var vertices: [SCNVector3] = positions
var indices: [Int32] = []
for i in 0..<(vertices.count - 1) {
indices.append(Int32(i))
indices.append(Int32(i + 1))
}
let vertexSource = SCNGeometrySource(vertices: vertices)
let element = SCNGeometryElement(
indices: indices,
primitiveType: .line
)
let geometry = SCNGeometry(sources: [vertexSource], elements: [element])
geometry.firstMaterial?.diffuse.contents = color
geometry.firstMaterial?.lightingModel = .constant
return SCNNode(geometry: geometry)
}
private func hourLabel(from date: Date) -> String {
let fmt = DateFormatter()
fmt.dateFormat = "ha"
fmt.amSymbol = "am"
fmt.pmSymbol = "pm"
return fmt.string(from: date)
}
// MARK: - CoreMotion Fallback
private func activateCoreMotionFallback() {
// In fallback mode we rely on CMMotionManager attitude (already running in SunseekerViewModel)
// and just keep the scene nodes updated via the 1s tick in the VM.
print("[Sunseeker] Switched to CoreMotion fallback — ARKit tracking unavailable.")
}
}
}
// MARK: - Degree helpers
private extension Double {
var radians: Double { self * .pi / 180.0 }
}

124
iOS/velocity/velocity/Features/Inventory/InventoryView.swift
View File

@@ -75,11 +75,8 @@ struct InventoryView: View {
switch store.mode {
case .sunseeker:
#if targetEnvironment(simulator)
SimulatorUnavailableCard(
icon: "camera.metering.unknown",
title: "AR Not Available in Simulator",
message: "Sunseeker requires a real device with a camera and compass. Run on iPhone or iPad to use this feature."
)
SimulatorSunOverlayView(sunNodesReady: $store.sunNodesReady)
.clipShape(RoundedRectangle(cornerRadius: 20))
#else
SunseekerPanel(sunNodesReady: $store.sunNodesReady)
#endif
@@ -178,28 +175,70 @@ private struct SimulatorUnavailableCard: View {
private struct SunseekerPanel: View {
@Binding var sunNodesReady: Bool
@State private var vm = SunseekerViewModel()
var body: some View {
ZStack(alignment: .topLeading) {
ARSunOverlayView(sunNodesReady: $sunNodesReady)
ARSunOverlayView(sunNodesReady: $sunNodesReady, vm: vm)
.clipShape(RoundedRectangle(cornerRadius: 20))
// Retained as a stylistic design element framing the AR view
DashedSunLine()
.stroke(Color.yellow.opacity(0.9), style: StrokeStyle(lineWidth: 3, dash: [10, 8]))
.padding(.horizontal, 24)
.padding(.vertical, 80)
VStack(alignment: .leading, spacing: 8) {
Text("Sunseeker")
.font(.headline)
Text("Point the iPad toward windows to inspect yearly sun-entry path.")
.font(.subheadline)
.foregroundStyle(.secondary)
}
.padding(14)
.background {
GlassBlurView(style: .systemThinMaterial)
.clipShape(RoundedRectangle(cornerRadius: 14))
VStack(alignment: .leading, spacing: 12) {
// Info block
VStack(alignment: .leading, spacing: 8) {
Text("Sunseeker")
.font(.headline)
Text("Point the iPad toward windows to inspect yearly sun-entry path.")
.font(.subheadline)
.foregroundStyle(.secondary)
}
.padding(14)
.background {
GlassBlurView(style: .systemThinMaterial)
.clipShape(RoundedRectangle(cornerRadius: 14))
}
if !vm.isReady && vm.locationError == nil {
// Loading state
HStack(spacing: 8) {
ProgressView().tint(.white)
Text("Looking for the Sun...")
.font(.footnote)
.foregroundStyle(.white)
}
.padding(.horizontal, 14)
.padding(.vertical, 10)
.background(Color.black.opacity(0.6).clipShape(Capsule()))
}
// Error banner (e.g. Location Denied)
if let error = vm.locationError {
HStack(spacing: 12) {
Image(systemName: "exclamationmark.triangle.fill")
.foregroundStyle(.yellow)
Text(error)
.font(.subheadline)
.foregroundStyle(.white)
Spacer()
Button("Settings") {
if let url = URL(string: UIApplication.openSettingsURLString) {
UIApplication.shared.open(url)
}
}
.buttonStyle(.borderedProminent)
.tint(.white.opacity(0.2))
}
.padding(14)
.background {
RoundedRectangle(cornerRadius: 14)
.fill(Color.red.opacity(0.8))
}
}
}
.padding(20)
}
@@ -208,19 +247,22 @@ private struct SunseekerPanel: View {
// MARK: - Dream Weaver
/// Available interior styles from integration guide §2.3
private struct InteriorStyle: Identifiable {
let id: String // sent as the `style` form field
/// Available room types from integration guide §2
private struct RoomType: Identifiable {
let id: String // sent as the `room_type` form field
let displayName: String
let icon: String // SF Symbol
}
private let dreamWeaverStyles: [InteriorStyle] = [
InteriorStyle(id: "scandinavian", displayName: "Scandi", icon: "snowflake"),
InteriorStyle(id: "art_deco", displayName: "Art Deco", icon: "sparkles"),
InteriorStyle(id: "biophilic", displayName: "Biophilic",icon: "leaf"),
InteriorStyle(id: "cyberpunk", displayName: "Cyberpunk",icon: "bolt"),
InteriorStyle(id: "japandi", displayName: "Japandi", icon: "mountain.2"),
private let roomTypes: [RoomType] = [
RoomType(id: "bedroom", displayName: "Bedroom", icon: "bed.double"),
RoomType(id: "living_room", displayName: "Living Rm", icon: "sofa"),
RoomType(id: "bathroom", displayName: "Bathroom", icon: "drop"),
RoomType(id: "kitchen", displayName: "Kitchen", icon: "refrigerator"),
RoomType(id: "dining_room", displayName: "Dining Rm", icon: "fork.knife"),
RoomType(id: "home_office", displayName: "Office", icon: "desktopcomputer"),
RoomType(id: "hallway", displayName: "Hallway", icon: "door.left.hand.open"),
RoomType(id: "balcony", displayName: "Balcony", icon: "sun.max"),
]
private struct DreamWeaverPanel: View {
@@ -230,8 +272,8 @@ private struct DreamWeaverPanel: View {
@Binding var errorMessage: String?
@Binding var showCamera: Bool
/// Selected style ID — sent as `style` field (§3.2). nil = none chosen yet.
@State private var selectedStyle: String? = nil
/// Selected room type ID — sent as `room_type` field. nil = none chosen yet.
@State private var selectedRoomType: String? = nil
/// Optional extra keywords — sent as `keywords` field (§3.2)
@State private var keywords: String = ""
/// Server health: nil = checking, true = online, false = offline
@@ -319,34 +361,34 @@ private struct DreamWeaverPanel: View {
.transition(.move(edge: .top).combined(with: .opacity))
}
// ── Style picker (§2.3) ───────────────────────────────────────
// ── Room Type picker ───────────────────────────────────────
ScrollView(.horizontal, showsIndicators: false) {
HStack(spacing: 8) {
ForEach(dreamWeaverStyles) { style in
ForEach(roomTypes) { room in
Button {
withAnimation(.spring(response: 0.3)) {
// Tap again to deselect
selectedStyle = selectedStyle == style.id ? nil : style.id
selectedRoomType = selectedRoomType == room.id ? nil : room.id
}
} label: {
HStack(spacing: 6) {
Image(systemName: style.icon)
Image(systemName: room.icon)
.font(.system(size: 11, weight: .medium))
Text(style.displayName)
Text(room.displayName)
.font(.system(size: 13, weight: .medium))
}
.padding(.horizontal, 12)
.padding(.vertical, 7)
.background(
Capsule()
.fill(selectedStyle == style.id
.fill(selectedRoomType == room.id
? Color(red: 0.231, green: 0.510, blue: 0.965)
: Color.white.opacity(0.08))
)
.foregroundStyle(selectedStyle == style.id ? .white : .white.opacity(0.6))
.foregroundStyle(selectedRoomType == room.id ? .white : .white.opacity(0.6))
.overlay(
Capsule()
.stroke(selectedStyle == style.id
.stroke(selectedRoomType == room.id
? Color.clear
: Color.white.opacity(0.12), lineWidth: 1)
)
@@ -360,7 +402,7 @@ private struct DreamWeaverPanel: View {
// ── Keywords input ───────────────────────────────────────────
PromptInputBar(
text: $keywords,
isDisabled: sourceImage == nil || isProcessing
isDisabled: sourceImage == nil || isProcessing || serverOnline == false
) {
Task { await generate() }
}
@@ -391,12 +433,16 @@ private struct DreamWeaverPanel: View {
@MainActor
private func generate() async {
guard let sourceImage, !isProcessing else { return }
if serverOnline == false {
errorMessage = "Server is currently offline. Please try again later."
return
}
isProcessing = true
errorMessage = nil
do {
let result = try await ComfyClient.shared.generateImage(
source: sourceImage,
style: selectedStyle ?? dreamWeaverStyles[0].id, // default: scandinavian
roomType: selectedRoomType ?? roomTypes[0].id, // default: bedroom
keywords: keywords.trimmingCharacters(in: .whitespaces)
)
withAnimation(.easeInOut(duration: 0.4)) {
@@ -419,7 +465,7 @@ private struct PromptInputBar: View {
@FocusState private var isFocused: Bool
@State private var shimmer = false
private let placeholder = "gold, marble, luxury... (optional keywords)"
private let placeholder = "gold, marble, luxury, etc."
var body: some View {
HStack(spacing: 10) {

256
iOS/velocity/velocity/Features/Inventory/SimulatorSunOverlayView.swift Normal file
View File

@@ -0,0 +1,256 @@
import CoreLocation
import SceneKit
import SwiftUI
#if targetEnvironment(simulator)
/// A non-AR 3D view for testing the Sunseeker path logic on the iOS Simulator.
/// Uses a synthetic camera, fake location, and mock heading instead of ARKit.
struct SimulatorSunOverlayView: UIViewRepresentable {
@Binding var sunNodesReady: Bool
// Fake location (e.g. San Francisco)
private let mockLocation = CLLocationCoordinate2D(latitude: 37.7749, longitude: -122.4194)
private let mockHeading: Double = 0 // North
func makeCoordinator() -> Coordinator {
Coordinator(sunNodesReady: $sunNodesReady, mockLocation: mockLocation, mockHeading: mockHeading)
}
func makeUIView(context: Context) -> SCNView {
let view = SCNView(frame: .zero)
view.scene = SCNScene()
view.allowsCameraControl = true // Swipe around the 3D space
view.autoenablesDefaultLighting = true
view.backgroundColor = UIColor(white: 0.1, alpha: 1.0)
view.isPlaying = true // Force render loop
view.showsStatistics = true // Prove it's rendering
// Setup synthetic camera
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.camera?.zFar = 100
cameraNode.position = SCNVector3(x: 0, y: 0, z: 0) // Centered
view.scene?.rootNode.addChildNode(cameraNode)
context.coordinator.attach(to: view)
return view
}
func updateUIView(_ uiView: SCNView, context: Context) {}
final class Coordinator: NSObject {
private weak var sceneView: SCNView?
@Binding private var sunNodesReady: Bool
private let mockLocation: CLLocationCoordinate2D
private let mockHeading: Double
private var arcRootNode = SCNNode()
private var currentSunNode = SCNNode()
private var updateTimer: Timer?
init(sunNodesReady: Binding<Bool>, mockLocation: CLLocationCoordinate2D, mockHeading: Double) {
_sunNodesReady = sunNodesReady
self.mockLocation = mockLocation
self.mockHeading = mockHeading
super.init()
}
func attach(to view: SCNView) {
self.sceneView = view
view.scene?.rootNode.addChildNode(arcRootNode)
view.scene?.rootNode.addChildNode(currentSunNode)
buildScene()
startRealTimeTick()
}
deinit {
updateTimer?.invalidate()
}
private func startRealTimeTick() {
// Update current sun position every second
updateTimer = Timer.scheduledTimer(withTimeInterval: 1.0, repeats: true) { [weak self] _ in
guard let self = self else { return }
let cur = SunMath.calculateSunPosition(date: Date(), coordinate: self.mockLocation)
// Need to remove previous child as we are completely replacing it
self.currentSunNode.childNodes.forEach { $0.removeFromParentNode() }
let radius: Float = 1.8
let orb = SCNSphere(radius: 0.055)
orb.firstMaterial?.diffuse.contents = UIColor.systemOrange
orb.firstMaterial?.emission.contents = UIColor.systemYellow
orb.firstMaterial?.lightingModel = .constant
let orbNode = SCNNode(geometry: orb)
orbNode.position = self.worldPosition(for: cur, radius: radius)
let pulse = CABasicAnimation(keyPath: "scale")
pulse.fromValue = SCNVector3(1, 1, 1)
pulse.toValue = SCNVector3(1.3, 1.3, 1.3)
pulse.duration = 1.2
pulse.autoreverses = true
pulse.repeatCount = .infinity
orbNode.addAnimation(pulse, forKey: "pulse")
let label = self.makeTextNode(text: "Now", color: .systemYellow, fontSize: 0.05)
label.position = SCNVector3(0, 0.09, 0)
orbNode.addChildNode(label)
self.currentSunNode.addChildNode(orbNode)
}
}
private func buildScene() {
let arc = SunMath.sunPathArc(for: Date(), coordinate: mockLocation)
let riseSet = SunMath.sunRiseSet(for: Date(), coordinate: mockLocation)
let radius: Float = 1.8
var positions: [SCNVector3] = []
// Hourly blocks
for (date, pos) in arc {
guard pos.elevation > -5 else { continue }
let worldPos = worldPosition(for: pos, radius: radius)
positions.append(worldPos)
let sphere = SCNSphere(radius: 0.018)
sphere.firstMaterial?.diffuse.contents = UIColor.systemYellow.withAlphaComponent(0.85)
sphere.firstMaterial?.lightingModel = .constant
let markerNode = SCNNode(geometry: sphere)
markerNode.position = worldPos
arcRootNode.addChildNode(markerNode)
let calendar = Calendar.current
let hour = calendar.component(.hour, from: date)
if hour % 2 == 0 {
let labelNode = makeTextNode(text: hourLabel(from: date), color: .white, fontSize: 0.04)
labelNode.position = SCNVector3(worldPos.x, worldPos.y + 0.06, worldPos.z)
arcRootNode.addChildNode(labelNode)
}
}
if positions.count >= 2 {
let lineNode = makeLineNode(through: positions, color: UIColor.systemYellow.withAlphaComponent(0.55))
arcRootNode.addChildNode(lineNode)
}
if let riseDate = riseSet.rise {
let risePos = SunMath.calculateSunPosition(date: riseDate, coordinate: mockLocation)
let wPos = worldPosition(for: risePos, radius: radius)
arcRootNode.addChildNode(makeSpecialMarker(at: wPos, color: .systemOrange, label: "Sunrise \(hourLabel(from: riseDate))"))
}
if let setDate = riseSet.set {
let setPos = SunMath.calculateSunPosition(date: setDate, coordinate: mockLocation)
let wPos = worldPosition(for: setPos, radius: radius)
arcRootNode.addChildNode(makeSpecialMarker(at: wPos, color: .systemRed, label: "Sunset \(hourLabel(from: setDate))"))
}
// Generate current sun node synchronously for first frame
updateTimer?.fire()
DispatchQueue.main.async {
self.sunNodesReady = true
}
}
// MARK: Math equivalent from SunseekerViewModel
private func worldPosition(for sun: SunPosition, radius: Float) -> SCNVector3 {
let elev = Float(sun.elevation * .pi / 180.0)
let az = Float(sun.azimuth * .pi / 180.0)
let x = radius * cos(elev) * sin(az)
let y = radius * sin(elev)
let z = -radius * cos(elev) * cos(az)
return SCNVector3(x, y, z)
}
// MARK: SceneKit Factories
private func makeSpecialMarker(at pos: SCNVector3, color: UIColor, label: String) -> SCNNode {
let root = SCNNode()
let sphere = SCNSphere(radius: 0.035)
sphere.firstMaterial?.diffuse.contents = color
sphere.firstMaterial?.emission.contents = color.withAlphaComponent(0.5)
sphere.firstMaterial?.lightingModel = .constant
let markerNode = SCNNode(geometry: sphere)
markerNode.position = pos
root.addChildNode(markerNode)
let labelNode = makeTextNode(text: label, color: .white, fontSize: 0.04)
labelNode.position = SCNVector3(pos.x, pos.y + 0.07, pos.z)
root.addChildNode(labelNode)
return root
}
private func makeTextNode(text: String, color: UIColor, fontSize: CGFloat) -> SCNNode {
// SCNText is buggy in Simulator. Render text to a UIImage instead.
let font = UIFont.systemFont(ofSize: 40, weight: .bold)
let attributes: [NSAttributedString.Key: Any] = [
.font: font,
.foregroundColor: color
]
let size = (text as NSString).size(withAttributes: attributes)
// Add some padding
let paddedSize = CGSize(width: size.width + 10, height: size.height + 10)
let renderer = UIGraphicsImageRenderer(size: paddedSize)
let image = renderer.image { context in
(text as NSString).draw(
in: CGRect(x: 5, y: 5, width: size.width, height: size.height),
withAttributes: attributes
)
}
// Map the image onto an SCNPlane
// A 100x50 image becomes a 0.1 x 0.05 meter plane
let plane = SCNPlane(width: paddedSize.width / 1000.0, height: paddedSize.height / 1000.0)
plane.firstMaterial?.diffuse.contents = image
plane.firstMaterial?.isDoubleSided = true
plane.firstMaterial?.lightingModel = .constant
let textNode = SCNNode(geometry: plane)
// Statically scale the plane up so it is readable next to the spheres
textNode.scale = SCNVector3(1.5, 1.5, 1.5)
let constraint = SCNBillboardConstraint()
constraint.freeAxes = .all
textNode.constraints = [constraint]
return textNode
}
private func makeLineNode(through positions: [SCNVector3], color: UIColor) -> SCNNode {
guard positions.count >= 2 else { return SCNNode() }
var vertices: [SCNVector3] = positions
var indices: [Int32] = []
for i in 0..<(vertices.count - 1) {
indices.append(Int32(i))
indices.append(Int32(i + 1))
}
let vertexSource = SCNGeometrySource(vertices: vertices)
let element = SCNGeometryElement(
indices: indices,
primitiveType: .line
)
let geometry = SCNGeometry(sources: [vertexSource], elements: [element])
geometry.firstMaterial?.diffuse.contents = color
geometry.firstMaterial?.lightingModel = .constant
return SCNNode(geometry: geometry)
}
private func hourLabel(from date: Date) -> String {
let fmt = DateFormatter()
fmt.dateFormat = "ha"
fmt.amSymbol = "am"
fmt.pmSymbol = "pm"
return fmt.string(from: date)
}
}
}
#endif

140
iOS/velocity/velocity/Features/Inventory/SunseekerViewModel.swift Normal file
View File

@@ -0,0 +1,140 @@
import CoreLocation
import CoreMotion
import Foundation
import Observation
import SceneKit
// MARK: - SunseekerViewModel
/// Owns all sensor state for the Sunseeker AR overlay.
/// Separates CoreLocation / CoreMotion concerns from the ARKit view layer.
@Observable
final class SunseekerViewModel: NSObject, CLLocationManagerDelegate {
// MARK: - Published State
/// True once we have both a GPS fix and a valid heading.
private(set) var isReady = false
/// Latest GPS coordinate. nil until first fix.
private(set) var coordinate: CLLocationCoordinate2D?
/// Latest true heading (0 = North, clockwise).
private(set) var heading: Double = 0
/// Dense hourly arc for today.
private(set) var arc: [(date: Date, position: SunPosition)] = []
/// Current real-time sun position (updated every second).
private(set) var currentPosition: SunPosition?
/// Sunrise and sunset for today.
private(set) var riseSet: (rise: Date?, set: Date?) = (nil, nil)
/// Diagnostic string for the UI when location access is denied.
private(set) var locationError: String?
// MARK: - Private
private let locationManager = CLLocationManager()
private let motionManager = CMMotionManager()
private var updateTimer: Timer?
// MARK: - Init / Deinit
override init() {
super.init()
locationManager.delegate = self
locationManager.desiredAccuracy = kCLLocationAccuracyBest
locationManager.headingFilter = 1.0
locationManager.requestWhenInUseAuthorization()
locationManager.startUpdatingLocation()
locationManager.startUpdatingHeading()
startMotionUpdates()
startRealTimeTick()
}
deinit {
stop()
}
// MARK: - Control
func stop() {
motionManager.stopDeviceMotionUpdates()
locationManager.stopUpdatingLocation()
locationManager.stopUpdatingHeading()
updateTimer?.invalidate()
updateTimer = nil
}
// MARK: - World-Space Transform
/// Converts a solar `SunPosition` into a SceneKit world-space position on a sphere of given `radius`.
/// Orientation is relative to ARWorldTrackingConfiguration(.gravityAndHeading), so north = -Z axis.
func worldPosition(for sun: SunPosition, radius: Float) -> SCNVector3 {
let elev = Float(sun.elevation.radians)
let az = Float(sun.azimuth.radians) // clockwise from north
let x = radius * cos(elev) * sin(az)
let y = radius * sin(elev)
let z = -radius * cos(elev) * cos(az) // -Z = north in ARKit gravity+heading
return SCNVector3(x, y, z)
}
// MARK: - Private helpers
private func startMotionUpdates() {
guard motionManager.isDeviceMotionAvailable else { return }
motionManager.deviceMotionUpdateInterval = 0.05
motionManager.startDeviceMotionUpdates()
}
private func startRealTimeTick() {
updateTimer = Timer.scheduledTimer(withTimeInterval: 1.0, repeats: true) { [weak self] _ in
guard let self, let coord = self.coordinate else { return }
self.currentPosition = SunMath.calculateSunPosition(date: Date(), coordinate: coord)
}
}
private func refreshArc() {
guard let coord = coordinate else { return }
arc = SunMath.sunPathArc(for: Date(), coordinate: coord)
riseSet = SunMath.sunRiseSet(for: Date(), coordinate: coord)
currentPosition = SunMath.calculateSunPosition(date: Date(), coordinate: coord)
isReady = true
}
// MARK: - CLLocationManagerDelegate
func locationManager(_ manager: CLLocationManager, didUpdateLocations locations: [CLLocation]) {
guard coordinate == nil, let loc = locations.last else { return }
coordinate = loc.coordinate
refreshArc()
}
func locationManager(_ manager: CLLocationManager, didUpdateHeading newHeading: CLHeading) {
heading = newHeading.trueHeading > 0 ? newHeading.trueHeading : newHeading.magneticHeading
if coordinate != nil { isReady = true }
}
func locationManager(_ manager: CLLocationManager, didFailWithError error: Error) {
print("[Sunseeker] Location error: \(error.localizedDescription)")
}
func locationManagerDidChangeAuthorization(_ manager: CLLocationManager) {
switch manager.authorizationStatus {
case .denied, .restricted:
locationError = "Location access needed to calculate the sun path. Please enable it in Settings."
case .notDetermined:
manager.requestWhenInUseAuthorization()
default:
locationError = nil
}
}
}
// MARK: - Degree helpers (internal to this file)
private extension Double {
var radians: Double { self * .pi / 180.0 }
}

3
iOS/velocity/velocity/Info.plist
View File

@@ -13,5 +13,8 @@
<string>Velocity would like to save your AI-generated room design to your photo library.</string>
<key>NSCameraUsageDescription</key>
<string>Velocity needs camera access to capture room photos for Dream Weaver.</string>
<key>NSLocationWhenInUseUsageDescription</key>
<string>Velocity uses your location to calculate the accurate sun path for your property.</string>
</dict>
</plist>