sagnik/Project_Velocity
2
1
Fork 2
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Overlay the mathematical Sun Path over the live camera feed or 3D model view (#8)

Browse Source 
#7 Task completed.

Co-authored-by: Sayan Datta <sayan@Sayans-MacBook-Air.local>
Reviewed-on: #8
This commit was merged in pull request #8.
This commit is contained in:
sayan
2026-03-21 17:01:06 +05:30
parent 6c98affe53
commit 023ba48da2
16 changed files with 3344 additions and 2939 deletions
Download Patch File Download Diff File Expand all files Collapse all files

709
.Agent Context/Bibels/dreamweaver_ios_integration_guide.md
View File

@@ -1,512 +1,197 @@
# Dream Weaver — iOS ↔ Backend Integration Guide # Dream Weaver API v2 — iOS Integration Guide (Dynamic Keywords)
**Version:** 2.0 | **Updated:** 2026-03-09 | **Server:** `54.172.172.2` | **Port:** `8080` **Version:** 2.0-FINAL | **Updated:** 2026-03-09 | **Server:** `54.172.172.2` | **Port:** `8082`
> This document is for **Sayan** (iOS / Swift) and **Sourik** (backend review). > This document is for **Sayan** (iOS / Swift).
> It describes exactly how the iPad app should talk to the Dream Weaver AI and how keywords from a user tap become a full ComfyUI generation. > Dream Weaver API v2 introduces a **Dynamic Keyword to Local LLM Prompt Expansion** system.
> The app no longer relies on 5 hardcoded styles. Users can pick ANY keywords, and a local LLM (Qwen 3.5 27B via Ollama) will generate a photorealistic interior design prompt based on the room type without sending data to the cloud.
---
> [!CAUTION]
## 1. Architecture Overview > **PORT 8080 IS DEAD.** Do not use port 8080 anymore. The old gateway process has been completely killed. If you try to send `POST /dream-weaver` or `/docs` to port 8080 you will get a 404. You MUST change your `AppConfig.baseURL` parameter to use port **`8082`**.
``` ---
┌────────────────────┐ HTTP/S ┌──────────────────────────────┐
│ │ ─── POST image ───► │ Dream Weaver Gateway │ ## 1. Architecture Overview (API v2)
│ iPad App (Swift) │ │ FastAPI port 8080 │
│ │ ◄── PNG result ─── │ dw_gateway.py │ ```
└────────────────────┘ └─────────────┬────────────────┘ ┌────────────────────┐ HTTP/S ┌──────────────────────────────┐
│ internal HTTP │ │ ── keywords ────► │ Dream Weaver Gateway v2 │
▼ │ iPad App (Swift) │ │ FastAPI port 8082 │
┌─────────────────────────┐ │ │ ◄── PNG result ── │ dw_gateway_v2.py │
│ ComfyUI Engine │ └────────────────────┘ └─────────────┬────────────────┘
│ port 8188 │ │ LLM Prompt Expansion
│ RealVisXL V5.0 Ltng │ │ (Local Ollama: Qwen 3.5 27B)
│ 4× NVIDIA L4 (96 GB) │ ▼
└─────────────────────────┘ ┌─────────────────────────┐
``` │ ComfyUI Engine │
│ port 8188 │
**Key rule:** The iPad app **never** talks to ComfyUI directly. It only talks to the Gateway on `:8080`. │ RealVisXL V5.0 Ltng │
└─────────────────────────┘
--- ```
## 2. How Keywords Become Prompts **Key changes in v2:**
1. The API now runs on port **`8082`** to avoid conflicts.
### 2.1 The Prompt Expansion System 2. The [style](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/dreamweaver_batch_processor.py#394-414) parameter is deprecated in favor of `keywords` (array of strings) and [room_type](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/dw_gateway_v2.py#171-186).
Each interior style in the app is backed by a **prompt template** in `comfy_engine/prompts/`. When the user taps a style card (or types keywords), those keywords get **merged into the template** to build the final ComfyUI prompt injected into node `9` (positive CLIPTextEncode). ---
**Prompt template structure** (from [scandinavian_minimalist.txt](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/prompts/scandinavian_minimalist.txt)): ## 2. Dynamic Keyword Expansion Flow
```
POSITIVE PROMPT: Instead of injecting keywords into a rigid template, the new backend reads the `keywords` and [room_type](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/dw_gateway_v2.py#171-186), and asks a local LLM (Qwen 3.5 27B) to act as an interior designer:
scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles,
abundant natural light streaming through large windows, clean white walls, ... 1. **User input:** `keywords: ["blue marble", "gold veins", "renaissance"]`, `room_type: "bathroom"`
Style Weight: <lora:Interior_Style_Scandi:0.8> 2. **Backend LLM Expansion:** The LLM knows that a "bathroom" cannot have beds and needs wet-area materials. It creates a rich positive prompt: *"renaissance revival luxury interior design, blue veined marble flooring, gold brass fixtures..."*
3. **ComfyUI Generation:** The expanded prompt is sent to ComfyUI for generation.
NEGATIVE PROMPT:
(worst quality, low quality, illustration, 3d render...), heavy ornamentation,... **Supported Room Types:**
`bedroom`, `living_room`, `bathroom`, `kitchen`, `dining_room`, `home_office`, `hallway`, `balcony`.
TECHNICAL PARAMETERS:
- Denoising Strength: 0.70 ---
- CFG Scale: 7.0
- Recommended Sampler: dpmpp_2m_karras ## 3. API Reference — New v2 Endpoints
- Steps: 30-40
``` ### BASE URL
```
### 2.2 Keyword Expansion Flow http://54.172.172.2:8082
```
```
User taps: ["marble", "gold", "luxury"] ### 3.1 `GET /health` — Liveness Check
+ Call this on app launch to confirm the v2 server is up.
Style selected: "art_deco" **Response:**
│ ```json
▼ {
Backend expands: "status": "ok",
base_prompt = art_deco_luxe.txt (POSITIVE PROMPT section) "comfyui": true,
user_keywords_str = "marble, gold, luxury" "gpu": "4x NVIDIA L4 (96GB VRAM)",
final_prompt = base_prompt + ", " + user_keywords_str "model": "RealVisXL V5.0 Lightning",
│ "llm_expansion": true,
▼ "version": "2.0.0"
Injected into ComfyUI workflow: }
node "9" → CLIPTextEncode → text: [final_prompt] ```
node "10" → CLIPTextEncode → text: [negative_prompt from template]
node "1" → LoadImage → image: [uploaded filename] ### 3.2 `GET /room-types`
node "13" → KSampler → denoise: 0.72, cfg: 7.5, steps: 35 Returns all supported room types and their required design context (useful if you want to build UI tooltips).
``` ```json
{
### 2.3 Available Styles and Their Keywords (for the Style Picker UI) "room_types": {
"bedroom": {
| Style ID | Display Name | Suggested Keywords Palette | "description": "a private sleeping space",
|---|---|---| "key_elements": ["bed", "bedside tables", "wardrobe", "soft lighting", "textiles", "headboard"]
| `scandinavian` | Scandinavian Minimalist | oak, linen, white, hygge, cozy, birch, natural | },
| `art_deco` | Art Deco Luxe | gold, marble, velvet, geometric, 1920s, brass, crystal | ...
| `biophilic` | Biophilic Organic | green wall, stone, rattan, terracotta, botanical, moss | }
| `cyberpunk` | Cyberpunk Neon | neon, chrome, holographic, dark, LED, futuristic, blade runner | }
| `japandi` | Japandi Fusion | wabi-sabi, ash wood, ceramic, zen, minimal, shoji, serene | ```
--- ### 3.3 `POST /dream-weaver/expand` (Preview Prompt)
Use this if you want the user to **preview** the LLM's generated prompt before committing to a generation.
## 3. API Reference — What Sayan Needs to Call **Request (JSON):**
```json
### BASE URL {
``` "keywords": ["blue marble", "gold veins", "renaissance"],
http://54.172.172.2:8080 "room_type": "bathroom"
``` }
```
> [!NOTE] **Response:**
> Once we attach an Elastic IP or domain, swap this in `AppConfig.swift`. ```json
{
--- "style_name": "Renaissance Luxury",
"positive_prompt": "renaissance revival luxury interior design, blue veined marble flooring...",
### 3.1 `GET /health` — Liveness Check "negative_prompt": "(worst quality, low quality...), extra windows...",
"cfg": 7.5,
Call this on app launch to confirm the server is up before showing the Generate button. "denoise": 0.72,
"steps": 30,
**Request:** "source": "ollama_local"
```http }
GET http://54.172.172.2:8080/health ```
```
### 3.4 `POST /dream-weaver` (Submit Generation)
**Response:** Use this for the main generation flow.
```json **Request:** `multipart/form-data`
{ | Field | Type | Required | Description |
"status": "ok", |---|---|---|---|
"comfyui": true, | [image](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/a100_deployment_executor.py#306-322) | File | ✅ | The room photo (JPEG/PNG) |
"gpu": "4x NVIDIA L4 (96GB VRAM)", | `keywords` | String | ✅ | Comma-separated user keywords e.g. `"gold, marble, luxury"` |
"model": "RealVisXL V5.0 Lightning" | [room_type](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/dw_gateway_v2.py#171-186) | String | ✅ | e.g. `"living_room"`, `"bedroom"` |
} | `additional_notes` | String | ➕ | (Optional) e.g. `"make it feel like a luxury hotel"` |
``` | `denoise` | Float | ➕ | (Optional) 0.5–0.85. If omitted, LLM decides. |
**Swift:** **Response:**
```swift ```json
func checkServerHealth() async throws -> Bool { {
let url = URL(string: "\(AppConfig.baseURL)/health")! "job_id": "a1b2c3d4-...",
let (data, _) = try await URLSession.shared.data(from: url) "status": "processing",
let json = try JSONDecoder().decode(HealthResponse.self, from: data) "prompt_preview": "renaissance revival luxury interior design...",
return json.status == "ok" "poll_url": "/dream-weaver/status/a1b2c3d4-...",
} "result_url": "/dream-weaver/result/a1b2c3d4-..."
``` }
```
---
---
### 3.2 `POST /dream-weaver` — Submit Generation Job (Async)
## 4. Polling & Downloading (Unchanged from v1)
Use this for the main generation flow. Returns a `job_id` immediately; poll for result.
**Poll Job Status:**
**Request:** `multipart/form-data` `GET /dream-weaver/status/{job_id}` every 2 seconds until `ready == true`.
| Field | Type | Required | Description |
|---|---|---|---| **Download Result:**
| [image](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/a100_deployment_executor.py#306-322) | File (JPEG/PNG) | ✅ | The room photo from camera or library | `GET /dream-weaver/result/{job_id}` returns the raw PNG stream.
| [style](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/dreamweaver_batch_processor.py#394-414) | String | ✅ | One of: `scandinavian`, `art_deco`, `biophilic`, `cyberpunk`, `japandi` |
| `keywords` | String | ➕ | Comma-separated user keywords e.g. `"gold, marble, luxury"` | ---
| `denoise` | Float | ➕ | 0.5–0.85 (default `0.72`). Higher = more creative |
## 5. Updated Swift Example (v2)
**Response:**
```json ```swift
{ func submitGenerationV2(image: UIImage, roomType: String, keywords: [String]) async throws -> GenerationJob {
"job_id": "a1b2c3d4-...", let url = URL(string: "\(AppConfig.baseURL)/dream-weaver")!
"status": "processing", var request = URLRequest(url: url)
"poll_url": "/dream-weaver/status/a1b2c3d4-...", request.httpMethod = "POST"
"result_url": "/dream-weaver/result/a1b2c3d4-..."
} let boundary = UUID().uuidString
``` request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")
**Swift example:** var body = Data()
```swift
func submitGeneration(image: UIImage, style: String, keywords: [String]) async throws -> GenerationJob { // 1. Image
let url = URL(string: "\(AppConfig.baseURL)/dream-weaver")! let imageData = image.jpegData(compressionQuality: 0.85)!
var request = URLRequest(url: url) body.appendMultipartForm(boundary: boundary, name: "image", filename: "room.jpg", contentType: "image/jpeg", data: imageData)
request.httpMethod = "POST"
// 2. Room Type
let boundary = UUID().uuidString body.appendMultipartForm(boundary: boundary, name: "room_type", value: roomType)
request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")
// 3. Keywords
var body = Data() let kwString = keywords.joined(separator: ", ")
// Image field body.appendMultipartForm(boundary: boundary, name: "keywords", value: kwString)
let imageData = image.jpegData(compressionQuality: 0.85)!
body.append("--\(boundary)\r\n".data(using: .utf8)!) body.append("--\(boundary)--\r\n".data(using: .utf8)!)
body.append("Content-Disposition: form-data; name=\"image\"; filename=\"room.jpg\"\r\n".data(using: .utf8)!) request.httpBody = body
body.append("Content-Type: image/jpeg\r\n\r\n".data(using: .utf8)!)
body.append(imageData) let (data, _) = try await URLSession.shared.data(for: request)
body.append("\r\n".data(using: .utf8)!) return try JSONDecoder().decode(GenerationJob.self, from: data)
}
// Style field
body.append("--\(boundary)\r\n".data(using: .utf8)!) // Helper extension for building multipart forms cleanly
body.append("Content-Disposition: form-data; name=\"style\"\r\n\r\n".data(using: .utf8)!) extension Data {
body.append("\(style)\r\n".data(using: .utf8)!) mutating func appendMultipartForm(boundary: String, name: String, value: String) {
self.append("--\(boundary)\r\n".data(using: .utf8)!)
// Keywords field (user tapped keywords) self.append("Content-Disposition: form-data; name=\"\(name)\"\r\n\r\n".data(using: .utf8)!)
if !keywords.isEmpty { self.append("\(value)\r\n".data(using: .utf8)!)
let kwString = keywords.joined(separator: ", ") }
body.append("--\(boundary)\r\n".data(using: .utf8)!)
body.append("Content-Disposition: form-data; name=\"keywords\"\r\n\r\n".data(using: .utf8)!) mutating func appendMultipartForm(boundary: String, name: String, filename: String, contentType: String, data: Data) {
body.append("\(kwString)\r\n".data(using: .utf8)!) self.append("--\(boundary)\r\n".data(using: .utf8)!)
} self.append("Content-Disposition: form-data; name=\"\(name)\"; filename=\"\(filename)\"\r\n".data(using: .utf8)!)
self.append("Content-Type: \(contentType)\r\n\r\n".data(using: .utf8)!)
body.append("--\(boundary)--\r\n".data(using: .utf8)!) self.append(data)
request.httpBody = body self.append("\r\n".data(using: .utf8)!)
}
let (data, _) = try await URLSession.shared.data(for: request) }
return try JSONDecoder().decode(GenerationJob.self, from: data) ```
}
``` ---
--- ## 6. Sayan's Action Checklist (v2)
### 3.3 `GET /dream-weaver/status/{job_id}` — Poll Job Status - [ ] Change `AppConfig.baseURL` port to `8082` (e.g., `http://54.172.172.2:8082`).
- [ ] Add a UI element for the user to select the **Room Type** (`bedroom`, `living_room`, `bathroom`, etc.).
Poll every **2 seconds** until `ready == true`. - [ ] Change the `POST /dream-weaver` payload from `{style}` to `{keywords, room_type}`.
- [ ] (Optional) Use the new `GET /dream-weaver/expand` endpoint to let the user preview and edit the AI-generated prompt before generating.
**Response while processing:**
```json
{ "status": "processing", "ready": false, "style": "art_deco" }
```
**Response when done:**
```json
{
"status": "done",
"ready": true,
"result_url": "/dream-weaver/result/a1b2c3d4-...",
"style": "art_deco"
}
```
**Swift polling loop:**
```swift
func pollForResult(jobId: String) async throws -> URL {
let statusURL = URL(string: "\(AppConfig.baseURL)/dream-weaver/status/\(jobId)")!
for _ in 0..<150 { // max 5 min (150 × 2s)
try await Task.sleep(nanoseconds: 2_000_000_000) // 2 seconds
let (data, _) = try await URLSession.shared.data(from: statusURL)
let status = try JSONDecoder().decode(JobStatus.self, from: data)
if status.ready {
return URL(string: "\(AppConfig.baseURL)/dream-weaver/result/\(jobId)")!
}
if status.status == "error" {
throw DreamWeaverError.generationFailed(status.error ?? "Unknown")
}
}
throw DreamWeaverError.timeout
}
```
---
### 3.4 `GET /dream-weaver/result/{job_id}` — Download Result Image
Returns a PNG image stream directly. Download and display to user.
```swift
func downloadResult(resultURL: URL) async throws -> UIImage {
let (data, _) = try await URLSession.shared.data(from: resultURL)
guard let image = UIImage(data: data) else {
throw DreamWeaverError.invalidImageData
}
return image
}
```
---
### 3.5 `POST /dream-weaver/sync` — One-Shot Blocking Call
For **testing only** or fast network connections. Waits up to 120 seconds and returns the image directly.
```swift
// Same multipart form as /dream-weaver but returns PNG bytes directly
// Not recommended for production — use async flow above
```
---
## 4. Keyword → Prompt Expansion (Backend Change Required)
> [!IMPORTANT]
> The current [dw_gateway.py](file:///C:/Windows/Temp/dw_gateway.py) does NOT yet accept `keywords` from the app.
> Sagnik needs to add keyword expansion to the gateway. Here is the exact code change:
**In [dw_gateway.py](file:///C:/Windows/Temp/dw_gateway.py), update [build_workflow()](file:///C:/Windows/Temp/dw_gateway.py#33-65):**
```python
# Prompt library — maps style ID to (positive_base, negative, cfg, denoise, steps)
STYLE_LIBRARY = {
"scandinavian": {
"pos": "scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles, abundant natural light, clean white walls, simple functional furniture, cozy hygge atmosphere, architectural photography, 8k resolution, photorealistic",
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, extra windows, unrealistic lighting, structural changes",
"cfg": 7.0, "denoise": 0.70, "steps": 30,
},
"art_deco": {
"pos": "art deco luxury interior design, geometric chevron patterns, gold brass accents, rich velvet upholstery emerald and sapphire, sunburst mirrors, polished marble flooring, crystal chandeliers, 1920s glamour, 8k resolution, photorealistic",
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, rustic, minimalism, cheap materials",
"cfg": 7.5, "denoise": 0.72, "steps": 30,
},
"biophilic": {
"pos": "biophilic organic interior design, living green walls with ferns and moss, natural stone accent walls, rattan and bamboo furniture, abundant houseplants, earth tone sage green and terracotta, 8k resolution, photorealistic, dappled sunlight",
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, synthetic materials, plastic plants",
"cfg": 7.0, "denoise": 0.68, "steps": 30,
},
"cyberpunk": {
"pos": "cyberpunk neon interior design, high contrast LED strip lighting electric blue and hot pink, reflective chrome surfaces, dark matte walls, futuristic furniture, glowing circuit patterns, tech-noir blade runner aesthetic, 8k resolution, photorealistic, volumetric fog",
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, natural daylight, rustic elements",
"cfg": 8.0, "denoise": 0.75, "steps": 30,
},
"japandi": {
"pos": "japandi fusion interior design, wabi-sabi textures, low-profile furniture, muted earth tones warm grays soft browns, handmade ceramic accents, light ash wood, shoji screen elements, minimal decoration, zen garden elements, 8k resolution, photorealistic, serene",
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, bright colors, ornate decoration, cluttered",
"cfg": 6.5, "denoise": 0.70, "steps": 30,
},
}
def build_workflow(img_name: str, style: str = "scandinavian",
keywords: str = "", denoise_override: float = None) -> dict:
s = STYLE_LIBRARY.get(style, STYLE_LIBRARY["scandinavian"])
# Merge user keywords into base positive prompt
pos = s["pos"]
if keywords.strip():
pos = pos + ", " + keywords.strip()
cfg = s["cfg"]
denoise = denoise_override if denoise_override else s["denoise"]
steps = s["steps"]
neg = s["neg"]
return {
"1": {"class_type": "CheckpointLoaderSimple",
"inputs": {"ckpt_name": "realvisxlV50_v50LightningBakedvae.safetensors"}},
"2": {"class_type": "LoadImage",
"inputs": {"image": img_name, "upload": "image"}},
"3": {"class_type": "CLIPTextEncode",
"inputs": {"text": pos, "clip": ["1", 1]}}, # ← POSITIVE PROMPT
"4": {"class_type": "CLIPTextEncode",
"inputs": {"text": neg, "clip": ["1", 1]}}, # ← NEGATIVE PROMPT
"5": {"class_type": "VAEEncode",
"inputs": {"pixels": ["2", 0], "vae": ["1", 2]}},
"6": {"class_type": "KSampler",
"inputs": {"model": ["1", 0], "positive": ["3", 0], "negative": ["4", 0],
"latent_image": ["5", 0],
"seed": int(time.time()) % 999983,
"steps": steps, "cfg": cfg,
"sampler_name": "dpmpp_2m", "scheduler": "karras",
"denoise": denoise}},
"7": {"class_type": "VAEDecode",
"inputs": {"samples": ["6", 0], "vae": ["1", 2]}},
"8": {"class_type": "SaveImage",
"inputs": {"images": ["7", 0], "filename_prefix": f"dreamweaver_{style}"}}
}
```
**Also update the endpoint signatures** to accept `keywords`:
```python
@app.post("/dream-weaver")
async def dream_weaver(
image: UploadFile = File(...),
style: str = Form(default="scandinavian"),
keywords: str = Form(default=""), # ← ADD THIS
denoise: float = Form(default=0.0), # 0.0 = use style default
):
...
wf = build_workflow(comfy_name, style=style, keywords=keywords,
denoise_override=denoise if denoise > 0 else None)
```
---
## 5. Swift Data Models
```swift
// AppConfig.swift
struct AppConfig {
static let baseURL = "http://54.172.172.2:8080"
// Change this to HTTPS domain once SSL is set up
}
// Models
struct GenerationJob: Codable {
let jobId: String
let status: String
let pollUrl: String
let resultUrl: String
enum CodingKeys: String, CodingKey {
case jobId = "job_id"
case status, pollUrl = "poll_url", resultUrl = "result_url"
}
}
struct JobStatus: Codable {
let status: String
let ready: Bool
let resultUrl: String?
let error: String?
enum CodingKeys: String, CodingKey {
case status, ready, resultUrl = "result_url", error
}
}
struct HealthResponse: Codable {
let status: String
let comfyui: Bool
}
enum DreamWeaverError: Error {
case generationFailed(String)
case timeout
case invalidImageData
}
// Style model for the style picker
struct InteriorStyle: Identifiable {
let id: String // used as the `style` form field value
let displayName: String
let keywords: [String] // shown as tappable chips in UI
let previewImage: String // local asset name
}
let availableStyles: [InteriorStyle] = [
InteriorStyle(id: "scandinavian", displayName: "Scandinavian", keywords: ["oak","linen","white","hygge","cozy","birch"], previewImage: "style_scandi"),
InteriorStyle(id: "art_deco", displayName: "Art Deco Luxe", keywords: ["gold","marble","velvet","geometric","brass","crystal"], previewImage: "style_artdeco"),
InteriorStyle(id: "biophilic", displayName: "Biophilic", keywords: ["green wall","stone","rattan","terracotta","botanical"], previewImage: "style_biophilic"),
InteriorStyle(id: "cyberpunk", displayName: "Cyberpunk Neon", keywords: ["neon","chrome","LED","futuristic","dark","holographic"], previewImage: "style_cyberpunk"),
InteriorStyle(id: "japandi", displayName: "Japandi Fusion", keywords: ["wabi-sabi","ceramic","ash wood","zen","minimal"], previewImage: "style_japandi"),
]
```
---
## 6. Complete Generation Flow (UI → Server → Result)
```
1. User opens camera / library → picks a room photo
2. User selects a style card → style ID captured
3. User optionally taps keyword chips → keywords[] array built
4. User taps "Generate" →
POST /dream-weaver (multipart)
image: <jpeg data>
style: "art_deco"
keywords: "gold, marble, luxury hotel"
denoise: 0.72
5. Server returns { job_id: "abc123", status: "processing" }
6. App shows loading/progress UI
7. App polls GET /dream-weaver/status/abc123 every 2 seconds
8. When ready == true →
GET /dream-weaver/result/abc123 → returns PNG bytes
9. App displays result full-screen with save/share options
```
**Expected latency on 4× L4 GPU server:** `~15–20 seconds` end-to-end.
---
## 7. WebSocket Progress (Optional Advanced Feature)
If Sayan wants a real-time progress bar (e.g. "Step 12/30"), connect directly to ComfyUI's WebSocket **only if port 8188 is opened**. Otherwise, polling `/status` is sufficient.
```swift
// WebSocket — only if 8188 is exposed externally
class ComfyProgressWebSocket: NSObject, URLSessionWebSocketDelegate {
var onProgress: ((Int, Int) -> Void)?
var task: URLSessionWebSocketTask?
func connect(clientId: String) {
let url = URL(string: "ws://54.172.172.2:8188/ws?clientId=\(clientId)")!
task = URLSession.shared.webSocketTask(with: url)
task?.resume()
listen()
}
private func listen() {
task?.receive { [weak self] result in
if case .success(let message) = result,
case .string(let text) = message,
let data = text.data(using: .utf8),
let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
let type = json["type"] as? String, type == "progress",
let inner = json["data"] as? [String: Int] {
self?.onProgress?(inner["value"] ?? 0, inner["max"] ?? 30)
}
self?.listen() // recurse
}
}
}
```
> [!NOTE]
> Port 8188 is currently **not open externally** in the security group. Only port 8080 is. To use WebSocket progress, Sagnik needs to add an inbound rule for 8188. Until then, using `/status` polling every 2s gives good enough UX.
---
## 8. Error Handling
| HTTP Status | Meaning | UI Action |
|---|---|---|
| `200` | Success | Show result or job_id |
| `404` on `/status` | Job expired (> 30 min) | "Session expired. Please retry." |
| `500` | Generation failed (OOM, model error) | "Generation failed. Try a simpler image." |
| Connection error | Server down or no internet | "Checking server…" + retry logic |
The job [status](file:///C:/Windows/Temp/dw_gateway.py#154-164) field can also be `"error"` with an `error` field explaining what failed.
---
## 9. Quick Checklist for Sayan
- [ ] Update `AppConfig.swift` with `baseURL = "http://54.172.172.2:8080"`
- [ ] Implement `POST /dream-weaver` multipart with `image + style + keywords`
- [ ] Implement polling loop on `GET /dream-weaver/status/{job_id}`
- [ ] Implement image download from `GET /dream-weaver/result/{job_id}`
- [ ] Add `GET /health` check on app launch
- [ ] Build keyword chips UI with the 5 style palettes from Section 2.3
- [ ] Test with the 20 sample images in `comfy_engine/test_inputs/`
## 10. Quick Checklist for Sagnik (backend)
- [ ] Update [dw_gateway.py](file:///C:/Windows/Temp/dw_gateway.py) with the full `STYLE_LIBRARY` dict (Section 4)
- [ ] Add `keywords: str = Form(default="")` to both POST endpoints
- [ ] Pass keywords into [build_workflow()](file:///C:/Windows/Temp/dw_gateway.py#33-65) for prompt expansion
- [ ] Redeploy gateway on port 8080 (`nohup python3 dw_gateway.py &`)
- [ ] (Optional) Open port 8188 in security group for WebSocket progress

72
.Agent Context/Bibels/infrastructure_deployment_manifest.md Normal file
View File

@@ -0,0 +1,72 @@
# Dream Weaver — Infrastructure & Connectivity Manifest
**Environment:** Production AWS Node
**Last Verified:** 2026-03-14
**Status:** ✅ HEALTHY
---
## 🖥️ Server Instance
> [!IMPORTANT]
> **Active Public IP: `54.91.19.60`**
> The previous Elastic IP (`54.172.172.2`) is currently detached and will time out. Ensure all your connection strings use the active IP.
| Component | Value |
|---|---|
| **Instance ID** | `i-0e4eab5fe67cf9abe` |
| **Instance Name** | Desineuron AWS Node 4x L4 (96GB VRAM) Spot |
| **Public IP (Active)** | **`54.91.19.60`** |
| **Private IP** | `172.31.46.190` |
| **VPC ID** | `vpc-081d2397920aad268` |
---
## 🛡️ Security Group Settings
**Security Group ID:** `sg-0b144c17b1b89f4c6` (Synapse-Ops)
The following Inbound rules are explicitly confirmed and open:
| Protocol | Port | Source | Description |
|---|---|---|---|
| TCP | **22** | `0.0.0.0/0` | SSH Access |
| TCP | **8082** | `0.0.0.0/0` | **Dream Weaver API v2 (Current)** |
| TCP | **8188** | `0.0.0.0/0` | ComfyUI Internal API |
| TCP | **8000** | `0.0.0.0/0` | ComfyUI Web UI (Alternate) |
---
## 🚀 Services & Endpoints
### 1. Dream Weaver Gateway v2
* **Port:** `8082`
* **Status:** ✅ Active
* **Health Check:** `http://54.91.19.60:8082/health`
* **Main Endpoint:** `POST http://54.91.19.60:8082/dream-weaver`
### 2. ComfyUI Engine
* **Port:** `8188`
* **Status:** ✅ Active
* **Prompt Endpoint:** `POST http://54.91.19.60:8188/prompt`
---
## 🔑 SSH Configuration
**Local Key File Path:** `f:\Workin In Progress\DESINEURON\GITLAB\Project_Velocity\desineuron-l4-node.pem`
### Quick Connect Command
```bash
ssh -i "path/to/desineuron-l4-node.pem" ubuntu@54.91.19.60
```
---
## 📝 Operator Checklist (Troubleshooting)
1. **Verify API Process:**
`ps aux | grep dw_gateway_v2`
2. **Check Logs:**
`tail -f /home/ubuntu/gw_v2.log`
3. **Check Port Listeners:**
`sudo netstat -tulpn | grep 8082`
4. **No Zombie Processes:**
Port 8080 has been cleared. Only 8082 is serving the gateway.

342
comfy_engine/scripts/README.md
View File

@@ -1,171 +1,171 @@
# Dream Weaver Automation Scripts # Dream Weaver Automation Scripts
This directory contains Python automation scripts for the Dream Weaver interior restyling workflow. This directory contains Python automation scripts for the Dream Weaver interior restyling workflow.
## Scripts Overview ## Scripts Overview
### 1. dreamweaver_batch_processor.py ### 1. dreamweaver_batch_processor.py
Main batch processing controller for automated image restyling. Main batch processing controller for automated image restyling.
**Features:** **Features:**
- Directory monitoring for automatic job queueing - Directory monitoring for automatic job queueing
- Automatic mask caching for improved performance - Automatic mask caching for improved performance
- Queue management with status tracking - Queue management with status tracking
- Support for all three processing phases - Support for all three processing phases
- WebSocket integration with ComfyUI for real-time status - WebSocket integration with ComfyUI for real-time status
**Usage:** **Usage:**
```bash ```bash
# Process single image # Process single image
python dreamweaver_batch_processor.py --input image.jpg --style scandinavian_minimalist --phase 1 python dreamweaver_batch_processor.py --input image.jpg --style scandinavian_minimalist --phase 1
# Process all images in directory # Process all images in directory
python dreamweaver_batch_processor.py --batch --style art_deco_luxe --phase 2 python dreamweaver_batch_processor.py --batch --style art_deco_luxe --phase 2
# Start directory monitoring mode # Start directory monitoring mode
python dreamweaver_batch_processor.py --monitor python dreamweaver_batch_processor.py --monitor
``` ```
### 2. mask_preprocessor.py ### 2. mask_preprocessor.py
Utility for preprocessing and caching segmentation masks. Utility for preprocessing and caching segmentation masks.
**Features:** **Features:**
- Offline mask generation and caching - Offline mask generation and caching
- Mask refinement (grow, feather, invert) - Mask refinement (grow, feather, invert)
- Multi-region mask support (walls, floor, ceiling) - Multi-region mask support (walls, floor, ceiling)
- Batch preprocessing for entire directories - Batch preprocessing for entire directories
- Cache management and statistics - Cache management and statistics
**Usage:** **Usage:**
```bash ```bash
# Preprocess single image # Preprocess single image
python mask_preprocessor.py --image image.jpg python mask_preprocessor.py --image image.jpg
# Preprocess entire directory # Preprocess entire directory
python mask_preprocessor.py --directory ../test_inputs/ python mask_preprocessor.py --directory ../test_inputs/
# Show cache statistics # Show cache statistics
python mask_preprocessor.py --stats python mask_preprocessor.py --stats
# Clear all cached masks # Clear all cached masks
python mask_preprocessor.py --clear-cache python mask_preprocessor.py --clear-cache
# Custom mask parameters # Custom mask parameters
python mask_preprocessor.py --image image.jpg --grow 5 --feather 8 python mask_preprocessor.py --image image.jpg --grow 5 --feather 8
``` ```
## Configuration ## Configuration
Scripts use configuration from `CONFIG` dictionary in each file. Key settings: Scripts use configuration from `CONFIG` dictionary in each file. Key settings:
- `comfyui_server`: ComfyUI HTTP endpoint (default: http://localhost:8188) - `comfyui_server`: ComfyUI HTTP endpoint (default: http://localhost:8188)
- `comfyui_ws`: ComfyUI WebSocket endpoint (default: ws://localhost:8188/ws) - `comfyui_ws`: ComfyUI WebSocket endpoint (default: ws://localhost:8188/ws)
- `input_directory`: Default input images directory - `input_directory`: Default input images directory
- `output_directory`: Generated images output directory - `output_directory`: Generated images output directory
- `cache_directory`: Mask cache storage location - `cache_directory`: Mask cache storage location
- `batch_size`: Number of images to process in batch (Phase 3) - `batch_size`: Number of images to process in batch (Phase 3)
## Integration with ComfyUI ## Integration with ComfyUI
These scripts require ComfyUI to be running with the Dream Weaver workflows loaded. These scripts require ComfyUI to be running with the Dream Weaver workflows loaded.
**Starting ComfyUI:** **Starting ComfyUI:**
```bash ```bash
cd Project_Velocity/comfy_engine cd Project_Velocity/comfy_engine
python main.py --fp16 --lowvram python main.py --fp16 --lowvram
``` ```
**For Production (Dual RTX PRO 6000):** **For Production (Dual RTX PRO 6000):**
```bash ```bash
python main.py --bf16 --highvram --xformers --gpu-batch-size 8 python main.py --bf16 --highvram --xformers --gpu-batch-size 8
``` ```
## Workflow Files ## Workflow Files
Scripts reference these workflow JSON files: Scripts reference these workflow JSON files:
- `workflows/dreamweaver_phase1_depth.json` - Single ControlNet (RTX 3080Ti) - `workflows/dreamweaver_phase1_depth.json` - Single ControlNet (RTX 3080Ti)
- `workflows/dreamweaver_phase2_multicontrol.json` - Multi-ControlNet (RTX 3080Ti) - `workflows/dreamweaver_phase2_multicontrol.json` - Multi-ControlNet (RTX 3080Ti)
- `workflows/dreamweaver_phase3_batch.json` - Batch processing (Dual RTX PRO 6000) - `workflows/dreamweaver_phase3_batch.json` - Batch processing (Dual RTX PRO 6000)
## Style Templates ## Style Templates
Available style templates (located in `../prompts/`): Available style templates (located in `../prompts/`):
- `scandinavian_minimalist` - Light, airy Nordic design - `scandinavian_minimalist` - Light, airy Nordic design
- `art_deco_luxe` - Glamorous 1920s aesthetic - `art_deco_luxe` - Glamorous 1920s aesthetic
- `cyberpunk_neon` - High-tech futuristic - `cyberpunk_neon` - High-tech futuristic
- `biophilic_organic` - Nature-connected sustainable - `biophilic_organic` - Nature-connected sustainable
- `japandi_fusion` - Japanese-Scandinavian blend - `japandi_fusion` - Japanese-Scandinavian blend
## Dependencies ## Dependencies
Install required packages: Install required packages:
```bash ```bash
pip install -r ../requirements.txt pip install -r ../requirements.txt
``` ```
## Logging ## Logging
Scripts output logs to: Scripts output logs to:
- Console (real-time) - Console (real-time)
- `dreamweaver_batch.log` (file) - `dreamweaver_batch.log` (file)
Log level can be adjusted in script `logging.basicConfig()` calls. Log level can be adjusted in script `logging.basicConfig()` calls.
## Hardware Requirements ## Hardware Requirements
**Phase 1 & 2 (Development):** **Phase 1 & 2 (Development):**
- NVIDIA RTX 3080Ti (12GB VRAM) - NVIDIA RTX 3080Ti (12GB VRAM)
- 32GB System RAM - 32GB System RAM
- SSD Storage - SSD Storage
**Phase 3 (Production):** **Phase 3 (Production):**
- Dual NVIDIA RTX PRO 6000 Blackwell (96GB VRAM each) - Dual NVIDIA RTX PRO 6000 Blackwell (96GB VRAM each)
- 128GB System RAM - 128GB System RAM
- NVMe SSD Storage - NVMe SSD Storage
- NVLink enabled for GPU memory pooling - NVLink enabled for GPU memory pooling
## API Reference ## API Reference
### ComfyUI Endpoints Used ### ComfyUI Endpoints Used
- `POST /prompt` - Submit workflow to queue - `POST /prompt` - Submit workflow to queue
- `GET /queue` - Get queue status - `GET /queue` - Get queue status
- `WS /ws` - WebSocket for real-time updates - `WS /ws` - WebSocket for real-time updates
### Job Status Values ### Job Status Values
- `pending` - Waiting in queue - `pending` - Waiting in queue
- `processing` - Currently generating - `processing` - Currently generating
- `completed` - Successfully finished - `completed` - Successfully finished
- `failed` - Error occurred - `failed` - Error occurred
## Troubleshooting ## Troubleshooting
**Connection Refused Error:** **Connection Refused Error:**
- Ensure ComfyUI is running - Ensure ComfyUI is running
- Check server URL in configuration - Check server URL in configuration
- Verify firewall settings - Verify firewall settings
**Out of Memory:** **Out of Memory:**
- Reduce batch size - Reduce batch size
- Lower resolution - Lower resolution
- Enable tiled VAE decoding - Enable tiled VAE decoding
**Mask Cache Issues:** **Mask Cache Issues:**
- Clear cache: `python mask_preprocessor.py --clear-cache` - Clear cache: `python mask_preprocessor.py --clear-cache`
- Check cache directory permissions - Check cache directory permissions
- Verify available disk space - Verify available disk space
## Development ## Development
To extend functionality: To extend functionality:
1. Modify `BatchProcessor` class for new processing logic 1. Modify `BatchProcessor` class for new processing logic
2. Add new style templates in `../prompts/` 2. Add new style templates in `../prompts/`
3. Update workflow JSON files for new ControlNet configurations 3. Update workflow JSON files for new ControlNet configurations
## Support ## Support
For issues related to: For issues related to:
- **Scripts**: Check logs in `dreamweaver_batch.log` - **Scripts**: Check logs in `dreamweaver_batch.log`
- **ComfyUI**: Refer to ComfyUI documentation - **ComfyUI**: Refer to ComfyUI documentation
- **Workflows**: See technical specification in `../docs/DREAMWEAVER_TECHNICAL_SPEC.md` - **Workflows**: See technical specification in `../docs/DREAMWEAVER_TECHNICAL_SPEC.md`

1244
comfy_engine/scripts/a100_deployment_executor.py
View File

File diff suppressed because it is too large Load Diff

996
comfy_engine/scripts/dreamweaver_batch_processor.py
View File

@@ -1,498 +1,498 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Dream Weaver Batch Processor Dream Weaver Batch Processor
============================ ============================
Automated batch processing script for Dream Weaver interior restyling workflow. Automated batch processing script for Dream Weaver interior restyling workflow.
Handles directory monitoring, automatic mask caching, and queue management. Handles directory monitoring, automatic mask caching, and queue management.
Target Hardware: Dual NVIDIA RTX PRO 6000 Blackwell (96GB GDDR7 each) Target Hardware: Dual NVIDIA RTX PRO 6000 Blackwell (96GB GDDR7 each)
Author: Project Velocity Team Author: Project Velocity Team
Version: 1.0.0 Version: 1.0.0
""" """
import os import os
import sys import sys
import json import json
import time import time
import hashlib import hashlib
import asyncio import asyncio
import argparse import argparse
import logging import logging
from pathlib import Path from pathlib import Path
from datetime import datetime from datetime import datetime
from typing import Dict, List, Optional, Tuple from typing import Dict, List, Optional, Tuple
from dataclasses import dataclass, asdict from dataclasses import dataclass, asdict
import requests import requests
import websockets import websockets
import aiofiles import aiofiles
from watchdog.observers import Observer from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler from watchdog.events import FileSystemEventHandler
# Configuration # Configuration
CONFIG = { CONFIG = {
"comfyui_server": "http://localhost:8188", "comfyui_server": "http://localhost:8188",
"comfyui_ws": "ws://localhost:8188/ws", "comfyui_ws": "ws://localhost:8188/ws",
"input_directory": "Project_Velocity/comfy_engine/test_inputs/", "input_directory": "Project_Velocity/comfy_engine/test_inputs/",
"output_directory": "Project_Velocity/comfy_engine/test_outputs/", "output_directory": "Project_Velocity/comfy_engine/test_outputs/",
"cache_directory": "Project_Velocity/comfy_engine/cache/masks/", "cache_directory": "Project_Velocity/comfy_engine/cache/masks/",
"workflow_phase1": "Project_Velocity/comfy_engine/workflows/dreamweaver_phase1_depth.json", "workflow_phase1": "Project_Velocity/comfy_engine/workflows/dreamweaver_phase1_depth.json",
"workflow_phase2": "Project_Velocity/comfy_engine/workflows/dreamweaver_phase2_multicontrol.json", "workflow_phase2": "Project_Velocity/comfy_engine/workflows/dreamweaver_phase2_multicontrol.json",
"workflow_phase3": "Project_Velocity/comfy_engine/workflows/dreamweaver_phase3_batch.json", "workflow_phase3": "Project_Velocity/comfy_engine/workflows/dreamweaver_phase3_batch.json",
"batch_size": 8, "batch_size": 8,
"target_resolution": (1024, 1024), "target_resolution": (1024, 1024),
"enable_mask_cache": True, "enable_mask_cache": True,
"gpu_sharding": True, "gpu_sharding": True,
"dual_gpu": True, "dual_gpu": True,
} }
# Setup logging # Setup logging
logging.basicConfig( logging.basicConfig(
level=logging.INFO, level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
handlers=[ handlers=[
logging.FileHandler('dreamweaver_batch.log'), logging.FileHandler('dreamweaver_batch.log'),
logging.StreamHandler() logging.StreamHandler()
] ]
) )
logger = logging.getLogger('DreamWeaver') logger = logging.getLogger('DreamWeaver')
@dataclass @dataclass
class ProcessingJob: class ProcessingJob:
"""Represents a single image processing job.""" """Represents a single image processing job."""
job_id: str job_id: str
input_path: str input_path: str
output_path: str output_path: str
style_template: str style_template: str
phase: int phase: int
status: str = "pending" status: str = "pending"
created_at: datetime = None created_at: datetime = None
started_at: datetime = None started_at: datetime = None
completed_at: datetime = None completed_at: datetime = None
error_message: str = None error_message: str = None
mask_cached: bool = False mask_cached: bool = False
def __post_init__(self): def __post_init__(self):
if self.created_at is None: if self.created_at is None:
self.created_at = datetime.now() self.created_at = datetime.now()
def to_dict(self) -> Dict: def to_dict(self) -> Dict:
return { return {
"job_id": self.job_id, "job_id": self.job_id,
"input_path": self.input_path, "input_path": self.input_path,
"output_path": self.output_path, "output_path": self.output_path,
"style_template": self.style_template, "style_template": self.style_template,
"phase": self.phase, "phase": self.phase,
"status": self.status, "status": self.status,
"created_at": self.created_at.isoformat() if self.created_at else None, "created_at": self.created_at.isoformat() if self.created_at else None,
"started_at": self.started_at.isoformat() if self.started_at else None, "started_at": self.started_at.isoformat() if self.started_at else None,
"completed_at": self.completed_at.isoformat() if self.completed_at else None, "completed_at": self.completed_at.isoformat() if self.completed_at else None,
"error_message": self.error_message, "error_message": self.error_message,
"mask_cached": self.mask_cached "mask_cached": self.mask_cached
} }
class MaskCacheManager: class MaskCacheManager:
"""Manages caching of segmentation masks for improved performance.""" """Manages caching of segmentation masks for improved performance."""
def __init__(self, cache_dir: str): def __init__(self, cache_dir: str):
self.cache_dir = Path(cache_dir) self.cache_dir = Path(cache_dir)
self.cache_dir.mkdir(parents=True, exist_ok=True) self.cache_dir.mkdir(parents=True, exist_ok=True)
logger.info(f"Mask cache initialized at: {self.cache_dir}") logger.info(f"Mask cache initialized at: {self.cache_dir}")
def _get_cache_key(self, image_path: str) -> str: def _get_cache_key(self, image_path: str) -> str:
"""Generate cache key from image content hash.""" """Generate cache key from image content hash."""
hasher = hashlib.md5() hasher = hashlib.md5()
with open(image_path, 'rb') as f: with open(image_path, 'rb') as f:
hasher.update(f.read()) hasher.update(f.read())
return hasher.hexdigest() return hasher.hexdigest()
def get_cached_mask(self, image_path: str) -> Optional[str]: def get_cached_mask(self, image_path: str) -> Optional[str]:
"""Retrieve cached mask path if it exists.""" """Retrieve cached mask path if it exists."""
cache_key = self._get_cache_key(image_path) cache_key = self._get_cache_key(image_path)
cached_path = self.cache_dir / f"{cache_key}.png" cached_path = self.cache_dir / f"{cache_key}.png"
if cached_path.exists(): if cached_path.exists():
logger.info(f"Cache hit for {image_path}") logger.info(f"Cache hit for {image_path}")
return str(cached_path) return str(cached_path)
return None return None
def cache_mask(self, image_path: str, mask_path: str) -> str: def cache_mask(self, image_path: str, mask_path: str) -> str:
"""Cache a mask file for future use.""" """Cache a mask file for future use."""
cache_key = self._get_cache_key(image_path) cache_key = self._get_cache_key(image_path)
cached_path = self.cache_dir / f"{cache_key}.png" cached_path = self.cache_dir / f"{cache_key}.png"
import shutil import shutil
shutil.copy2(mask_path, cached_path) shutil.copy2(mask_path, cached_path)
logger.info(f"Cached mask for {image_path} at {cached_path}") logger.info(f"Cached mask for {image_path} at {cached_path}")
return str(cached_path) return str(cached_path)
class ComfyUIClient: class ComfyUIClient:
"""Client for communicating with ComfyUI server.""" """Client for communicating with ComfyUI server."""
def __init__(self, server_url: str, ws_url: str): def __init__(self, server_url: str, ws_url: str):
self.server_url = server_url self.server_url = server_url
self.ws_url = ws_url self.ws_url = ws_url
self.client_id = self._generate_client_id() self.client_id = self._generate_client_id()
logger.info(f"ComfyUI client initialized with ID: {self.client_id}") logger.info(f"ComfyUI client initialized with ID: {self.client_id}")
def _generate_client_id(self) -> str: def _generate_client_id(self) -> str:
"""Generate unique client ID.""" """Generate unique client ID."""
return f"dreamweaver_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{os.urandom(4).hex()}" return f"dreamweaver_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{os.urandom(4).hex()}"
async def submit_workflow(self, workflow: Dict, input_image: str) -> str: async def submit_workflow(self, workflow: Dict, input_image: str) -> str:
"""Submit a workflow to ComfyUI queue.""" """Submit a workflow to ComfyUI queue."""
# Update workflow with input image # Update workflow with input image
for node_id, node in workflow.items(): for node_id, node in workflow.items():
if node.get("class_type") == "LoadImage": if node.get("class_type") == "LoadImage":
node["inputs"]["image"] = input_image node["inputs"]["image"] = input_image
if node.get("class_type") == "LoadImageBatch": if node.get("class_type") == "LoadImageBatch":
node["inputs"]["directory"] = os.path.dirname(input_image) node["inputs"]["directory"] = os.path.dirname(input_image)
payload = { payload = {
"prompt": workflow, "prompt": workflow,
"client_id": self.client_id "client_id": self.client_id
} }
response = requests.post( response = requests.post(
f"{self.server_url}/prompt", f"{self.server_url}/prompt",
json=payload json=payload
) )
response.raise_for_status() response.raise_for_status()
result = response.json() result = response.json()
prompt_id = result.get("prompt_id") prompt_id = result.get("prompt_id")
logger.info(f"Submitted workflow with prompt_id: {prompt_id}") logger.info(f"Submitted workflow with prompt_id: {prompt_id}")
return prompt_id return prompt_id
async def get_queue_status(self) -> Dict: async def get_queue_status(self) -> Dict:
"""Get current queue status.""" """Get current queue status."""
response = requests.get(f"{self.server_url}/queue") response = requests.get(f"{self.server_url}/queue")
return response.json() return response.json()
async def wait_for_completion(self, prompt_id: str, timeout: int = 300) -> bool: async def wait_for_completion(self, prompt_id: str, timeout: int = 300) -> bool:
"""Wait for workflow completion via WebSocket.""" """Wait for workflow completion via WebSocket."""
start_time = time.time() start_time = time.time()
async with websockets.connect( async with websockets.connect(
f"{self.ws_url}?clientId={self.client_id}" f"{self.ws_url}?clientId={self.client_id}"
) as websocket: ) as websocket:
while time.time() - start_time < timeout: while time.time() - start_time < timeout:
try: try:
message = await asyncio.wait_for( message = await asyncio.wait_for(
websocket.recv(), websocket.recv(),
timeout=5.0 timeout=5.0
) )
data = json.loads(message) data = json.loads(message)
if data.get("type") == "executing": if data.get("type") == "executing":
if data["data"].get("prompt_id") == prompt_id: if data["data"].get("prompt_id") == prompt_id:
node_id = data["data"].get("node") node_id = data["data"].get("node")
logger.debug(f"Executing node: {node_id}") logger.debug(f"Executing node: {node_id}")
elif data.get("type") == "completed": elif data.get("type") == "completed":
if data["data"].get("prompt_id") == prompt_id: if data["data"].get("prompt_id") == prompt_id:
logger.info(f"Workflow {prompt_id} completed") logger.info(f"Workflow {prompt_id} completed")
return True return True
elif data.get("type") == "error": elif data.get("type") == "error":
logger.error(f"Workflow error: {data}") logger.error(f"Workflow error: {data}")
return False return False
except asyncio.TimeoutError: except asyncio.TimeoutError:
continue continue
logger.warning(f"Workflow {prompt_id} timed out") logger.warning(f"Workflow {prompt_id} timed out")
return False return False
class BatchProcessor: class BatchProcessor:
"""Main batch processing controller.""" """Main batch processing controller."""
def __init__(self, config: Dict): def __init__(self, config: Dict):
self.config = config self.config = config
self.queue: List[ProcessingJob] = [] self.queue: List[ProcessingJob] = []
self.processing = False self.processing = False
self.cache_manager = MaskCacheManager(config["cache_directory"]) self.cache_manager = MaskCacheManager(config["cache_directory"])
self.comfy_client = ComfyUIClient( self.comfy_client = ComfyUIClient(
config["comfyui_server"], config["comfyui_server"],
config["comfyui_ws"] config["comfyui_ws"]
) )
# Load workflow templates # Load workflow templates
self.workflows = self._load_workflows() self.workflows = self._load_workflows()
# Ensure output directory exists # Ensure output directory exists
Path(config["output_directory"]).mkdir(parents=True, exist_ok=True) Path(config["output_directory"]).mkdir(parents=True, exist_ok=True)
def _load_workflows(self) -> Dict[int, Dict]: def _load_workflows(self) -> Dict[int, Dict]:
"""Load workflow JSON files.""" """Load workflow JSON files."""
workflows = {} workflows = {}
workflow_paths = { workflow_paths = {
1: self.config["workflow_phase1"], 1: self.config["workflow_phase1"],
2: self.config["workflow_phase2"], 2: self.config["workflow_phase2"],
3: self.config["workflow_phase3"] 3: self.config["workflow_phase3"]
} }
for phase, path in workflow_paths.items(): for phase, path in workflow_paths.items():
try: try:
with open(path, 'r') as f: with open(path, 'r') as f:
workflows[phase] = json.load(f) workflows[phase] = json.load(f)
logger.info(f"Loaded Phase {phase} workflow") logger.info(f"Loaded Phase {phase} workflow")
except Exception as e: except Exception as e:
logger.error(f"Failed to load Phase {phase} workflow: {e}") logger.error(f"Failed to load Phase {phase} workflow: {e}")
return workflows return workflows
def add_job(self, input_path: str, style_template: str = "scandinavian_minimalist", phase: int = 1) -> str: def add_job(self, input_path: str, style_template: str = "scandinavian_minimalist", phase: int = 1) -> str:
"""Add a new processing job to the queue.""" """Add a new processing job to the queue."""
job_id = hashlib.md5(f"{input_path}_{time.time()}".encode()).hexdigest()[:12] job_id = hashlib.md5(f"{input_path}_{time.time()}".encode()).hexdigest()[:12]
output_filename = f"{Path(input_path).stem}_restyled_{job_id}.png" output_filename = f"{Path(input_path).stem}_restyled_{job_id}.png"
output_path = os.path.join(self.config["output_directory"], output_filename) output_path = os.path.join(self.config["output_directory"], output_filename)
job = ProcessingJob( job = ProcessingJob(
job_id=job_id, job_id=job_id,
input_path=input_path, input_path=input_path,
output_path=output_path, output_path=output_path,
style_template=style_template, style_template=style_template,
phase=phase phase=phase
) )
# Check if mask is cached # Check if mask is cached
if self.config["enable_mask_cache"]: if self.config["enable_mask_cache"]:
cached_mask = self.cache_manager.get_cached_mask(input_path) cached_mask = self.cache_manager.get_cached_mask(input_path)
job.mask_cached = cached_mask is not None job.mask_cached = cached_mask is not None
self.queue.append(job) self.queue.append(job)
logger.info(f"Added job {job_id} to queue. Queue size: {len(self.queue)}") logger.info(f"Added job {job_id} to queue. Queue size: {len(self.queue)}")
return job_id return job_id
async def process_single(self, job: ProcessingJob) -> bool: async def process_single(self, job: ProcessingJob) -> bool:
"""Process a single job.""" """Process a single job."""
job.status = "processing" job.status = "processing"
job.started_at = datetime.now() job.started_at = datetime.now()
try: try:
logger.info(f"Processing job {job.job_id}: {job.input_path}") logger.info(f"Processing job {job.job_id}: {job.input_path}")
# Get workflow for phase # Get workflow for phase
workflow = self.workflows.get(job.phase) workflow = self.workflows.get(job.phase)
if not workflow: if not workflow:
raise ValueError(f"Workflow for phase {job.phase} not found") raise ValueError(f"Workflow for phase {job.phase} not found")
# Submit to ComfyUI # Submit to ComfyUI
prompt_id = await self.comfy_client.submit_workflow( prompt_id = await self.comfy_client.submit_workflow(
workflow, workflow,
job.input_path job.input_path
) )
# Wait for completion # Wait for completion
success = await self.comfy_client.wait_for_completion(prompt_id) success = await self.comfy_client.wait_for_completion(prompt_id)
if success: if success:
job.status = "completed" job.status = "completed"
job.completed_at = datetime.now() job.completed_at = datetime.now()
logger.info(f"Job {job.job_id} completed successfully") logger.info(f"Job {job.job_id} completed successfully")
return True return True
else: else:
job.status = "failed" job.status = "failed"
job.error_message = "Workflow execution failed or timed out" job.error_message = "Workflow execution failed or timed out"
logger.error(f"Job {job.job_id} failed") logger.error(f"Job {job.job_id} failed")
return False return False
except Exception as e: except Exception as e:
job.status = "failed" job.status = "failed"
job.error_message = str(e) job.error_message = str(e)
logger.error(f"Error processing job {job.job_id}: {e}") logger.error(f"Error processing job {job.job_id}: {e}")
return False return False
async def process_batch(self, jobs: List[ProcessingJob]) -> List[bool]: async def process_batch(self, jobs: List[ProcessingJob]) -> List[bool]:
"""Process multiple jobs in batch (Phase 3).""" """Process multiple jobs in batch (Phase 3)."""
if not jobs: if not jobs:
return [] return []
logger.info(f"Processing batch of {len(jobs)} jobs") logger.info(f"Processing batch of {len(jobs)} jobs")
results = [] results = []
# For batch processing, use Phase 3 workflow # For batch processing, use Phase 3 workflow
workflow = self.workflows.get(3) workflow = self.workflows.get(3)
if not workflow: if not workflow:
logger.warning("Phase 3 workflow not available, processing sequentially") logger.warning("Phase 3 workflow not available, processing sequentially")
for job in jobs: for job in jobs:
result = await self.process_single(job) result = await self.process_single(job)
results.append(result) results.append(result)
return results return results
# TODO: Implement true batch processing with Phase 3 workflow # TODO: Implement true batch processing with Phase 3 workflow
# This would require grouping images and processing together # This would require grouping images and processing together
for job in jobs: for job in jobs:
result = await self.process_single(job) result = await self.process_single(job)
results.append(result) results.append(result)
return results return results
async def run(self): async def run(self):
"""Main processing loop.""" """Main processing loop."""
logger.info("Starting batch processor") logger.info("Starting batch processor")
self.processing = True self.processing = True
while self.processing: while self.processing:
# Get pending jobs # Get pending jobs
pending_jobs = [j for j in self.queue if j.status == "pending"] pending_jobs = [j for j in self.queue if j.status == "pending"]
if not pending_jobs: if not pending_jobs:
await asyncio.sleep(1) await asyncio.sleep(1)
continue continue
# Check if batch processing is appropriate # Check if batch processing is appropriate
if len(pending_jobs) >= self.config["batch_size"] and self.config.get("dual_gpu"): if len(pending_jobs) >= self.config["batch_size"] and self.config.get("dual_gpu"):
# Process in batches for Phase 3 # Process in batches for Phase 3
batch = pending_jobs[:self.config["batch_size"]] batch = pending_jobs[:self.config["batch_size"]]
await self.process_batch(batch) await self.process_batch(batch)
else: else:
# Process single job with appropriate phase # Process single job with appropriate phase
job = pending_jobs[0] job = pending_jobs[0]
await self.process_single(job) await self.process_single(job)
def stop(self): def stop(self):
"""Stop the processing loop.""" """Stop the processing loop."""
logger.info("Stopping batch processor") logger.info("Stopping batch processor")
self.processing = False self.processing = False
def get_status(self) -> Dict: def get_status(self) -> Dict:
"""Get current processing status.""" """Get current processing status."""
total = len(self.queue) total = len(self.queue)
pending = len([j for j in self.queue if j.status == "pending"]) pending = len([j for j in self.queue if j.status == "pending"])
processing = len([j for j in self.queue if j.status == "processing"]) processing = len([j for j in self.queue if j.status == "processing"])
completed = len([j for j in self.queue if j.status == "completed"]) completed = len([j for j in self.queue if j.status == "completed"])
failed = len([j for j in self.queue if j.status == "failed"]) failed = len([j for j in self.queue if j.status == "failed"])
return { return {
"total_jobs": total, "total_jobs": total,
"pending": pending, "pending": pending,
"processing": processing, "processing": processing,
"completed": completed, "completed": completed,
"failed": failed, "failed": failed,
"is_running": self.processing "is_running": self.processing
} }
class InputDirectoryHandler(FileSystemEventHandler): class InputDirectoryHandler(FileSystemEventHandler):
"""Handles new file events in input directory.""" """Handles new file events in input directory."""
def __init__(self, processor: BatchProcessor): def __init__(self, processor: BatchProcessor):
self.processor = processor self.processor = processor
def on_created(self, event): def on_created(self, event):
if not event.is_directory: if not event.is_directory:
file_path = event.src_path file_path = event.src_path
if file_path.lower().endswith(('.jpg', '.jpeg', '.png', '.webp')): if file_path.lower().endswith(('.jpg', '.jpeg', '.png', '.webp')):
logger.info(f"New image detected: {file_path}") logger.info(f"New image detected: {file_path}")
self.processor.add_job(file_path) self.processor.add_job(file_path)
def load_style_template(template_name: str) -> str: def load_style_template(template_name: str) -> str:
"""Load a style template from prompts directory.""" """Load a style template from prompts directory."""
template_path = Path("Project_Velocity/comfy_engine/prompts/") / f"{template_name}.txt" template_path = Path("Project_Velocity/comfy_engine/prompts/") / f"{template_name}.txt"
if template_path.exists(): if template_path.exists():
with open(template_path, 'r') as f: with open(template_path, 'r') as f:
content = f.read() content = f.read()
# Extract positive prompt # Extract positive prompt
lines = content.split('\n') lines = content.split('\n')
positive_lines = [] positive_lines = []
in_positive = False in_positive = False
for line in lines: for line in lines:
if 'POSITIVE PROMPT:' in line: if 'POSITIVE PROMPT:' in line:
in_positive = True in_positive = True
continue continue
if in_positive and line.startswith('Style Weight:'): if in_positive and line.startswith('Style Weight:'):
break break
if in_positive and line.strip() and not line.startswith('-'): if in_positive and line.strip() and not line.startswith('-'):
positive_lines.append(line.strip()) positive_lines.append(line.strip())
return ' '.join(positive_lines) return ' '.join(positive_lines)
return "" return ""
async def main(): async def main():
"""Main entry point.""" """Main entry point."""
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
description="Dream Weaver Batch Processor" description="Dream Weaver Batch Processor"
) )
parser.add_argument( parser.add_argument(
"--monitor", "--monitor",
action="store_true", action="store_true",
help="Enable directory monitoring mode" help="Enable directory monitoring mode"
) )
parser.add_argument( parser.add_argument(
"--input", "--input",
type=str, type=str,
help="Single input image to process" help="Single input image to process"
) )
parser.add_argument( parser.add_argument(
"--style", "--style",
type=str, type=str,
default="scandinavian_minimalist", default="scandinavian_minimalist",
choices=["scandinavian_minimalist", "art_deco_luxe", "cyberpunk_neon", "biophilic_organic", "japandi_fusion"], choices=["scandinavian_minimalist", "art_deco_luxe", "cyberpunk_neon", "biophilic_organic", "japandi_fusion"],
help="Style template to apply" help="Style template to apply"
) )
parser.add_argument( parser.add_argument(
"--phase", "--phase",
type=int, type=int,
default=1, default=1,
choices=[1, 2, 3], choices=[1, 2, 3],
help="Processing phase to use" help="Processing phase to use"
) )
parser.add_argument( parser.add_argument(
"--batch", "--batch",
action="store_true", action="store_true",
help="Process all images in input directory" help="Process all images in input directory"
) )
args = parser.parse_args() args = parser.parse_args()
# Initialize processor # Initialize processor
processor = BatchProcessor(CONFIG) processor = BatchProcessor(CONFIG)
if args.input: if args.input:
# Process single image # Process single image
job_id = processor.add_job(args.input, args.style, args.phase) job_id = processor.add_job(args.input, args.style, args.phase)
await processor.process_single(processor.queue[-1]) await processor.process_single(processor.queue[-1])
print(f"Processed image: {args.input}") print(f"Processed image: {args.input}")
print(f"Job ID: {job_id}") print(f"Job ID: {job_id}")
elif args.batch: elif args.batch:
# Process all images in directory # Process all images in directory
input_dir = Path(CONFIG["input_directory"]) input_dir = Path(CONFIG["input_directory"])
image_files = list(input_dir.glob("*.jpg")) + list(input_dir.glob("*.png")) image_files = list(input_dir.glob("*.jpg")) + list(input_dir.glob("*.png"))
for img_file in image_files: for img_file in image_files:
processor.add_job(str(img_file), args.style, args.phase) processor.add_job(str(img_file), args.style, args.phase)
await processor.run() await processor.run()
elif args.monitor: elif args.monitor:
# Start directory monitoring # Start directory monitoring
event_handler = InputDirectoryHandler(processor) event_handler = InputDirectoryHandler(processor)
observer = Observer() observer = Observer()
observer.schedule( observer.schedule(
event_handler, event_handler,
CONFIG["input_directory"], CONFIG["input_directory"],
recursive=False recursive=False
) )
observer.start() observer.start()
logger.info(f"Started monitoring: {CONFIG['input_directory']}") logger.info(f"Started monitoring: {CONFIG['input_directory']}")
try: try:
# Run processor # Run processor
await processor.run() await processor.run()
except KeyboardInterrupt: except KeyboardInterrupt:
processor.stop() processor.stop()
observer.stop() observer.stop()
observer.join() observer.join()
else: else:
print("No action specified. Use --help for usage information.") print("No action specified. Use --help for usage information.")
if __name__ == "__main__": if __name__ == "__main__":
asyncio.run(main()) asyncio.run(main())

840
comfy_engine/scripts/dw_gateway_v2.py
View File

@@ -1,420 +1,420 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Dream Weaver API Gateway v2 — Dynamic Keyword → Local LLM → ComfyUI Pipeline Dream Weaver API Gateway v2 — Dynamic Keyword → Local LLM → ComfyUI Pipeline
======================================================================== ========================================================================
Port: 8080 (public-facing) Port: 8080 (public-facing)
ComfyUI: localhost:8188 (internal) ComfyUI: localhost:8188 (internal)
NEW IN v2: NEW IN v2:
- POST /dream-weaver now accepts keywords[] + room_type for LLM-based prompt generation - POST /dream-weaver now accepts keywords[] + room_type for LLM-based prompt generation
- POST /dream-weaver/expand — expand keywords to prompt WITHOUT generating (preview) - POST /dream-weaver/expand — expand keywords to prompt WITHOUT generating (preview)
- GET /room-types — list available room types - GET /room-types — list available room types
- Uses local Ollama model (qwen3.5:27b) for prompt expansion (no cloud API dependencies) - Uses local Ollama model (qwen3.5:27b) for prompt expansion (no cloud API dependencies)
Environment variables: Environment variables:
OLLAMA_URL — Ollama server (default: http://localhost:11434) OLLAMA_URL — Ollama server (default: http://localhost:11434)
OLLAMA_MODEL — Model name (default: qwen3.5:27b) OLLAMA_MODEL — Model name (default: qwen3.5:27b)
""" """
import asyncio, json, time, uuid, io, sys, os, logging import asyncio, json, time, uuid, io, sys, os, logging
from pathlib import Path from pathlib import Path
from typing import Optional, List from typing import Optional, List
import httpx import httpx
import uvicorn import uvicorn
from fastapi import FastAPI, UploadFile, File, HTTPException, Form, BackgroundTasks from fastapi import FastAPI, UploadFile, File, HTTPException, Form, BackgroundTasks
from fastapi.responses import JSONResponse, StreamingResponse from fastapi.responses import JSONResponse, StreamingResponse
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel from pydantic import BaseModel
# Add scripts dir to path so we can import prompt_expander # Add scripts dir to path so we can import prompt_expander
SCRIPTS_DIR = Path(__file__).parent / "scripts" SCRIPTS_DIR = Path(__file__).parent / "scripts"
sys.path.insert(0, str(SCRIPTS_DIR)) sys.path.insert(0, str(SCRIPTS_DIR))
try: try:
from prompt_expander import expand_prompt, expand_prompt_simple, ROOM_CONTEXTS, ExpandedPrompt from prompt_expander import expand_prompt, expand_prompt_simple, ROOM_CONTEXTS, ExpandedPrompt
LLM_AVAILABLE = True LLM_AVAILABLE = True
except ImportError: except ImportError:
LLM_AVAILABLE = False LLM_AVAILABLE = False
logging.warning("prompt_expander not found — LLM expansion disabled") logging.warning("prompt_expander not found — LLM expansion disabled")
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s") logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
logger = logging.getLogger("DreamWeaverGateway") logger = logging.getLogger("DreamWeaverGateway")
COMFY = "http://127.0.0.1:8188" COMFY = "http://127.0.0.1:8188"
COMFY_ROOT = "/opt/dlami/nvme/ComfyUI" COMFY_ROOT = "/opt/dlami/nvme/ComfyUI"
app = FastAPI( app = FastAPI(
title="Dream Weaver API v2", title="Dream Weaver API v2",
version="2.0.0", version="2.0.0",
description="Dynamic keyword-to-interior-design generation powered by LLM + ComfyUI" description="Dynamic keyword-to-interior-design generation powered by LLM + ComfyUI"
) )
app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"]) app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"])
# In-memory job store (swap for Redis in production) # In-memory job store (swap for Redis in production)
jobs: dict = {} jobs: dict = {}
# ─── Models ────────────────────────────────────────────────────────────────── # ─── Models ──────────────────────────────────────────────────────────────────
class ExpandRequest(BaseModel): class ExpandRequest(BaseModel):
keywords: List[str] keywords: List[str]
room_type: str = "living_room" room_type: str = "living_room"
additional_notes: str = "" additional_notes: str = ""
class ExpandResponse(BaseModel): class ExpandResponse(BaseModel):
style_name: str style_name: str
positive_prompt: str positive_prompt: str
negative_prompt: str negative_prompt: str
cfg: float cfg: float
denoise: float denoise: float
steps: int steps: int
reasoning: str reasoning: str
source: str source: str
# ─── ComfyUI helpers ────────────────────────────────────────────────────────── # ─── ComfyUI helpers ──────────────────────────────────────────────────────────
async def upload_to_comfy(data: bytes, filename: str) -> str: async def upload_to_comfy(data: bytes, filename: str) -> str:
async with httpx.AsyncClient(timeout=30) as client: async with httpx.AsyncClient(timeout=30) as client:
r = await client.post(f"{COMFY}/upload/image", r = await client.post(f"{COMFY}/upload/image",
files={"image": (filename, data, "image/jpeg")}, files={"image": (filename, data, "image/jpeg")},
data={"overwrite": "true"}) data={"overwrite": "true"})
r.raise_for_status() r.raise_for_status()
return r.json()["name"] return r.json()["name"]
def build_workflow(img_name: str, expanded: "ExpandedPrompt") -> dict: def build_workflow(img_name: str, expanded: "ExpandedPrompt") -> dict:
"""Build ComfyUI API workflow from an ExpandedPrompt result.""" """Build ComfyUI API workflow from an ExpandedPrompt result."""
return { return {
"1": {"class_type": "CheckpointLoaderSimple", "1": {"class_type": "CheckpointLoaderSimple",
"inputs": {"ckpt_name": "realvisxlV50_v50LightningBakedvae.safetensors"}}, "inputs": {"ckpt_name": "realvisxlV50_v50LightningBakedvae.safetensors"}},
"2": {"class_type": "LoadImage", "2": {"class_type": "LoadImage",
"inputs": {"image": img_name, "upload": "image"}}, "inputs": {"image": img_name, "upload": "image"}},
"3": {"class_type": "CLIPTextEncode", # Positive prompt "3": {"class_type": "CLIPTextEncode", # Positive prompt
"inputs": {"text": expanded.positive_prompt, "clip": ["1", 1]}}, "inputs": {"text": expanded.positive_prompt, "clip": ["1", 1]}},
"4": {"class_type": "CLIPTextEncode", # Negative prompt "4": {"class_type": "CLIPTextEncode", # Negative prompt
"inputs": {"text": expanded.negative_prompt, "clip": ["1", 1]}}, "inputs": {"text": expanded.negative_prompt, "clip": ["1", 1]}},
"5": {"class_type": "VAEEncode", "5": {"class_type": "VAEEncode",
"inputs": {"pixels": ["2", 0], "vae": ["1", 2]}}, "inputs": {"pixels": ["2", 0], "vae": ["1", 2]}},
"6": {"class_type": "KSampler", "6": {"class_type": "KSampler",
"inputs": {"model": ["1", 0], "inputs": {"model": ["1", 0],
"positive": ["3", 0], "positive": ["3", 0],
"negative": ["4", 0], "negative": ["4", 0],
"latent_image": ["5", 0], "latent_image": ["5", 0],
"seed": int(time.time()) % 999983, "seed": int(time.time()) % 999983,
"steps": expanded.steps, "steps": expanded.steps,
"cfg": expanded.cfg, "cfg": expanded.cfg,
"sampler_name": "dpmpp_2m", "sampler_name": "dpmpp_2m",
"scheduler": "karras", "scheduler": "karras",
"denoise": expanded.denoise}}, "denoise": expanded.denoise}},
"7": {"class_type": "VAEDecode", "7": {"class_type": "VAEDecode",
"inputs": {"samples": ["6", 0], "vae": ["1", 2]}}, "inputs": {"samples": ["6", 0], "vae": ["1", 2]}},
"8": {"class_type": "SaveImage", "8": {"class_type": "SaveImage",
"inputs": {"images": ["7", 0], "inputs": {"images": ["7", 0],
"filename_prefix": f"dw_{expanded.style_name.replace(' ', '_')[:30]}"}}, "filename_prefix": f"dw_{expanded.style_name.replace(' ', '_')[:30]}"}},
} }
async def queue_prompt(workflow: dict) -> str: async def queue_prompt(workflow: dict) -> str:
async with httpx.AsyncClient(timeout=30) as client: async with httpx.AsyncClient(timeout=30) as client:
r = await client.post(f"{COMFY}/prompt", r = await client.post(f"{COMFY}/prompt",
json={"prompt": workflow, "client_id": str(uuid.uuid4())}) json={"prompt": workflow, "client_id": str(uuid.uuid4())})
r.raise_for_status() r.raise_for_status()
return r.json()["prompt_id"] return r.json()["prompt_id"]
async def poll_result(prompt_id: str, timeout: int = 300): async def poll_result(prompt_id: str, timeout: int = 300):
start = time.time() start = time.time()
async with httpx.AsyncClient(timeout=10) as client: async with httpx.AsyncClient(timeout=10) as client:
while time.time() - start < timeout: while time.time() - start < timeout:
r = await client.get(f"{COMFY}/history/{prompt_id}") r = await client.get(f"{COMFY}/history/{prompt_id}")
if r.status_code == 200: if r.status_code == 200:
h = r.json().get(prompt_id, {}) h = r.json().get(prompt_id, {})
if h.get("status", {}).get("status_str") == "error": if h.get("status", {}).get("status_str") == "error":
return None, h.get("status", {}).get("messages", ["unknown"]) return None, h.get("status", {}).get("messages", ["unknown"])
imgs = [img for nd in h.get("outputs", {}).values() imgs = [img for nd in h.get("outputs", {}).values()
for img in nd.get("images", [])] for img in nd.get("images", [])]
if imgs: if imgs:
return imgs[0], None return imgs[0], None
await asyncio.sleep(2) await asyncio.sleep(2)
return None, "timeout" return None, "timeout"
async def background_poll(job_id: str, prompt_id: str): async def background_poll(job_id: str, prompt_id: str):
img, err = await poll_result(prompt_id) img, err = await poll_result(prompt_id)
if img: if img:
jobs[job_id].update({"status": "done", "output": img, "completed": time.time()}) jobs[job_id].update({"status": "done", "output": img, "completed": time.time()})
else: else:
jobs[job_id].update({"status": "error", "error": str(err)}) jobs[job_id].update({"status": "error", "error": str(err)})
# ─── Endpoints ─────────────────────────────────────────────────────────────── # ─── Endpoints ───────────────────────────────────────────────────────────────
@app.get("/health") @app.get("/health")
async def health(): async def health():
comfy_ok = False comfy_ok = False
try: try:
async with httpx.AsyncClient(timeout=5) as c: async with httpx.AsyncClient(timeout=5) as c:
r = await c.get(f"{COMFY}/system_stats") r = await c.get(f"{COMFY}/system_stats")
comfy_ok = r.status_code == 200 comfy_ok = r.status_code == 200
except Exception: except Exception:
pass pass
return { return {
"status": "ok", "status": "ok",
"comfyui": comfy_ok, "comfyui": comfy_ok,
"gpu": "4x NVIDIA L4 (96GB VRAM)", "gpu": "4x NVIDIA L4 (96GB VRAM)",
"model": "RealVisXL V5.0 Lightning", "model": "RealVisXL V5.0 Lightning",
"llm_expansion": LLM_AVAILABLE, "llm_expansion": LLM_AVAILABLE,
"version": "2.0.0" "version": "2.0.0"
} }
@app.get("/room-types") @app.get("/room-types")
async def room_types(): async def room_types():
"""List all supported room types with their context.""" """List all supported room types with their context."""
if not LLM_AVAILABLE: if not LLM_AVAILABLE:
return {"room_types": ["bedroom", "living_room", "bathroom", "kitchen", return {"room_types": ["bedroom", "living_room", "bathroom", "kitchen",
"dining_room", "home_office", "hallway", "balcony"]} "dining_room", "home_office", "hallway", "balcony"]}
return { return {
"room_types": { "room_types": {
k: { k: {
"description": v["description"], "description": v["description"],
"key_elements": v["key_elements"] "key_elements": v["key_elements"]
} }
for k, v in ROOM_CONTEXTS.items() for k, v in ROOM_CONTEXTS.items()
} }
} }
@app.post("/dream-weaver/expand", response_model=ExpandResponse) @app.post("/dream-weaver/expand", response_model=ExpandResponse)
async def expand_endpoint(req: ExpandRequest): async def expand_endpoint(req: ExpandRequest):
""" """
Preview the LLM-generated prompt WITHOUT submitting to ComfyUI. Preview the LLM-generated prompt WITHOUT submitting to ComfyUI.
Use this to let the user review/edit the prompt before generating. Use this to let the user review/edit the prompt before generating.
Request body: Request body:
{ {
"keywords": ["blue marble", "gold veins", "renaissance", "sharp contours"], "keywords": ["blue marble", "gold veins", "renaissance", "sharp contours"],
"room_type": "bedroom", "room_type": "bedroom",
"additional_notes": "luxury hotel feel" "additional_notes": "luxury hotel feel"
} }
""" """
if not req.keywords: if not req.keywords:
raise HTTPException(status_code=400, detail="keywords list cannot be empty") raise HTTPException(status_code=400, detail="keywords list cannot be empty")
try: try:
if LLM_AVAILABLE: if LLM_AVAILABLE:
result = await asyncio.to_thread( result = await asyncio.to_thread(
expand_prompt, expand_prompt,
keywords=req.keywords, keywords=req.keywords,
room_type=req.room_type, room_type=req.room_type,
additional_notes=req.additional_notes additional_notes=req.additional_notes
) )
else: else:
result = expand_prompt_simple(req.keywords, req.room_type) result = expand_prompt_simple(req.keywords, req.room_type)
except Exception as e: except Exception as e:
logger.error(f"Prompt expansion failed: {e}") logger.error(f"Prompt expansion failed: {e}")
raise HTTPException(status_code=500, detail=f"LLM expansion failed: {str(e)}") raise HTTPException(status_code=500, detail=f"LLM expansion failed: {str(e)}")
return ExpandResponse( return ExpandResponse(
style_name=result.style_name, style_name=result.style_name,
positive_prompt=result.positive_prompt, positive_prompt=result.positive_prompt,
negative_prompt=result.negative_prompt, negative_prompt=result.negative_prompt,
cfg=result.cfg, cfg=result.cfg,
denoise=result.denoise, denoise=result.denoise,
steps=result.steps, steps=result.steps,
reasoning=result.reasoning, reasoning=result.reasoning,
source=result.source source=result.source
) )
@app.post("/dream-weaver") @app.post("/dream-weaver")
async def dream_weaver( async def dream_weaver(
image: UploadFile = File(...), image: UploadFile = File(...),
# ── Dynamic keyword mode (new) ── # ── Dynamic keyword mode (new) ──
keywords: str = Form(default=""), # comma-separated: "blue marble, gold, renaissance" keywords: str = Form(default=""), # comma-separated: "blue marble, gold, renaissance"
room_type: str = Form(default="living_room"), room_type: str = Form(default="living_room"),
additional_notes: str = Form(default=""), additional_notes: str = Form(default=""),
# ── Optional overrides ── # ── Optional overrides ──
custom_positive: str = Form(default=""), # skip LLM, use this prompt directly custom_positive: str = Form(default=""), # skip LLM, use this prompt directly
custom_negative: str = Form(default=""), custom_negative: str = Form(default=""),
denoise: float = Form(default=0.0), # 0.0 = use LLM recommendation denoise: float = Form(default=0.0), # 0.0 = use LLM recommendation
cfg_scale: float = Form(default=0.0), # 0.0 = use LLM recommendation cfg_scale: float = Form(default=0.0), # 0.0 = use LLM recommendation
): ):
""" """
Submit a room photo for AI redesign using dynamic keyword → LLM → ComfyUI pipeline. Submit a room photo for AI redesign using dynamic keyword → LLM → ComfyUI pipeline.
Two modes: Two modes:
1. KEYWORD MODE (recommended): Provide keywords + room_type, LLM generates prompt 1. KEYWORD MODE (recommended): Provide keywords + room_type, LLM generates prompt
2. DIRECT MODE: Provide custom_positive + custom_negative to bypass LLM 2. DIRECT MODE: Provide custom_positive + custom_negative to bypass LLM
Returns job_id for async polling. Returns job_id for async polling.
""" """
job_id = str(uuid.uuid4()) job_id = str(uuid.uuid4())
jobs[job_id] = {"status": "uploading", "created": time.time()} jobs[job_id] = {"status": "uploading", "created": time.time()}
try: try:
# Upload image to ComfyUI # Upload image to ComfyUI
data = await image.read() data = await image.read()
filename = f"dw_{job_id[:8]}_{image.filename or 'room.jpg'}" filename = f"dw_{job_id[:8]}_{image.filename or 'room.jpg'}"
comfy_name = await upload_to_comfy(data, filename) comfy_name = await upload_to_comfy(data, filename)
jobs[job_id]["status"] = "expanding_prompt" jobs[job_id]["status"] = "expanding_prompt"
# ── Determine prompt ────────────────────────────────────────────── # ── Determine prompt ──────────────────────────────────────────────
if custom_positive: if custom_positive:
# Direct mode — user provided prompts explicitly # Direct mode — user provided prompts explicitly
from dataclasses import dataclass from dataclasses import dataclass
@dataclass @dataclass
class DirectPrompt: class DirectPrompt:
style_name: str = "custom" style_name: str = "custom"
positive_prompt: str = custom_positive positive_prompt: str = custom_positive
negative_prompt: str = custom_negative or ( negative_prompt: str = custom_negative or (
"(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), " "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), "
"blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes" "blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes"
) )
cfg: float = cfg_scale or 7.5 cfg: float = cfg_scale or 7.5
denoise: float = denoise or 0.72 denoise: float = denoise or 0.72
steps: int = 30 steps: int = 30
reasoning: str = "Direct user input" reasoning: str = "Direct user input"
source: str = "direct" source: str = "direct"
expanded = DirectPrompt() expanded = DirectPrompt()
elif keywords: elif keywords:
# Keyword mode — expand via LLM # Keyword mode — expand via LLM
kw_list = [k.strip() for k in keywords.split(",") if k.strip()] kw_list = [k.strip() for k in keywords.split(",") if k.strip()]
if LLM_AVAILABLE: if LLM_AVAILABLE:
expanded = await asyncio.to_thread( expanded = await asyncio.to_thread(
expand_prompt, expand_prompt,
keywords=kw_list, keywords=kw_list,
room_type=room_type, room_type=room_type,
additional_notes=additional_notes additional_notes=additional_notes
) )
else: else:
expanded = expand_prompt_simple(kw_list, room_type) expanded = expand_prompt_simple(kw_list, room_type)
# Apply manual overrides if provided # Apply manual overrides if provided
if denoise > 0: if denoise > 0:
expanded.denoise = denoise expanded.denoise = denoise
if cfg_scale > 0: if cfg_scale > 0:
expanded.cfg = cfg_scale expanded.cfg = cfg_scale
else: else:
raise HTTPException(status_code=400, raise HTTPException(status_code=400,
detail="Provide either 'keywords' or 'custom_positive'") detail="Provide either 'keywords' or 'custom_positive'")
jobs[job_id].update({ jobs[job_id].update({
"status": "queued", "status": "queued",
"style": expanded.style_name, "style": expanded.style_name,
"prompt_source": expanded.source, "prompt_source": expanded.source,
"positive_prompt": expanded.positive_prompt, "positive_prompt": expanded.positive_prompt,
"negative_prompt": expanded.negative_prompt, "negative_prompt": expanded.negative_prompt,
"room_type": room_type, "room_type": room_type,
}) })
# Submit workflow # Submit workflow
wf = build_workflow(comfy_name, expanded) wf = build_workflow(comfy_name, expanded)
prompt_id = await queue_prompt(wf) prompt_id = await queue_prompt(wf)
jobs[job_id].update({"status": "processing", "prompt_id": prompt_id}) jobs[job_id].update({"status": "processing", "prompt_id": prompt_id})
# Start background polling # Start background polling
asyncio.create_task(background_poll(job_id, prompt_id)) asyncio.create_task(background_poll(job_id, prompt_id))
return { return {
"job_id": job_id, "job_id": job_id,
"status": "processing", "status": "processing",
"style": expanded.style_name, "style": expanded.style_name,
"prompt_preview": expanded.positive_prompt[:120] + "...", "prompt_preview": expanded.positive_prompt[:120] + "...",
"reasoning": expanded.reasoning, "reasoning": expanded.reasoning,
"poll_url": f"/dream-weaver/status/{job_id}", "poll_url": f"/dream-weaver/status/{job_id}",
"result_url": f"/dream-weaver/result/{job_id}" "result_url": f"/dream-weaver/result/{job_id}"
} }
except HTTPException: except HTTPException:
raise raise
except Exception as e: except Exception as e:
jobs[job_id] = {"status": "error", "error": str(e)} jobs[job_id] = {"status": "error", "error": str(e)}
logger.error(f"Generation failed: {e}") logger.error(f"Generation failed: {e}")
raise HTTPException(status_code=500, detail=str(e)) raise HTTPException(status_code=500, detail=str(e))
@app.get("/dream-weaver/status/{job_id}") @app.get("/dream-weaver/status/{job_id}")
async def status(job_id: str): async def status(job_id: str):
job = jobs.get(job_id) job = jobs.get(job_id)
if not job: if not job:
raise HTTPException(status_code=404, detail="Job not found") raise HTTPException(status_code=404, detail="Job not found")
result = {k: v for k, v in job.items() if k != "output"} result = {k: v for k, v in job.items() if k != "output"}
result["ready"] = job.get("status") == "done" result["ready"] = job.get("status") == "done"
if result["ready"]: if result["ready"]:
result["result_url"] = f"/dream-weaver/result/{job_id}" result["result_url"] = f"/dream-weaver/result/{job_id}"
return result return result
@app.get("/dream-weaver/result/{job_id}") @app.get("/dream-weaver/result/{job_id}")
async def result(job_id: str): async def result(job_id: str):
job = jobs.get(job_id) job = jobs.get(job_id)
if not job or job.get("status") != "done": if not job or job.get("status") != "done":
raise HTTPException(status_code=404, detail="Result not ready") raise HTTPException(status_code=404, detail="Result not ready")
img = job["output"] img = job["output"]
url = (f"{COMFY}/view?filename={img['filename']}" url = (f"{COMFY}/view?filename={img['filename']}"
f"&subfolder={img.get('subfolder','')}&type={img.get('type','output')}") f"&subfolder={img.get('subfolder','')}&type={img.get('type','output')}")
async with httpx.AsyncClient(timeout=30) as c: async with httpx.AsyncClient(timeout=30) as c:
r = await c.get(url) r = await c.get(url)
return StreamingResponse( return StreamingResponse(
io.BytesIO(r.content), io.BytesIO(r.content),
media_type="image/png", media_type="image/png",
headers={"Content-Disposition": f"attachment; filename=dreamweaver_{job_id[:8]}.png"} headers={"Content-Disposition": f"attachment; filename=dreamweaver_{job_id[:8]}.png"}
) )
@app.post("/dream-weaver/sync") @app.post("/dream-weaver/sync")
async def dream_weaver_sync( async def dream_weaver_sync(
image: UploadFile = File(...), image: UploadFile = File(...),
keywords: str = Form(default=""), keywords: str = Form(default=""),
room_type: str = Form(default="living_room"), room_type: str = Form(default="living_room"),
additional_notes: str = Form(default=""), additional_notes: str = Form(default=""),
custom_positive: str = Form(default=""), custom_positive: str = Form(default=""),
custom_negative: str = Form(default=""), custom_negative: str = Form(default=""),
): ):
""" """
Blocking version — waits up to 120s and returns image bytes directly. Blocking version — waits up to 120s and returns image bytes directly.
Use for testing. Prefer async /dream-weaver for production. Use for testing. Prefer async /dream-weaver for production.
""" """
data = await image.read() data = await image.read()
filename = f"sync_{uuid.uuid4().hex[:8]}_{image.filename or 'room.jpg'}" filename = f"sync_{uuid.uuid4().hex[:8]}_{image.filename or 'room.jpg'}"
comfy_name = await upload_to_comfy(data, filename) comfy_name = await upload_to_comfy(data, filename)
if custom_positive: if custom_positive:
from dataclasses import dataclass from dataclasses import dataclass
@dataclass @dataclass
class _P: class _P:
style_name = "custom" style_name = "custom"
positive_prompt = custom_positive positive_prompt = custom_positive
negative_prompt = custom_negative or "(worst quality, low quality), blurry, structural changes" negative_prompt = custom_negative or "(worst quality, low quality), blurry, structural changes"
cfg = 7.5; denoise = 0.72; steps = 30 cfg = 7.5; denoise = 0.72; steps = 30
reasoning = ""; source = "direct" reasoning = ""; source = "direct"
expanded = _P() expanded = _P()
elif keywords: elif keywords:
kw_list = [k.strip() for k in keywords.split(",") if k.strip()] kw_list = [k.strip() for k in keywords.split(",") if k.strip()]
expanded = (expand_prompt(kw_list, room_type, additional_notes) expanded = (expand_prompt(kw_list, room_type, additional_notes)
if LLM_AVAILABLE else expand_prompt_simple(kw_list, room_type)) if LLM_AVAILABLE else expand_prompt_simple(kw_list, room_type))
else: else:
raise HTTPException(status_code=400, detail="Provide keywords or custom_positive") raise HTTPException(status_code=400, detail="Provide keywords or custom_positive")
wf = build_workflow(comfy_name, expanded) wf = build_workflow(comfy_name, expanded)
prompt_id = await queue_prompt(wf) prompt_id = await queue_prompt(wf)
img, err = await poll_result(prompt_id, timeout=120) img, err = await poll_result(prompt_id, timeout=120)
if err: if err:
raise HTTPException(status_code=500, detail=str(err)) raise HTTPException(status_code=500, detail=str(err))
url = (f"{COMFY}/view?filename={img['filename']}" url = (f"{COMFY}/view?filename={img['filename']}"
f"&subfolder={img.get('subfolder','')}&type={img.get('type','output')}") f"&subfolder={img.get('subfolder','')}&type={img.get('type','output')}")
async with httpx.AsyncClient(timeout=30) as c: async with httpx.AsyncClient(timeout=30) as c:
r = await c.get(url) r = await c.get(url)
return StreamingResponse(io.BytesIO(r.content), media_type="image/png", return StreamingResponse(io.BytesIO(r.content), media_type="image/png",
headers={"X-Style": expanded.style_name, headers={"X-Style": expanded.style_name,
"X-Prompt-Source": expanded.source}) "X-Prompt-Source": expanded.source})
if __name__ == "__main__": if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=int(os.environ.get("PORT", "8082")), log_level="info") uvicorn.run(app, host="0.0.0.0", port=int(os.environ.get("PORT", "8082")), log_level="info")

776
comfy_engine/scripts/mask_preprocessor.py
View File

@@ -1,388 +1,388 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Dream Weaver Mask Preprocessor Dream Weaver Mask Preprocessor
============================== ==============================
Utility script for preprocessing and caching segmentation masks. Utility script for preprocessing and caching segmentation masks.
Enables offline mask generation to speed up production workflows. Enables offline mask generation to speed up production workflows.
Target Hardware: Dual NVIDIA RTX PRO 6000 Blackwell Target Hardware: Dual NVIDIA RTX PRO 6000 Blackwell
Author: Project Velocity Team Author: Project Velocity Team
Version: 1.0.0 Version: 1.0.0
""" """
import os import os
import sys import sys
import json import json
import hashlib import hashlib
import argparse import argparse
import logging import logging
from pathlib import Path from pathlib import Path
from typing import List, Optional, Tuple, Dict from typing import List, Optional, Tuple, Dict
from dataclasses import dataclass from dataclasses import dataclass
import numpy as np import numpy as np
from PIL import Image from PIL import Image
import cv2 import cv2
# Configure logging # Configure logging
logging.basicConfig( logging.basicConfig(
level=logging.INFO, level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
) )
logger = logging.getLogger('MaskPreprocessor') logger = logging.getLogger('MaskPreprocessor')
@dataclass @dataclass
class MaskConfig: class MaskConfig:
"""Configuration for mask generation.""" """Configuration for mask generation."""
grow_pixels: int = 3 grow_pixels: int = 3
feather_pixels: int = 5 feather_pixels: int = 5
threshold: float = 0.3 threshold: float = 0.3
target_classes: List[str] = None target_classes: List[str] = None
def __post_init__(self): def __post_init__(self):
if self.target_classes is None: if self.target_classes is None:
self.target_classes = ["wall", "floor", "ceiling"] self.target_classes = ["wall", "floor", "ceiling"]
class MaskPreprocessor: class MaskPreprocessor:
"""Preprocesses and caches segmentation masks for Dream Weaver.""" """Preprocesses and caches segmentation masks for Dream Weaver."""
def __init__(self, cache_dir: str = "Project_Velocity/comfy_engine/cache/masks/"): def __init__(self, cache_dir: str = "Project_Velocity/comfy_engine/cache/masks/"):
self.cache_dir = Path(cache_dir) self.cache_dir = Path(cache_dir)
self.cache_dir.mkdir(parents=True, exist_ok=True) self.cache_dir.mkdir(parents=True, exist_ok=True)
self.config = MaskConfig() self.config = MaskConfig()
logger.info(f"MaskPreprocessor initialized. Cache directory: {self.cache_dir}") logger.info(f"MaskPreprocessor initialized. Cache directory: {self.cache_dir}")
def _get_image_hash(self, image_path: str) -> str: def _get_image_hash(self, image_path: str) -> str:
"""Generate MD5 hash of image for caching.""" """Generate MD5 hash of image for caching."""
hasher = hashlib.md5() hasher = hashlib.md5()
with open(image_path, 'rb') as f: with open(image_path, 'rb') as f:
hasher.update(f.read()) hasher.update(f.read())
return hasher.hexdigest() return hasher.hexdigest()
def _get_cache_path(self, image_path: str, suffix: str = "") -> Path: def _get_cache_path(self, image_path: str, suffix: str = "") -> Path:
"""Generate cache file path for an image.""" """Generate cache file path for an image."""
image_hash = self._get_image_hash(image_path) image_hash = self._get_image_hash(image_path)
filename = f"{image_hash}{suffix}.png" filename = f"{image_hash}{suffix}.png"
return self.cache_dir / filename return self.cache_dir / filename
def is_cached(self, image_path: str, suffix: str = "") -> bool: def is_cached(self, image_path: str, suffix: str = "") -> bool:
"""Check if a mask is already cached.""" """Check if a mask is already cached."""
cache_path = self._get_cache_path(image_path, suffix) cache_path = self._get_cache_path(image_path, suffix)
return cache_path.exists() return cache_path.exists()
def load_from_cache(self, image_path: str, suffix: str = "") -> Optional[np.ndarray]: def load_from_cache(self, image_path: str, suffix: str = "") -> Optional[np.ndarray]:
"""Load mask from cache if available.""" """Load mask from cache if available."""
cache_path = self._get_cache_path(image_path, suffix) cache_path = self._get_cache_path(image_path, suffix)
if cache_path.exists(): if cache_path.exists():
logger.info(f"Loading cached mask from {cache_path}") logger.info(f"Loading cached mask from {cache_path}")
mask = cv2.imread(str(cache_path), cv2.IMREAD_GRAYSCALE) mask = cv2.imread(str(cache_path), cv2.IMREAD_GRAYSCALE)
return mask return mask
return None return None
def save_to_cache(self, image_path: str, mask: np.ndarray, suffix: str = "") -> str: def save_to_cache(self, image_path: str, mask: np.ndarray, suffix: str = "") -> str:
"""Save mask to cache.""" """Save mask to cache."""
cache_path = self._get_cache_path(image_path, suffix) cache_path = self._get_cache_path(image_path, suffix)
cv2.imwrite(str(cache_path), mask) cv2.imwrite(str(cache_path), mask)
logger.info(f"Saved mask to cache: {cache_path}") logger.info(f"Saved mask to cache: {cache_path}")
return str(cache_path) return str(cache_path)
def create_structural_mask(self, image_path: str, mask_data: np.ndarray) -> np.ndarray: def create_structural_mask(self, image_path: str, mask_data: np.ndarray) -> np.ndarray:
""" """
Create a structural preservation mask from segmentation data. Create a structural preservation mask from segmentation data.
This mask identifies walls, floors, ceilings that must be preserved. This mask identifies walls, floors, ceilings that must be preserved.
""" """
# Ensure binary mask # Ensure binary mask
if len(mask_data.shape) == 3: if len(mask_data.shape) == 3:
mask_data = cv2.cvtColor(mask_data, cv2.COLOR_BGR2GRAY) mask_data = cv2.cvtColor(mask_data, cv2.COLOR_BGR2GRAY)
_, binary_mask = cv2.threshold( _, binary_mask = cv2.threshold(
mask_data, mask_data,
int(255 * self.config.threshold), int(255 * self.config.threshold),
255, 255,
cv2.THRESH_BINARY cv2.THRESH_BINARY
) )
return binary_mask.astype(np.uint8) return binary_mask.astype(np.uint8)
def grow_mask(self, mask: np.ndarray, pixels: int = None) -> np.ndarray: def grow_mask(self, mask: np.ndarray, pixels: int = None) -> np.ndarray:
""" """
Grow (dilate) the mask by specified pixels. Grow (dilate) the mask by specified pixels.
This prevents edge bleeding by expanding the mask slightly. This prevents edge bleeding by expanding the mask slightly.
""" """
if pixels is None: if pixels is None:
pixels = self.config.grow_pixels pixels = self.config.grow_pixels
kernel = np.ones((pixels * 2 + 1, pixels * 2 + 1), np.uint8) kernel = np.ones((pixels * 2 + 1, pixels * 2 + 1), np.uint8)
grown_mask = cv2.dilate(mask, kernel, iterations=1) grown_mask = cv2.dilate(mask, kernel, iterations=1)
return grown_mask return grown_mask
def feather_mask(self, mask: np.ndarray, pixels: int = None) -> np.ndarray: def feather_mask(self, mask: np.ndarray, pixels: int = None) -> np.ndarray:
""" """
Apply Gaussian blur to feather mask edges. Apply Gaussian blur to feather mask edges.
Creates smooth transitions at boundaries. Creates smooth transitions at boundaries.
""" """
if pixels is None: if pixels is None:
pixels = self.config.feather_pixels pixels = self.config.feather_pixels
# Ensure odd kernel size # Ensure odd kernel size
kernel_size = pixels * 2 + 1 kernel_size = pixels * 2 + 1
feathered = cv2.GaussianBlur(mask, (kernel_size, kernel_size), 0) feathered = cv2.GaussianBlur(mask, (kernel_size, kernel_size), 0)
return feathered return feathered
def invert_mask(self, mask: np.ndarray) -> np.ndarray: def invert_mask(self, mask: np.ndarray) -> np.ndarray:
"""Invert mask (structural -> stylable or vice versa).""" """Invert mask (structural -> stylable or vice versa)."""
return cv2.bitwise_not(mask) return cv2.bitwise_not(mask)
def combine_masks(self, masks: List[np.ndarray], operation: str = "union") -> np.ndarray: def combine_masks(self, masks: List[np.ndarray], operation: str = "union") -> np.ndarray:
""" """
Combine multiple masks. Combine multiple masks.
operation: 'union' (OR), 'intersection' (AND), 'difference' operation: 'union' (OR), 'intersection' (AND), 'difference'
""" """
if not masks: if not masks:
return None return None
result = masks[0].copy() result = masks[0].copy()
for mask in masks[1:]: for mask in masks[1:]:
if operation == "union": if operation == "union":
result = cv2.bitwise_or(result, mask) result = cv2.bitwise_or(result, mask)
elif operation == "intersection": elif operation == "intersection":
result = cv2.bitwise_and(result, mask) result = cv2.bitwise_and(result, mask)
elif operation == "difference": elif operation == "difference":
result = cv2.bitwise_and(result, cv2.bitwise_not(mask)) result = cv2.bitwise_and(result, cv2.bitwise_not(mask))
return result return result
def create_multi_region_mask( def create_multi_region_mask(
self, self,
image_path: str, image_path: str,
regions: Dict[str, np.ndarray] regions: Dict[str, np.ndarray]
) -> Dict[str, np.ndarray]: ) -> Dict[str, np.ndarray]:
""" """
Create masks for multiple regions (walls, floor, ceiling, etc.) Create masks for multiple regions (walls, floor, ceiling, etc.)
Returns dictionary of processed masks. Returns dictionary of processed masks.
""" """
processed_masks = {} processed_masks = {}
for region_name, mask_data in regions.items(): for region_name, mask_data in regions.items():
logger.info(f"Processing mask for region: {region_name}") logger.info(f"Processing mask for region: {region_name}")
# Create base mask # Create base mask
base_mask = self.create_structural_mask(image_path, mask_data) base_mask = self.create_structural_mask(image_path, mask_data)
# Grow mask to prevent edge bleeding # Grow mask to prevent edge bleeding
grown_mask = self.grow_mask(base_mask) grown_mask = self.grow_mask(base_mask)
# Feather edges # Feather edges
feathered_mask = self.feather_mask(grown_mask) feathered_mask = self.feather_mask(grown_mask)
# Cache the processed mask # Cache the processed mask
cache_path = self.save_to_cache( cache_path = self.save_to_cache(
image_path, image_path,
feathered_mask, feathered_mask,
suffix=f"_{region_name}" suffix=f"_{region_name}"
) )
processed_masks[region_name] = { processed_masks[region_name] = {
"mask": feathered_mask, "mask": feathered_mask,
"cache_path": cache_path "cache_path": cache_path
} }
# Create combined structural mask # Create combined structural mask
all_structural = [m["mask"] for m in processed_masks.values()] all_structural = [m["mask"] for m in processed_masks.values()]
combined_structural = self.combine_masks(all_structural, operation="union") combined_structural = self.combine_masks(all_structural, operation="union")
# Create stylable mask (inverse of structural) # Create stylable mask (inverse of structural)
stylable_mask = self.invert_mask(combined_structural) stylable_mask = self.invert_mask(combined_structural)
# Save combined masks # Save combined masks
structural_cache = self.save_to_cache( structural_cache = self.save_to_cache(
image_path, image_path,
combined_structural, combined_structural,
suffix="_structural" suffix="_structural"
) )
stylable_cache = self.save_to_cache( stylable_cache = self.save_to_cache(
image_path, image_path,
stylable_mask, stylable_mask,
suffix="_stylable" suffix="_stylable"
) )
processed_masks["combined_structural"] = { processed_masks["combined_structural"] = {
"mask": combined_structural, "mask": combined_structural,
"cache_path": structural_cache "cache_path": structural_cache
} }
processed_masks["stylable"] = { processed_masks["stylable"] = {
"mask": stylable_mask, "mask": stylable_mask,
"cache_path": stylable_cache "cache_path": stylable_cache
} }
return processed_masks return processed_masks
def preprocess_image(self, image_path: str) -> Dict: def preprocess_image(self, image_path: str) -> Dict:
""" """
Complete preprocessing pipeline for a single image. Complete preprocessing pipeline for a single image.
Returns metadata about generated masks. Returns metadata about generated masks.
""" """
logger.info(f"Preprocessing image: {image_path}") logger.info(f"Preprocessing image: {image_path}")
# Check if already cached # Check if already cached
if self.is_cached(image_path, "_structural"): if self.is_cached(image_path, "_structural"):
logger.info(f"Image already preprocessed: {image_path}") logger.info(f"Image already preprocessed: {image_path}")
return { return {
"image_path": image_path, "image_path": image_path,
"cached": True, "cached": True,
"masks": { "masks": {
"structural": str(self._get_cache_path(image_path, "_structural")), "structural": str(self._get_cache_path(image_path, "_structural")),
"stylable": str(self._get_cache_path(image_path, "_stylable")) "stylable": str(self._get_cache_path(image_path, "_stylable"))
} }
} }
# Load image for reference # Load image for reference
img = cv2.imread(image_path) img = cv2.imread(image_path)
if img is None: if img is None:
raise ValueError(f"Could not load image: {image_path}") raise ValueError(f"Could not load image: {image_path}")
height, width = img.shape[:2] height, width = img.shape[:2]
# Create placeholder masks (in production, these would come from SAM) # Create placeholder masks (in production, these would come from SAM)
# This simulates wall, floor, ceiling segmentation # This simulates wall, floor, ceiling segmentation
regions = {} regions = {}
# Wall mask (upper portion) # Wall mask (upper portion)
wall_mask = np.zeros((height, width), dtype=np.uint8) wall_mask = np.zeros((height, width), dtype=np.uint8)
wall_mask[0:int(height*0.6), :] = 255 wall_mask[0:int(height*0.6), :] = 255
regions["wall"] = wall_mask regions["wall"] = wall_mask
# Floor mask (lower portion) # Floor mask (lower portion)
floor_mask = np.zeros((height, width), dtype=np.uint8) floor_mask = np.zeros((height, width), dtype=np.uint8)
floor_mask[int(height*0.6):, :] = 255 floor_mask[int(height*0.6):, :] = 255
regions["floor"] = floor_mask regions["floor"] = floor_mask
# Ceiling mask (top portion) # Ceiling mask (top portion)
ceiling_mask = np.zeros((height, width), dtype=np.uint8) ceiling_mask = np.zeros((height, width), dtype=np.uint8)
ceiling_mask[0:int(height*0.15), :] = 255 ceiling_mask[0:int(height*0.15), :] = 255
regions["ceiling"] = ceiling_mask regions["ceiling"] = ceiling_mask
# Process all regions # Process all regions
processed = self.create_multi_region_mask(image_path, regions) processed = self.create_multi_region_mask(image_path, regions)
return { return {
"image_path": image_path, "image_path": image_path,
"cached": False, "cached": False,
"dimensions": (width, height), "dimensions": (width, height),
"masks": { "masks": {
name: data["cache_path"] name: data["cache_path"]
for name, data in processed.items() for name, data in processed.items()
} }
} }
def batch_preprocess(self, directory: str, pattern: str = "*.jpg") -> List[Dict]: def batch_preprocess(self, directory: str, pattern: str = "*.jpg") -> List[Dict]:
"""Preprocess all images in a directory.""" """Preprocess all images in a directory."""
input_dir = Path(directory) input_dir = Path(directory)
image_files = list(input_dir.glob(pattern)) image_files = list(input_dir.glob(pattern))
image_files.extend(list(input_dir.glob("*.png"))) image_files.extend(list(input_dir.glob("*.png")))
results = [] results = []
for img_file in image_files: for img_file in image_files:
try: try:
result = self.preprocess_image(str(img_file)) result = self.preprocess_image(str(img_file))
results.append(result) results.append(result)
except Exception as e: except Exception as e:
logger.error(f"Failed to preprocess {img_file}: {e}") logger.error(f"Failed to preprocess {img_file}: {e}")
return results return results
def clear_cache(self): def clear_cache(self):
"""Clear all cached masks.""" """Clear all cached masks."""
for cache_file in self.cache_dir.glob("*.png"): for cache_file in self.cache_dir.glob("*.png"):
cache_file.unlink() cache_file.unlink()
logger.info("Cache cleared") logger.info("Cache cleared")
def get_cache_stats(self) -> Dict: def get_cache_stats(self) -> Dict:
"""Get cache statistics.""" """Get cache statistics."""
cache_files = list(self.cache_dir.glob("*.png")) cache_files = list(self.cache_dir.glob("*.png"))
total_size = sum(f.stat().st_size for f in cache_files) total_size = sum(f.stat().st_size for f in cache_files)
return { return {
"cached_files": len(cache_files), "cached_files": len(cache_files),
"total_size_mb": total_size / (1024 * 1024), "total_size_mb": total_size / (1024 * 1024),
"cache_directory": str(self.cache_dir) "cache_directory": str(self.cache_dir)
} }
def main(): def main():
"""Main entry point for command-line usage.""" """Main entry point for command-line usage."""
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
description="Dream Weaver Mask Preprocessor" description="Dream Weaver Mask Preprocessor"
) )
parser.add_argument( parser.add_argument(
"--image", "--image",
type=str, type=str,
help="Single image to preprocess" help="Single image to preprocess"
) )
parser.add_argument( parser.add_argument(
"--directory", "--directory",
type=str, type=str,
help="Directory of images to preprocess" help="Directory of images to preprocess"
) )
parser.add_argument( parser.add_argument(
"--cache-dir", "--cache-dir",
type=str, type=str,
default="Project_Velocity/comfy_engine/cache/masks/", default="Project_Velocity/comfy_engine/cache/masks/",
help="Cache directory for masks" help="Cache directory for masks"
) )
parser.add_argument( parser.add_argument(
"--grow", "--grow",
type=int, type=int,
default=3, default=3,
help="Pixels to grow mask (dilation)" help="Pixels to grow mask (dilation)"
) )
parser.add_argument( parser.add_argument(
"--feather", "--feather",
type=int, type=int,
default=5, default=5,
help="Pixels to feather mask edges" help="Pixels to feather mask edges"
) )
parser.add_argument( parser.add_argument(
"--clear-cache", "--clear-cache",
action="store_true", action="store_true",
help="Clear all cached masks" help="Clear all cached masks"
) )
parser.add_argument( parser.add_argument(
"--stats", "--stats",
action="store_true", action="store_true",
help="Show cache statistics" help="Show cache statistics"
) )
args = parser.parse_args() args = parser.parse_args()
# Initialize preprocessor # Initialize preprocessor
preprocessor = MaskPreprocessor(cache_dir=args.cache_dir) preprocessor = MaskPreprocessor(cache_dir=args.cache_dir)
preprocessor.config.grow_pixels = args.grow preprocessor.config.grow_pixels = args.grow
preprocessor.config.feather_pixels = args.feather preprocessor.config.feather_pixels = args.feather
if args.clear_cache: if args.clear_cache:
preprocessor.clear_cache() preprocessor.clear_cache()
return return
if args.stats: if args.stats:
stats = preprocessor.get_cache_stats() stats = preprocessor.get_cache_stats()
print(json.dumps(stats, indent=2)) print(json.dumps(stats, indent=2))
return return
if args.image: if args.image:
result = preprocessor.preprocess_image(args.image) result = preprocessor.preprocess_image(args.image)
print(json.dumps(result, indent=2)) print(json.dumps(result, indent=2))
elif args.directory: elif args.directory:
results = preprocessor.batch_preprocess(args.directory) results = preprocessor.batch_preprocess(args.directory)
print(json.dumps(results, indent=2)) print(json.dumps(results, indent=2))
print(f"\nProcessed {len(results)} images") print(f"\nProcessed {len(results)} images")
else: else:
print("No action specified. Use --help for usage information.") print("No action specified. Use --help for usage information.")
if __name__ == "__main__": if __name__ == "__main__":
main() main()

412
comfy_engine/scripts/prompt_expander.py
View File

@@ -1,206 +1,206 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Dream Weaver — Local LLM Prompt Expander Dream Weaver — Local LLM Prompt Expander
======================================== ========================================
Converts user keywords + room type into a photorealistic interior design prompt Converts user keywords + room type into a photorealistic interior design prompt
using a local Ollama model (default: qwen3.5:27b). using a local Ollama model (default: qwen3.5:27b).
Cloud API calls (Gemini, OpenAI) have been completely removed for data privacy Cloud API calls (Gemini, OpenAI) have been completely removed for data privacy
and local inference requirements. and local inference requirements.
Usage: Usage:
from prompt_expander import expand_prompt from prompt_expander import expand_prompt
result = expand_prompt( result = expand_prompt(
keywords=["blue marble", "gold veins", "renaissance", "sharp contours"], keywords=["blue marble", "gold veins", "renaissance", "sharp contours"],
room_type="bedroom" room_type="bedroom"
) )
""" """
import os import os
import json import json
import logging import logging
import requests import requests
import re import re
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# ── Room-type context injected into every LLM call ─────────────────────────── # ── Room-type context injected into every LLM call ───────────────────────────
ROOM_CONTEXTS = { ROOM_CONTEXTS = {
"bedroom": { "bedroom": {
"description": "a private sleeping space", "description": "a private sleeping space",
"key_elements": ["bed", "bedside tables", "wardrobe", "soft lighting", "textiles", "headboard"], "key_elements": ["bed", "bedside tables", "wardrobe", "soft lighting", "textiles", "headboard"],
"must_haves": "bed linen, pillows, bedside lighting", "must_haves": "bed linen, pillows, bedside lighting",
"avoid": "office furniture, dining elements, cooking equipment" "avoid": "office furniture, dining elements, cooking equipment"
}, },
"living_room": { "living_room": {
"description": "a social gathering and relaxation space", "description": "a social gathering and relaxation space",
"key_elements": ["sofa", "coffee table", "TV unit", "accent chairs", "rugs"], "key_elements": ["sofa", "coffee table", "TV unit", "accent chairs", "rugs"],
"must_haves": "seating arrangement, focal point", "must_haves": "seating arrangement, focal point",
"avoid": "beds, cooking equipment, clinical elements" "avoid": "beds, cooking equipment, clinical elements"
}, },
"bathroom": { "bathroom": {
"description": "a private hygiene and wellness space", "description": "a private hygiene and wellness space",
"key_elements": ["vanity", "bathtub", "shower", "tiles", "mirrors"], "key_elements": ["vanity", "bathtub", "shower", "tiles", "mirrors"],
"must_haves": "wet-area materials, luxury fixtures", "must_haves": "wet-area materials, luxury fixtures",
"avoid": "soft furnishings, carpet, beds" "avoid": "soft furnishings, carpet, beds"
}, },
"kitchen": { "kitchen": {
"description": "a functional cooking space", "description": "a functional cooking space",
"key_elements": ["cabinetry", "countertops", "backsplash", "appliances", "island"], "key_elements": ["cabinetry", "countertops", "backsplash", "appliances", "island"],
"must_haves": "work surfaces, storage", "must_haves": "work surfaces, storage",
"avoid": "beds, lounge furniture" "avoid": "beds, lounge furniture"
}, },
"dining_room": { "dining_room": {
"description": "an eating and entertaining space", "description": "an eating and entertaining space",
"key_elements": ["dining table", "chairs", "sideboard", "pendant lighting"], "key_elements": ["dining table", "chairs", "sideboard", "pendant lighting"],
"must_haves": "central dining table, seating", "must_haves": "central dining table, seating",
"avoid": "beds, cooking appliances" "avoid": "beds, cooking appliances"
}, },
"home_office": { "home_office": {
"description": "a workspace within a home", "description": "a workspace within a home",
"key_elements": ["desk", "ergonomic chair", "shelving", "task lighting"], "key_elements": ["desk", "ergonomic chair", "shelving", "task lighting"],
"must_haves": "functional desk setup", "must_haves": "functional desk setup",
"avoid": "beds in foreground, dining furniture" "avoid": "beds in foreground, dining furniture"
}, },
"hallway": { "hallway": {
"description": "an entrance or transitional corridor", "description": "an entrance or transitional corridor",
"key_elements": ["console table", "mirror", "coat storage", "lighting"], "key_elements": ["console table", "mirror", "coat storage", "lighting"],
"must_haves": "welcoming entrance elements", "must_haves": "welcoming entrance elements",
"avoid": "beds, large seating" "avoid": "beds, large seating"
}, },
"balcony": { "balcony": {
"description": "an outdoor living extension", "description": "an outdoor living extension",
"key_elements": ["outdoor furniture", "planters", "lighting", "railings"], "key_elements": ["outdoor furniture", "planters", "lighting", "railings"],
"must_haves": "weather-resistant materials", "must_haves": "weather-resistant materials",
"avoid": "indoor bedding, non-weather-resistant elements" "avoid": "indoor bedding, non-weather-resistant elements"
}, },
} }
FEW_SHOT_EXAMPLES = """ FEW_SHOT_EXAMPLES = """
EXAMPLE 1: EXAMPLE 1:
Keywords: ["light oak", "white walls", "hygge", "natural light", "minimalist"] Keywords: ["light oak", "white walls", "hygge", "natural light", "minimalist"]
Room type: bedroom Room type: bedroom
Positive prompt: scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles, abundant natural light streaming through large windows, clean white walls, simple functional furniture, cozy hygge atmosphere, soft cream and warm gray tones, organic cotton fabrics, potted green plants, minimalist pendant lighting, decluttered space, architectural photography, 8k resolution, photorealistic, global illumination, soft shadows, natural materials, sustainable design Positive prompt: scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles, abundant natural light streaming through large windows, clean white walls, simple functional furniture, cozy hygge atmosphere, soft cream and warm gray tones, organic cotton fabrics, potted green plants, minimalist pendant lighting, decluttered space, architectural photography, 8k resolution, photorealistic, global illumination, soft shadows, natural materials, sustainable design
Negative prompt: (worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, heavy ornamentation, dark colors, cluttered space Negative prompt: (worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, heavy ornamentation, dark colors, cluttered space
EXAMPLE 2: EXAMPLE 2:
Keywords: ["gold brass", "marble", "velvet", "emerald green", "1920s", "geometric"] Keywords: ["gold brass", "marble", "velvet", "emerald green", "1920s", "geometric"]
Room type: living_room Room type: living_room
Positive prompt: art deco luxury interior design, geometric chevron patterns, gold brass accents, rich velvet upholstery in emerald green and sapphire blue, sunburst mirrors, polished marble flooring with brass inlay, crystal chandeliers, lacquered wood furniture, bold symmetrical arrangements, 1920s glamour, warm ambient lighting, architectural photography, 8k resolution, photorealistic, global illumination, elegant reflections, geometric motifs, stepped forms Positive prompt: art deco luxury interior design, geometric chevron patterns, gold brass accents, rich velvet upholstery in emerald green and sapphire blue, sunburst mirrors, polished marble flooring with brass inlay, crystal chandeliers, lacquered wood furniture, bold symmetrical arrangements, 1920s glamour, warm ambient lighting, architectural photography, 8k resolution, photorealistic, global illumination, elegant reflections, geometric motifs, stepped forms
Negative prompt: (worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, rustic elements, farmhouse style, minimalism, cheap materials Negative prompt: (worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, rustic elements, farmhouse style, minimalism, cheap materials
""" """
SYSTEM_PROMPT = """You are Dream Weaver's interior design prompt engineer. Convert user-provided keywords and a room type into a high-quality prompt for image generation. SYSTEM_PROMPT = """You are Dream Weaver's interior design prompt engineer. Convert user-provided keywords and a room type into a high-quality prompt for image generation.
TASK: TASK:
Generate JSON containing: Generate JSON containing:
1. "positive_prompt" (rich, photorealistic, 80-120 words) 1. "positive_prompt" (rich, photorealistic, 80-120 words)
2. "negative_prompt" (preventing artifacts, 30-50 words) 2. "negative_prompt" (preventing artifacts, 30-50 words)
3. "cfg" (float 6.0-9.0) 3. "cfg" (float 6.0-9.0)
4. "denoise" (float 0.5-0.85) 4. "denoise" (float 0.5-0.85)
5. "steps" (int 25-40) 5. "steps" (int 25-40)
RULES FOR POSITIVE PROMPT: RULES FOR POSITIVE PROMPT:
- Focus on the core aesthetic derived from keywords - Focus on the core aesthetic derived from keywords
- Include architecture, furniture, and lighting suitable for the room type - Include architecture, furniture, and lighting suitable for the room type
- End with: "architectural photography, 8k resolution, photorealistic" - End with: "architectural photography, 8k resolution, photorealistic"
RULES FOR NEGATIVE PROMPT: RULES FOR NEGATIVE PROMPT:
- Start with: (worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes - Start with: (worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes
OUTPUT FORMAT: OUTPUT FORMAT:
Provide valid JSON only, with keys: "style_name", "positive_prompt", "negative_prompt", "cfg", "denoise", "steps", "reasoning". Provide valid JSON only, with keys: "style_name", "positive_prompt", "negative_prompt", "cfg", "denoise", "steps", "reasoning".
FEW-SHOT EXAMPLES: FEW-SHOT EXAMPLES:
""" + FEW_SHOT_EXAMPLES """ + FEW_SHOT_EXAMPLES
class ExpandedPrompt: class ExpandedPrompt:
def __init__(self, style_name, positive_prompt, negative_prompt, cfg, denoise, steps, reasoning, source): def __init__(self, style_name, positive_prompt, negative_prompt, cfg, denoise, steps, reasoning, source):
self.style_name = style_name self.style_name = style_name
self.positive_prompt = positive_prompt self.positive_prompt = positive_prompt
self.negative_prompt = negative_prompt self.negative_prompt = negative_prompt
self.cfg = cfg self.cfg = cfg
self.denoise = denoise self.denoise = denoise
self.steps = steps self.steps = steps
self.reasoning = reasoning self.reasoning = reasoning
self.source = source self.source = source
def _call_ollama(user_message: str) -> str: def _call_ollama(user_message: str) -> str:
ollama_url = os.environ.get("OLLAMA_URL", "http://localhost:11434") ollama_url = os.environ.get("OLLAMA_URL", "http://localhost:11434")
# Using Qwen 3.5 27B as requested # Using Qwen 3.5 27B as requested
model = os.environ.get("OLLAMA_MODEL", "qwen3.5:27b") model = os.environ.get("OLLAMA_MODEL", "qwen3.5:27b")
full_prompt = f"{SYSTEM_PROMPT}\n\nUSER REQUEST:\n{user_message}\n\nReturn JSON ONLY. No markdown wrapping." full_prompt = f"{SYSTEM_PROMPT}\n\nUSER REQUEST:\n{user_message}\n\nReturn JSON ONLY. No markdown wrapping."
r = requests.post( r = requests.post(
f"{ollama_url}/api/generate", f"{ollama_url}/api/generate",
json={ json={
"model": model, "model": model,
"prompt": full_prompt, "prompt": full_prompt,
"stream": False, "stream": False,
"format": "json", "format": "json",
"options": {"temperature": 0.5} "options": {"temperature": 0.5}
}, },
timeout=180 # Large models take time timeout=180 # Large models take time
) )
r.raise_for_status() r.raise_for_status()
return r.json()["response"] return r.json()["response"]
def expand_prompt(keywords: list[str], room_type: str = "living_room", additional_notes: str = "") -> ExpandedPrompt: def expand_prompt(keywords: list[str], room_type: str = "living_room", additional_notes: str = "") -> ExpandedPrompt:
if not keywords: if not keywords:
raise ValueError("Keywords required") raise ValueError("Keywords required")
room_type = room_type.lower().replace(" ", "_") room_type = room_type.lower().replace(" ", "_")
if room_type not in ROOM_CONTEXTS: if room_type not in ROOM_CONTEXTS:
room_type = "living_room" room_type = "living_room"
ctx = ROOM_CONTEXTS[room_type] ctx = ROOM_CONTEXTS[room_type]
user_message = f"""KEYWORDS: {', '.join(keywords)} user_message = f"""KEYWORDS: {', '.join(keywords)}
ROOM TYPE: {room_type} ({ctx['description']}) ROOM TYPE: {room_type} ({ctx['description']})
MUST HAVE: {ctx['must_haves']} MUST HAVE: {ctx['must_haves']}
AVOID: {ctx['avoid']} AVOID: {ctx['avoid']}
{f'NOTES: {additional_notes}' if additional_notes else ''}""" {f'NOTES: {additional_notes}' if additional_notes else ''}"""
try: try:
logger.info("Calling local Ollama LLM...") logger.info("Calling local Ollama LLM...")
raw = _call_ollama(user_message).strip() raw = _call_ollama(user_message).strip()
json_match = re.search(r'\{[\s\S]*\}', raw) json_match = re.search(r'\{[\s\S]*\}', raw)
if json_match: if json_match:
raw_json = json_match.group(0) raw_json = json_match.group(0)
else: else:
raw_json = raw raw_json = raw
data = json.loads(raw_json) data = json.loads(raw_json)
return ExpandedPrompt( return ExpandedPrompt(
style_name=data.get("style_name", "custom_local"), style_name=data.get("style_name", "custom_local"),
positive_prompt=data["positive_prompt"], positive_prompt=data["positive_prompt"],
negative_prompt=data["negative_prompt"], negative_prompt=data["negative_prompt"],
cfg=float(data.get("cfg", 7.5)), cfg=float(data.get("cfg", 7.5)),
denoise=float(data.get("denoise", 0.72)), denoise=float(data.get("denoise", 0.72)),
steps=int(data.get("steps", 30)), steps=int(data.get("steps", 30)),
reasoning=data.get("reasoning", ""), reasoning=data.get("reasoning", ""),
source="ollama_local" source="ollama_local"
) )
except Exception as e: except Exception as e:
logger.warning(f"Ollama failed, using sync fallback: {e}") logger.warning(f"Ollama failed, using sync fallback: {e}")
return expand_prompt_simple(keywords, room_type) return expand_prompt_simple(keywords, room_type)
def expand_prompt_simple(keywords: list[str], room_type: str = "living_room") -> ExpandedPrompt: def expand_prompt_simple(keywords: list[str], room_type: str = "living_room") -> ExpandedPrompt:
ctx = ROOM_CONTEXTS.get(room_type.replace(" ", "_"), ROOM_CONTEXTS["living_room"]) ctx = ROOM_CONTEXTS.get(room_type.replace(" ", "_"), ROOM_CONTEXTS["living_room"])
kw_str = ", ".join(keywords) kw_str = ", ".join(keywords)
positive = f"{kw_str} interior design, {', '.join(ctx['key_elements'][:4])}, photorealistic {room_type.replace('_', ' ')} interior, architectural photography, 8k resolution, photorealistic" positive = f"{kw_str} interior design, {', '.join(ctx['key_elements'][:4])}, photorealistic {room_type.replace('_', ' ')} interior, architectural photography, 8k resolution, photorealistic"
negative = "(worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, extra windows, unrealistic lighting, structural changes" negative = "(worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch), blurry, distorted, extra windows, unrealistic lighting, structural changes"
return ExpandedPrompt( return ExpandedPrompt(
style_name="fallback", positive_prompt=positive, negative_prompt=negative, style_name="fallback", positive_prompt=positive, negative_prompt=negative,
cfg=7.5, denoise=0.72, steps=30, reasoning="No LLM", source="fallback" cfg=7.5, denoise=0.72, steps=30, reasoning="No LLM", source="fallback"
) )
if __name__ == "__main__": if __name__ == "__main__":
import sys import sys
logging.basicConfig(level=logging.INFO) logging.basicConfig(level=logging.INFO)
ans = expand_prompt(["blue marble", "gold"], "bathroom") ans = expand_prompt(["blue marble", "gold"], "bathroom")
print(ans.positive_prompt) print(ans.positive_prompt)

2
iOS/velocity/velocity/Core/Config/AppConfig.swift
View File

@@ -11,6 +11,6 @@ enum AppConfig {
!override.isEmpty, override != "$(BASE_URL)" { !override.isEmpty, override != "$(BASE_URL)" {
return override return override
} }
return "http://54.172.172.2:8080" return "http://54.91.19.60:8082"
}() }()
} }

66
iOS/velocity/velocity/Core/Math/SunMath.swift
View File

@@ -2,11 +2,14 @@ import CoreLocation
import Foundation import Foundation
struct SunPosition { struct SunPosition {
let azimuth: Double // 0...360, degrees clockwise from true north let azimuth: Double // 0...360, degrees clockwise from true north
let elevation: Double // -90...90 degrees above horizon let elevation: Double // -90...90 degrees above horizon
} }
enum SunMath { enum SunMath {
// MARK: - Single Position
static func calculateSunPosition(date: Date, coordinate: CLLocationCoordinate2D) -> SunPosition { static func calculateSunPosition(date: Date, coordinate: CLLocationCoordinate2D) -> SunPosition {
let timezone = TimeZone.current let timezone = TimeZone.current
let localOffsetHours = Double(timezone.secondsFromGMT(for: date)) / 3600.0 let localOffsetHours = Double(timezone.secondsFromGMT(for: date)) / 3600.0
@@ -14,7 +17,7 @@ enum SunMath {
let n = julianDay - 2_451_545.0 let n = julianDay - 2_451_545.0
let meanLongitude = normalizeDegrees(280.46 + 0.985_647_4 * n) let meanLongitude = normalizeDegrees(280.46 + 0.985_647_4 * n)
let meanAnomaly = normalizeDegrees(357.528 + 0.985_600_3 * n) let meanAnomaly = normalizeDegrees(357.528 + 0.985_600_3 * n)
let lambda = meanLongitude let lambda = meanLongitude
+ 1.915 * sin(meanAnomaly.radians) + 1.915 * sin(meanAnomaly.radians)
@@ -32,9 +35,9 @@ enum SunMath {
let hourAngle = normalizeDegrees(lst - rightAscension) let hourAngle = normalizeDegrees(lst - rightAscension)
let signedHourAngle = hourAngle > 180.0 ? hourAngle - 360.0 : hourAngle let signedHourAngle = hourAngle > 180.0 ? hourAngle - 360.0 : hourAngle
let latitude = coordinate.latitude.radians let latitude = coordinate.latitude.radians
let declinationRad = declination.radians let declinationRad = declination.radians
let hourAngleRad = signedHourAngle.radians let hourAngleRad = signedHourAngle.radians
let elevation = asin( let elevation = asin(
sin(latitude) * sin(declinationRad) sin(latitude) * sin(declinationRad)
@@ -51,11 +54,13 @@ enum SunMath {
return SunPosition(azimuth: azimuth, elevation: elevation) return SunPosition(azimuth: azimuth, elevation: elevation)
} }
// MARK: - Hourly Arc (used by legacy code & DashedSunLine)
/// 5-sample dictionary kept for backward compat with the Dollhouse slider.
static func sunPathSamples(for date: Date, coordinate: CLLocationCoordinate2D) -> [Date: SunPosition] { static func sunPathSamples(for date: Date, coordinate: CLLocationCoordinate2D) -> [Date: SunPosition] {
let calendar = Calendar.current let calendar = Calendar.current
let sampleHours = [8, 10, 12, 14, 16] let sampleHours = [8, 10, 12, 14, 16]
var output: [Date: SunPosition] = [:] var output: [Date: SunPosition] = [:]
for hour in sampleHours { for hour in sampleHours {
if let sampleDate = calendar.date(bySettingHour: hour, minute: 0, second: 0, of: date) { if let sampleDate = calendar.date(bySettingHour: hour, minute: 0, second: 0, of: date) {
output[sampleDate] = calculateSunPosition(date: sampleDate, coordinate: coordinate) output[sampleDate] = calculateSunPosition(date: sampleDate, coordinate: coordinate)
@@ -64,25 +69,62 @@ enum SunMath {
return output return output
} }
private static func normalizeDegrees(_ value: Double) -> Double { /// Dense arc for the AR overlay — one sample per hour from 4 AM to 8 PM.
/// Filters out below-horizon positions (elevation < -5°).
static func sunPathArc(for date: Date, coordinate: CLLocationCoordinate2D) -> [(date: Date, position: SunPosition)] {
let calendar = Calendar.current
var result: [(Date, SunPosition)] = []
for hour in 4...20 {
guard let sampleDate = calendar.date(bySettingHour: hour, minute: 0, second: 0, of: date) else { continue }
let pos = calculateSunPosition(date: sampleDate, coordinate: coordinate)
// include a small below-horizon buffer so arc starts/ends smoothly
if pos.elevation > -5 {
result.append((sampleDate, pos))
}
}
return result
}
/// Approximate sunrise and sunset by scanning for elevation sign changes.
static func sunRiseSet(for date: Date, coordinate: CLLocationCoordinate2D) -> (rise: Date?, set: Date?) {
let calendar = Calendar.current
var rise: Date? = nil
var set: Date? = nil
var prevElevation: Double? = nil
var prevDate: Date? = nil
for minuteOffset in stride(from: 0, through: 24 * 60, by: 10) {
guard let sampleDate = calendar.date(byAdding: .minute, value: minuteOffset, to: calendar.startOfDay(for: date)) else { continue }
let pos = calculateSunPosition(date: sampleDate, coordinate: coordinate)
if let prev = prevElevation, let prevD = prevDate {
if prev < 0 && pos.elevation >= 0 { rise = prevD }
if prev >= 0 && pos.elevation < 0 { set = prevD }
}
prevElevation = pos.elevation
prevDate = sampleDate
}
return (rise, set)
}
// MARK: - Helpers
static func normalizeDegrees(_ value: Double) -> Double {
let reduced = value.truncatingRemainder(dividingBy: 360.0) let reduced = value.truncatingRemainder(dividingBy: 360.0)
return reduced >= 0 ? reduced : reduced + 360.0 return reduced >= 0 ? reduced : reduced + 360.0
} }
} }
// MARK: - Date helpers
private extension Date { private extension Date {
var utcHours: Double { var utcHours: Double {
let calendar = Calendar(identifier: .gregorian) let calendar = Calendar(identifier: .gregorian)
let comps = calendar.dateComponents(in: TimeZone(secondsFromGMT: 0)!, from: self) let comps = calendar.dateComponents(in: TimeZone(secondsFromGMT: 0)!, from: self)
let hours = Double(comps.hour ?? 0) return Double(comps.hour ?? 0) + Double(comps.minute ?? 0) / 60.0 + Double(comps.second ?? 0) / 3600.0
let minutes = Double(comps.minute ?? 0)
let seconds = Double(comps.second ?? 0)
return hours + minutes / 60.0 + seconds / 3600.0
} }
var julianDay: Double { var julianDay: Double {
let interval = timeIntervalSince1970 timeIntervalSince1970 / 86_400.0 + 2_440_587.5
return (interval / 86_400.0) + 2_440_587.5
} }
} }

23
iOS/velocity/velocity/Core/Networking/ComfyClient.swift
View File

@@ -17,7 +17,9 @@ final class ComfyClient {
/// Returns `true` if `{ "status": "ok" }`. /// Returns `true` if `{ "status": "ok" }`.
func checkHealth() async -> Bool { func checkHealth() async -> Bool {
guard let url = URL(string: "\(baseURL)/health") else { return false } guard let url = URL(string: "\(baseURL)/health") else { return false }
guard let (data, _) = try? await URLSession.shared.data(from: url), var request = URLRequest(url: url)
request.timeoutInterval = 30.0
guard let (data, _) = try? await URLSession.shared.data(for: request),
let json = try? JSONDecoder().decode(HealthResponse.self, from: data) else { let json = try? JSONDecoder().decode(HealthResponse.self, from: data) else {
return false return false
} }
@@ -30,9 +32,9 @@ final class ComfyClient {
/// Full pipeline: upload → queue → poll → download. /// Full pipeline: upload → queue → poll → download.
/// - Parameters: /// - Parameters:
/// - source: Room photo from camera or library. /// - source: Room photo from camera or library.
/// - style: One of `scandinavian`, `art_deco`, `biophilic`, `cyberpunk`, `japandi`. /// - roomType: The type of room being designed (e.g. "bedroom", "living_room").
/// - keywords: Comma-separated user keywords appended to the style prompt (can be empty). /// - keywords: Comma-separated user keywords appended to the style prompt (can be empty).
func generateImage(source: UIImage, style: String, keywords: String) async throws -> UIImage { func generateImage(source: UIImage, roomType: String, keywords: String) async throws -> UIImage {
let normalised = source.fixedOrientation() let normalised = source.fixedOrientation()
let resized = normalised.resizedSquare(to: 1024) let resized = normalised.resizedSquare(to: 1024)
guard let imageData = resized.jpegData(compressionQuality: 0.85) else { guard let imageData = resized.jpegData(compressionQuality: 0.85) else {
@@ -40,7 +42,7 @@ final class ComfyClient {
} }
// 1. Submit job → get job_id // 1. Submit job → get job_id
let job = try await submitJob(imageData: imageData, style: style, keywords: keywords) let job = try await submitJob(imageData: imageData, roomType: roomType, keywords: keywords)
// 2. Poll status every 2s until ready (max 5 min per integration guide §3.3) // 2. Poll status every 2s until ready (max 5 min per integration guide §3.3)
let resultURL = try await pollUntilReady(jobId: job.jobId) let resultURL = try await pollUntilReady(jobId: job.jobId)
@@ -51,7 +53,7 @@ final class ComfyClient {
// MARK: - Step 1: POST /dream-weaver // MARK: - Step 1: POST /dream-weaver
private func submitJob(imageData: Data, style: String, keywords: String) async throws -> GenerationJob { private func submitJob(imageData: Data, roomType: String, keywords: String) async throws -> GenerationJob {
guard let url = URL(string: "\(baseURL)/dream-weaver") else { guard let url = URL(string: "\(baseURL)/dream-weaver") else {
throw DreamWeaverError.generationFailed("Invalid gateway URL") throw DreamWeaverError.generationFailed("Invalid gateway URL")
} }
@@ -60,9 +62,10 @@ final class ComfyClient {
var request = URLRequest(url: url) var request = URLRequest(url: url)
request.httpMethod = "POST" request.httpMethod = "POST"
request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type") request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")
request.timeoutInterval = 180.0
request.httpBody = buildMultipart( request.httpBody = buildMultipart(
imageData: imageData, imageData: imageData,
style: style, roomType: roomType,
keywords: keywords, keywords: keywords,
boundary: boundary boundary: boundary
) )
@@ -110,7 +113,7 @@ final class ComfyClient {
// MARK: - Multipart Builder // MARK: - Multipart Builder
private func buildMultipart(imageData: Data, style: String, keywords: String, boundary: String) -> Data { private func buildMultipart(imageData: Data, roomType: String, keywords: String, boundary: String) -> Data {
var body = Data() var body = Data()
let crlf = "\r\n" let crlf = "\r\n"
@@ -121,10 +124,10 @@ final class ComfyClient {
body += imageData body += imageData
body += crlf body += crlf
// style field — must be one of the 5 preset IDs // roomType field
body += "--\(boundary)\(crlf)" body += "--\(boundary)\(crlf)"
body += "Content-Disposition: form-data; name=\"style\"\(crlf)\(crlf)" body += "Content-Disposition: form-data; name=\"room_type\"\(crlf)\(crlf)"
body += style body += roomType
body += crlf body += crlf
// keywords field — user's optional comma-separated additions // keywords field — user's optional comma-separated additions

278
iOS/velocity/velocity/Features/Inventory/ARSunOverlayView.swift
View File

@@ -4,11 +4,14 @@ import CoreMotion
import SceneKit import SceneKit
import SwiftUI import SwiftUI
// MARK: - ARSunOverlayView
struct ARSunOverlayView: UIViewRepresentable { struct ARSunOverlayView: UIViewRepresentable {
@Binding var sunNodesReady: Bool @Binding var sunNodesReady: Bool
let vm: SunseekerViewModel
func makeCoordinator() -> Coordinator { func makeCoordinator() -> Coordinator {
Coordinator(sunNodesReady: $sunNodesReady) Coordinator(sunNodesReady: $sunNodesReady, vm: vm)
} }
func makeUIView(context: Context) -> ARSCNView { func makeUIView(context: Context) -> ARSCNView {
@@ -18,7 +21,7 @@ struct ARSunOverlayView: UIViewRepresentable {
view.automaticallyUpdatesLighting = true view.automaticallyUpdatesLighting = true
let config = ARWorldTrackingConfiguration() let config = ARWorldTrackingConfiguration()
config.worldAlignment = .gravityAndHeading config.worldAlignment = .gravityAndHeading // north = -Z axis
view.session.run(config) view.session.run(config)
context.coordinator.attach(to: view) context.coordinator.attach(to: view)
@@ -32,87 +35,242 @@ struct ARSunOverlayView: UIViewRepresentable {
coordinator.stop() coordinator.stop()
} }
// MARK: - Coordinator
final class Coordinator: NSObject, ARSCNViewDelegate, CLLocationManagerDelegate { final class Coordinator: NSObject, ARSCNViewDelegate, CLLocationManagerDelegate {
private let locationManager = CLLocationManager()
private let motionManager = CMMotionManager()
private weak var sceneView: ARSCNView? private weak var sceneView: ARSCNView?
private var heading: CLLocationDirection = 0 private let vm: SunseekerViewModel
private var coordinate: CLLocationCoordinate2D?
@Binding private var sunNodesReady: Bool @Binding private var sunNodesReady: Bool
init(sunNodesReady: Binding<Bool>) { // Scene node containers (replaced on each rebuild)
private var arcRootNode = SCNNode()
private var currentSunNode = SCNNode()
private var isSceneBuilt = false
// Fallback timer for CoreMotion-only mode
private var fallbackTimer: Timer?
private var limitedTrackingStart: Date?
init(sunNodesReady: Binding<Bool>, vm: SunseekerViewModel) {
_sunNodesReady = sunNodesReady _sunNodesReady = sunNodesReady
super.init() self.vm = vm
locationManager.delegate = self
locationManager.desiredAccuracy = kCLLocationAccuracyBest
locationManager.headingFilter = 1
locationManager.requestWhenInUseAuthorization()
locationManager.startUpdatingLocation()
locationManager.startUpdatingHeading()
startMotion()
} }
func attach(to sceneView: ARSCNView) { func attach(to sceneView: ARSCNView) {
self.sceneView = sceneView self.sceneView = sceneView
addSunPathNodesIfPossible() sceneView.scene.rootNode.addChildNode(arcRootNode)
sceneView.scene.rootNode.addChildNode(currentSunNode)
} }
func stop() { func stop() {
motionManager.stopDeviceMotionUpdates() vm.stop()
locationManager.stopUpdatingHeading() fallbackTimer?.invalidate()
locationManager.stopUpdatingLocation()
} }
func locationManager(_ manager: CLLocationManager, didUpdateLocations locations: [CLLocation]) { // MARK: - ARSCNViewDelegate — per-frame update
guard coordinate == nil, let location = locations.last else { return }
coordinate = location.coordinate
addSunPathNodesIfPossible()
}
func locationManager(_ manager: CLLocationManager, didUpdateHeading newHeading: CLHeading) { func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
heading = newHeading.trueHeading > 0 ? newHeading.trueHeading : newHeading.magneticHeading guard vm.isReady else { return }
addSunPathNodesIfPossible()
}
private func startMotion() { // Build arc once
guard motionManager.isDeviceMotionAvailable else { return } if !isSceneBuilt {
motionManager.deviceMotionUpdateInterval = 0.1 DispatchQueue.main.async { self.buildScene() }
motionManager.startDeviceMotionUpdates()
}
private func addSunPathNodesIfPossible() {
guard
let sceneView,
let coordinate,
!sunNodesReady
else { return }
let samples = SunMath.sunPathSamples(for: Date(), coordinate: coordinate)
let sorted = samples.sorted { $0.key < $1.key }
let root = SCNNode()
let northOffset = (heading).radians
let radius: Float = 1.8
for (_, pos) in sorted {
let elevation = Float(pos.elevation.radians)
let azimuth = Float((pos.azimuth).radians) - Float(northOffset)
let x = radius * cos(elevation) * sin(azimuth)
let y = radius * sin(elevation)
let z = -radius * cos(elevation) * cos(azimuth)
let sphere = SCNSphere(radius: 0.03)
sphere.firstMaterial?.diffuse.contents = UIColor.systemYellow
let node = SCNNode(geometry: sphere)
node.position = SCNVector3(x, y, z)
root.addChildNode(node)
} }
sceneView.scene.rootNode.addChildNode(root) // Update current sun orb every frame
if let cur = vm.currentPosition {
let pos = vm.worldPosition(for: cur, radius: 1.8)
currentSunNode.position = pos
}
}
func session(_ session: ARSession, cameraDidChangeTrackingState camera: ARCamera) {
switch camera.trackingState {
case .limited(let reason):
print("[Sunseeker] Tracking limited: \(reason)")
if limitedTrackingStart == nil {
limitedTrackingStart = Date()
// After 5s of limited tracking, switch to CoreMotion attitude fallback
fallbackTimer = Timer.scheduledTimer(withTimeInterval: 5, repeats: false) { [weak self] _ in
self?.activateCoreMotionFallback()
}
}
case .normal:
limitedTrackingStart = nil
fallbackTimer?.invalidate()
fallbackTimer = nil
case .notAvailable:
break
@unknown default:
break
}
}
// MARK: - Scene Building
private func buildScene() {
guard let sceneView else { return }
// Remove old nodes
arcRootNode.childNodes.forEach { $0.removeFromParentNode() }
currentSunNode.childNodes.forEach { $0.removeFromParentNode() }
let arc = vm.arc
let radius: Float = 1.8
var positions: [SCNVector3] = []
// Hourly marker spheres + time labels
for (date, pos) in arc {
guard pos.elevation > -5 else { continue }
let worldPos = vm.worldPosition(for: pos, radius: radius)
positions.append(worldPos)
let sphere = SCNSphere(radius: 0.018)
sphere.firstMaterial?.diffuse.contents = UIColor.systemYellow.withAlphaComponent(0.85)
sphere.firstMaterial?.lightingModel = .constant
let markerNode = SCNNode(geometry: sphere)
markerNode.position = worldPos
arcRootNode.addChildNode(markerNode)
// Time label (only on even hours to avoid clutter)
let calendar = Calendar.current
let hour = calendar.component(.hour, from: date)
if hour % 2 == 0 {
let labelNode = makeTextNode(text: hourLabel(from: date), color: .white, fontSize: 0.04)
labelNode.position = SCNVector3(worldPos.x, worldPos.y + 0.06, worldPos.z)
arcRootNode.addChildNode(labelNode)
}
}
// Continuous arc line
if positions.count >= 2 {
let lineNode = makeLineNode(through: positions, color: UIColor.systemYellow.withAlphaComponent(0.55))
arcRootNode.addChildNode(lineNode)
}
// Sunrise marker
if let riseDate = vm.riseSet.rise {
let risePos = SunMath.calculateSunPosition(date: riseDate, coordinate: vm.coordinate!)
let wPos = vm.worldPosition(for: risePos, radius: radius)
arcRootNode.addChildNode(makeSpecialMarker(at: wPos, color: .systemOrange, label: "Sunrise \(hourLabel(from: riseDate))"))
}
// Sunset marker
if let setDate = vm.riseSet.set {
let setPos = SunMath.calculateSunPosition(date: setDate, coordinate: vm.coordinate!)
let wPos = vm.worldPosition(for: setPos, radius: radius)
arcRootNode.addChildNode(makeSpecialMarker(at: wPos, color: .systemRed, label: "Sunset \(hourLabel(from: setDate))"))
}
// Current sun orb (large, animated glow)
if let cur = vm.currentPosition {
let orb = SCNSphere(radius: 0.055)
orb.firstMaterial?.diffuse.contents = UIColor.systemOrange
orb.firstMaterial?.emission.contents = UIColor.systemYellow
orb.firstMaterial?.lightingModel = .constant
let orbNode = SCNNode(geometry: orb)
orbNode.position = vm.worldPosition(for: cur, radius: radius)
// Pulse animation
let pulse = CABasicAnimation(keyPath: "scale")
pulse.fromValue = SCNVector3(1, 1, 1)
pulse.toValue = SCNVector3(1.3, 1.3, 1.3)
pulse.duration = 1.2
pulse.autoreverses = true
pulse.repeatCount = .infinity
orbNode.addAnimation(pulse, forKey: "pulse")
let label = makeTextNode(text: "Now", color: .systemYellow, fontSize: 0.05)
label.position = SCNVector3(0, 0.09, 0)
orbNode.addChildNode(label)
currentSunNode.addChildNode(orbNode)
}
isSceneBuilt = true
sunNodesReady = true sunNodesReady = true
} }
// MARK: - Node Factories
private func makeSpecialMarker(at pos: SCNVector3, color: UIColor, label: String) -> SCNNode {
let root = SCNNode()
let sphere = SCNSphere(radius: 0.035)
sphere.firstMaterial?.diffuse.contents = color
sphere.firstMaterial?.emission.contents = color.withAlphaComponent(0.5)
sphere.firstMaterial?.lightingModel = .constant
let markerNode = SCNNode(geometry: sphere)
markerNode.position = pos
root.addChildNode(markerNode)
let labelNode = makeTextNode(text: label, color: .white, fontSize: 0.04)
labelNode.position = SCNVector3(pos.x, pos.y + 0.07, pos.z)
root.addChildNode(labelNode)
return root
}
/// Creates a billboard SCNText node that always faces the camera.
private func makeTextNode(text: String, color: UIColor, fontSize: CGFloat) -> SCNNode {
let scnText = SCNText(string: text, extrusionDepth: 0)
scnText.font = UIFont.systemFont(ofSize: fontSize * 100, weight: .medium)
scnText.firstMaterial?.diffuse.contents = color
scnText.firstMaterial?.lightingModel = .constant
scnText.isWrapped = false
let textNode = SCNNode(geometry: scnText)
textNode.scale = SCNVector3(fontSize / 100, fontSize / 100, fontSize / 100)
// Billboard constraint — always face camera
let constraint = SCNBillboardConstraint()
constraint.freeAxes = .Y
textNode.constraints = [constraint]
// Centre text
let (min, max) = textNode.boundingBox
textNode.pivot = SCNMatrix4MakeTranslation((max.x - min.x) / 2, 0, 0)
return textNode
}
/// Builds a line strip SCNNode connecting all positions.
private func makeLineNode(through positions: [SCNVector3], color: UIColor) -> SCNNode {
guard positions.count >= 2 else { return SCNNode() }
var vertices: [SCNVector3] = positions
var indices: [Int32] = []
for i in 0..<(vertices.count - 1) {
indices.append(Int32(i))
indices.append(Int32(i + 1))
}
let vertexSource = SCNGeometrySource(vertices: vertices)
let element = SCNGeometryElement(
indices: indices,
primitiveType: .line
)
let geometry = SCNGeometry(sources: [vertexSource], elements: [element])
geometry.firstMaterial?.diffuse.contents = color
geometry.firstMaterial?.lightingModel = .constant
return SCNNode(geometry: geometry)
}
private func hourLabel(from date: Date) -> String {
let fmt = DateFormatter()
fmt.dateFormat = "ha"
fmt.amSymbol = "am"
fmt.pmSymbol = "pm"
return fmt.string(from: date)
}
// MARK: - CoreMotion Fallback
private func activateCoreMotionFallback() {
// In fallback mode we rely on CMMotionManager attitude (already running in SunseekerViewModel)
// and just keep the scene nodes updated via the 1s tick in the VM.
print("[Sunseeker] Switched to CoreMotion fallback — ARKit tracking unavailable.")
}
} }
} }
// MARK: - Degree helpers
private extension Double { private extension Double {
var radians: Double { self * .pi / 180.0 } var radians: Double { self * .pi / 180.0 }
} }

124
iOS/velocity/velocity/Features/Inventory/InventoryView.swift
View File

@@ -75,11 +75,8 @@ struct InventoryView: View {
switch store.mode { switch store.mode {
case .sunseeker: case .sunseeker:
#if targetEnvironment(simulator) #if targetEnvironment(simulator)
SimulatorUnavailableCard( SimulatorSunOverlayView(sunNodesReady: $store.sunNodesReady)
icon: "camera.metering.unknown", .clipShape(RoundedRectangle(cornerRadius: 20))
title: "AR Not Available in Simulator",
message: "Sunseeker requires a real device with a camera and compass. Run on iPhone or iPad to use this feature."
)
#else #else
SunseekerPanel(sunNodesReady: $store.sunNodesReady) SunseekerPanel(sunNodesReady: $store.sunNodesReady)
#endif #endif
@@ -178,28 +175,70 @@ private struct SimulatorUnavailableCard: View {
private struct SunseekerPanel: View { private struct SunseekerPanel: View {
@Binding var sunNodesReady: Bool @Binding var sunNodesReady: Bool
@State private var vm = SunseekerViewModel()
var body: some View { var body: some View {
ZStack(alignment: .topLeading) { ZStack(alignment: .topLeading) {
ARSunOverlayView(sunNodesReady: $sunNodesReady) ARSunOverlayView(sunNodesReady: $sunNodesReady, vm: vm)
.clipShape(RoundedRectangle(cornerRadius: 20)) .clipShape(RoundedRectangle(cornerRadius: 20))
// Retained as a stylistic design element framing the AR view
DashedSunLine() DashedSunLine()
.stroke(Color.yellow.opacity(0.9), style: StrokeStyle(lineWidth: 3, dash: [10, 8])) .stroke(Color.yellow.opacity(0.9), style: StrokeStyle(lineWidth: 3, dash: [10, 8]))
.padding(.horizontal, 24) .padding(.horizontal, 24)
.padding(.vertical, 80) .padding(.vertical, 80)
VStack(alignment: .leading, spacing: 8) { VStack(alignment: .leading, spacing: 12) {
Text("Sunseeker") // Info block
.font(.headline) VStack(alignment: .leading, spacing: 8) {
Text("Point the iPad toward windows to inspect yearly sun-entry path.") Text("Sunseeker")
.font(.subheadline) .font(.headline)
.foregroundStyle(.secondary) Text("Point the iPad toward windows to inspect yearly sun-entry path.")
} .font(.subheadline)
.padding(14) .foregroundStyle(.secondary)
.background { }
GlassBlurView(style: .systemThinMaterial) .padding(14)
.clipShape(RoundedRectangle(cornerRadius: 14)) .background {
GlassBlurView(style: .systemThinMaterial)
.clipShape(RoundedRectangle(cornerRadius: 14))
}
if !vm.isReady && vm.locationError == nil {
// Loading state
HStack(spacing: 8) {
ProgressView().tint(.white)
Text("Looking for the Sun...")
.font(.footnote)
.foregroundStyle(.white)
}
.padding(.horizontal, 14)
.padding(.vertical, 10)
.background(Color.black.opacity(0.6).clipShape(Capsule()))
}
// Error banner (e.g. Location Denied)
if let error = vm.locationError {
HStack(spacing: 12) {
Image(systemName: "exclamationmark.triangle.fill")
.foregroundStyle(.yellow)
Text(error)
.font(.subheadline)
.foregroundStyle(.white)
Spacer()
Button("Settings") {
if let url = URL(string: UIApplication.openSettingsURLString) {
UIApplication.shared.open(url)
}
}
.buttonStyle(.borderedProminent)
.tint(.white.opacity(0.2))
}
.padding(14)
.background {
RoundedRectangle(cornerRadius: 14)
.fill(Color.red.opacity(0.8))
}
}
} }
.padding(20) .padding(20)
} }
@@ -208,19 +247,22 @@ private struct SunseekerPanel: View {
// MARK: - Dream Weaver // MARK: - Dream Weaver
/// Available interior styles from integration guide §2.3 /// Available room types from integration guide §2
private struct InteriorStyle: Identifiable { private struct RoomType: Identifiable {
let id: String // sent as the `style` form field let id: String // sent as the `room_type` form field
let displayName: String let displayName: String
let icon: String // SF Symbol let icon: String // SF Symbol
} }
private let dreamWeaverStyles: [InteriorStyle] = [ private let roomTypes: [RoomType] = [
InteriorStyle(id: "scandinavian", displayName: "Scandi", icon: "snowflake"), RoomType(id: "bedroom", displayName: "Bedroom", icon: "bed.double"),
InteriorStyle(id: "art_deco", displayName: "Art Deco", icon: "sparkles"), RoomType(id: "living_room", displayName: "Living Rm", icon: "sofa"),
InteriorStyle(id: "biophilic", displayName: "Biophilic",icon: "leaf"), RoomType(id: "bathroom", displayName: "Bathroom", icon: "drop"),
InteriorStyle(id: "cyberpunk", displayName: "Cyberpunk",icon: "bolt"), RoomType(id: "kitchen", displayName: "Kitchen", icon: "refrigerator"),
InteriorStyle(id: "japandi", displayName: "Japandi", icon: "mountain.2"), RoomType(id: "dining_room", displayName: "Dining Rm", icon: "fork.knife"),
RoomType(id: "home_office", displayName: "Office", icon: "desktopcomputer"),
RoomType(id: "hallway", displayName: "Hallway", icon: "door.left.hand.open"),
RoomType(id: "balcony", displayName: "Balcony", icon: "sun.max"),
] ]
private struct DreamWeaverPanel: View { private struct DreamWeaverPanel: View {
@@ -230,8 +272,8 @@ private struct DreamWeaverPanel: View {
@Binding var errorMessage: String? @Binding var errorMessage: String?
@Binding var showCamera: Bool @Binding var showCamera: Bool
/// Selected style ID — sent as `style` field (§3.2). nil = none chosen yet. /// Selected room type ID — sent as `room_type` field. nil = none chosen yet.
@State private var selectedStyle: String? = nil @State private var selectedRoomType: String? = nil
/// Optional extra keywords — sent as `keywords` field (§3.2) /// Optional extra keywords — sent as `keywords` field (§3.2)
@State private var keywords: String = "" @State private var keywords: String = ""
/// Server health: nil = checking, true = online, false = offline /// Server health: nil = checking, true = online, false = offline
@@ -319,34 +361,34 @@ private struct DreamWeaverPanel: View {
.transition(.move(edge: .top).combined(with: .opacity)) .transition(.move(edge: .top).combined(with: .opacity))
} }
// ── Style picker (§2.3) ─────────────────────────────────────── // ── Room Type picker ───────────────────────────────────────
ScrollView(.horizontal, showsIndicators: false) { ScrollView(.horizontal, showsIndicators: false) {
HStack(spacing: 8) { HStack(spacing: 8) {
ForEach(dreamWeaverStyles) { style in ForEach(roomTypes) { room in
Button { Button {
withAnimation(.spring(response: 0.3)) { withAnimation(.spring(response: 0.3)) {
// Tap again to deselect // Tap again to deselect
selectedStyle = selectedStyle == style.id ? nil : style.id selectedRoomType = selectedRoomType == room.id ? nil : room.id
} }
} label: { } label: {
HStack(spacing: 6) { HStack(spacing: 6) {
Image(systemName: style.icon) Image(systemName: room.icon)
.font(.system(size: 11, weight: .medium)) .font(.system(size: 11, weight: .medium))
Text(style.displayName) Text(room.displayName)
.font(.system(size: 13, weight: .medium)) .font(.system(size: 13, weight: .medium))
} }
.padding(.horizontal, 12) .padding(.horizontal, 12)
.padding(.vertical, 7) .padding(.vertical, 7)
.background( .background(
Capsule() Capsule()
.fill(selectedStyle == style.id .fill(selectedRoomType == room.id
? Color(red: 0.231, green: 0.510, blue: 0.965) ? Color(red: 0.231, green: 0.510, blue: 0.965)
: Color.white.opacity(0.08)) : Color.white.opacity(0.08))
) )
.foregroundStyle(selectedStyle == style.id ? .white : .white.opacity(0.6)) .foregroundStyle(selectedRoomType == room.id ? .white : .white.opacity(0.6))
.overlay( .overlay(
Capsule() Capsule()
.stroke(selectedStyle == style.id .stroke(selectedRoomType == room.id
? Color.clear ? Color.clear
: Color.white.opacity(0.12), lineWidth: 1) : Color.white.opacity(0.12), lineWidth: 1)
) )
@@ -360,7 +402,7 @@ private struct DreamWeaverPanel: View {
// ── Keywords input ─────────────────────────────────────────── // ── Keywords input ───────────────────────────────────────────
PromptInputBar( PromptInputBar(
text: $keywords, text: $keywords,
isDisabled: sourceImage == nil || isProcessing isDisabled: sourceImage == nil || isProcessing || serverOnline == false
) { ) {
Task { await generate() } Task { await generate() }
} }
@@ -391,12 +433,16 @@ private struct DreamWeaverPanel: View {
@MainActor @MainActor
private func generate() async { private func generate() async {
guard let sourceImage, !isProcessing else { return } guard let sourceImage, !isProcessing else { return }
if serverOnline == false {
errorMessage = "Server is currently offline. Please try again later."
return
}
isProcessing = true isProcessing = true
errorMessage = nil errorMessage = nil
do { do {
let result = try await ComfyClient.shared.generateImage( let result = try await ComfyClient.shared.generateImage(
source: sourceImage, source: sourceImage,
style: selectedStyle ?? dreamWeaverStyles[0].id, // default: scandinavian roomType: selectedRoomType ?? roomTypes[0].id, // default: bedroom
keywords: keywords.trimmingCharacters(in: .whitespaces) keywords: keywords.trimmingCharacters(in: .whitespaces)
) )
withAnimation(.easeInOut(duration: 0.4)) { withAnimation(.easeInOut(duration: 0.4)) {
@@ -419,7 +465,7 @@ private struct PromptInputBar: View {
@FocusState private var isFocused: Bool @FocusState private var isFocused: Bool
@State private var shimmer = false @State private var shimmer = false
private let placeholder = "gold, marble, luxury... (optional keywords)" private let placeholder = "gold, marble, luxury, etc."
var body: some View { var body: some View {
HStack(spacing: 10) { HStack(spacing: 10) {

256
iOS/velocity/velocity/Features/Inventory/SimulatorSunOverlayView.swift Normal file
View File

@@ -0,0 +1,256 @@
import CoreLocation
import SceneKit
import SwiftUI
#if targetEnvironment(simulator)
/// A non-AR 3D view for testing the Sunseeker path logic on the iOS Simulator.
/// Uses a synthetic camera, fake location, and mock heading instead of ARKit.
struct SimulatorSunOverlayView: UIViewRepresentable {
@Binding var sunNodesReady: Bool
// Fake location (e.g. San Francisco)
private let mockLocation = CLLocationCoordinate2D(latitude: 37.7749, longitude: -122.4194)
private let mockHeading: Double = 0 // North
func makeCoordinator() -> Coordinator {
Coordinator(sunNodesReady: $sunNodesReady, mockLocation: mockLocation, mockHeading: mockHeading)
}
func makeUIView(context: Context) -> SCNView {
let view = SCNView(frame: .zero)
view.scene = SCNScene()
view.allowsCameraControl = true // Swipe around the 3D space
view.autoenablesDefaultLighting = true
view.backgroundColor = UIColor(white: 0.1, alpha: 1.0)
view.isPlaying = true // Force render loop
view.showsStatistics = true // Prove it's rendering
// Setup synthetic camera
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.camera?.zFar = 100
cameraNode.position = SCNVector3(x: 0, y: 0, z: 0) // Centered
view.scene?.rootNode.addChildNode(cameraNode)
context.coordinator.attach(to: view)
return view
}
func updateUIView(_ uiView: SCNView, context: Context) {}
final class Coordinator: NSObject {
private weak var sceneView: SCNView?
@Binding private var sunNodesReady: Bool
private let mockLocation: CLLocationCoordinate2D
private let mockHeading: Double
private var arcRootNode = SCNNode()
private var currentSunNode = SCNNode()
private var updateTimer: Timer?
init(sunNodesReady: Binding<Bool>, mockLocation: CLLocationCoordinate2D, mockHeading: Double) {
_sunNodesReady = sunNodesReady
self.mockLocation = mockLocation
self.mockHeading = mockHeading
super.init()
}
func attach(to view: SCNView) {
self.sceneView = view
view.scene?.rootNode.addChildNode(arcRootNode)
view.scene?.rootNode.addChildNode(currentSunNode)
buildScene()
startRealTimeTick()
}
deinit {
updateTimer?.invalidate()
}
private func startRealTimeTick() {
// Update current sun position every second
updateTimer = Timer.scheduledTimer(withTimeInterval: 1.0, repeats: true) { [weak self] _ in
guard let self = self else { return }
let cur = SunMath.calculateSunPosition(date: Date(), coordinate: self.mockLocation)
// Need to remove previous child as we are completely replacing it
self.currentSunNode.childNodes.forEach { $0.removeFromParentNode() }
let radius: Float = 1.8
let orb = SCNSphere(radius: 0.055)
orb.firstMaterial?.diffuse.contents = UIColor.systemOrange
orb.firstMaterial?.emission.contents = UIColor.systemYellow
orb.firstMaterial?.lightingModel = .constant
let orbNode = SCNNode(geometry: orb)
orbNode.position = self.worldPosition(for: cur, radius: radius)
let pulse = CABasicAnimation(keyPath: "scale")
pulse.fromValue = SCNVector3(1, 1, 1)
pulse.toValue = SCNVector3(1.3, 1.3, 1.3)
pulse.duration = 1.2
pulse.autoreverses = true
pulse.repeatCount = .infinity
orbNode.addAnimation(pulse, forKey: "pulse")
let label = self.makeTextNode(text: "Now", color: .systemYellow, fontSize: 0.05)
label.position = SCNVector3(0, 0.09, 0)
orbNode.addChildNode(label)
self.currentSunNode.addChildNode(orbNode)
}
}
private func buildScene() {
let arc = SunMath.sunPathArc(for: Date(), coordinate: mockLocation)
let riseSet = SunMath.sunRiseSet(for: Date(), coordinate: mockLocation)
let radius: Float = 1.8
var positions: [SCNVector3] = []
// Hourly blocks
for (date, pos) in arc {
guard pos.elevation > -5 else { continue }
let worldPos = worldPosition(for: pos, radius: radius)
positions.append(worldPos)
let sphere = SCNSphere(radius: 0.018)
sphere.firstMaterial?.diffuse.contents = UIColor.systemYellow.withAlphaComponent(0.85)
sphere.firstMaterial?.lightingModel = .constant
let markerNode = SCNNode(geometry: sphere)
markerNode.position = worldPos
arcRootNode.addChildNode(markerNode)
let calendar = Calendar.current
let hour = calendar.component(.hour, from: date)
if hour % 2 == 0 {
let labelNode = makeTextNode(text: hourLabel(from: date), color: .white, fontSize: 0.04)
labelNode.position = SCNVector3(worldPos.x, worldPos.y + 0.06, worldPos.z)
arcRootNode.addChildNode(labelNode)
}
}
if positions.count >= 2 {
let lineNode = makeLineNode(through: positions, color: UIColor.systemYellow.withAlphaComponent(0.55))
arcRootNode.addChildNode(lineNode)
}
if let riseDate = riseSet.rise {
let risePos = SunMath.calculateSunPosition(date: riseDate, coordinate: mockLocation)
let wPos = worldPosition(for: risePos, radius: radius)
arcRootNode.addChildNode(makeSpecialMarker(at: wPos, color: .systemOrange, label: "Sunrise \(hourLabel(from: riseDate))"))
}
if let setDate = riseSet.set {
let setPos = SunMath.calculateSunPosition(date: setDate, coordinate: mockLocation)
let wPos = worldPosition(for: setPos, radius: radius)
arcRootNode.addChildNode(makeSpecialMarker(at: wPos, color: .systemRed, label: "Sunset \(hourLabel(from: setDate))"))
}
// Generate current sun node synchronously for first frame
updateTimer?.fire()
DispatchQueue.main.async {
self.sunNodesReady = true
}
}
// MARK: Math equivalent from SunseekerViewModel
private func worldPosition(for sun: SunPosition, radius: Float) -> SCNVector3 {
let elev = Float(sun.elevation * .pi / 180.0)
let az = Float(sun.azimuth * .pi / 180.0)
let x = radius * cos(elev) * sin(az)
let y = radius * sin(elev)
let z = -radius * cos(elev) * cos(az)
return SCNVector3(x, y, z)
}
// MARK: SceneKit Factories
private func makeSpecialMarker(at pos: SCNVector3, color: UIColor, label: String) -> SCNNode {
let root = SCNNode()
let sphere = SCNSphere(radius: 0.035)
sphere.firstMaterial?.diffuse.contents = color
sphere.firstMaterial?.emission.contents = color.withAlphaComponent(0.5)
sphere.firstMaterial?.lightingModel = .constant
let markerNode = SCNNode(geometry: sphere)
markerNode.position = pos
root.addChildNode(markerNode)
let labelNode = makeTextNode(text: label, color: .white, fontSize: 0.04)
labelNode.position = SCNVector3(pos.x, pos.y + 0.07, pos.z)
root.addChildNode(labelNode)
return root
}
private func makeTextNode(text: String, color: UIColor, fontSize: CGFloat) -> SCNNode {
// SCNText is buggy in Simulator. Render text to a UIImage instead.
let font = UIFont.systemFont(ofSize: 40, weight: .bold)
let attributes: [NSAttributedString.Key: Any] = [
.font: font,
.foregroundColor: color
]
let size = (text as NSString).size(withAttributes: attributes)
// Add some padding
let paddedSize = CGSize(width: size.width + 10, height: size.height + 10)
let renderer = UIGraphicsImageRenderer(size: paddedSize)
let image = renderer.image { context in
(text as NSString).draw(
in: CGRect(x: 5, y: 5, width: size.width, height: size.height),
withAttributes: attributes
)
}
// Map the image onto an SCNPlane
// A 100x50 image becomes a 0.1 x 0.05 meter plane
let plane = SCNPlane(width: paddedSize.width / 1000.0, height: paddedSize.height / 1000.0)
plane.firstMaterial?.diffuse.contents = image
plane.firstMaterial?.isDoubleSided = true
plane.firstMaterial?.lightingModel = .constant
let textNode = SCNNode(geometry: plane)
// Statically scale the plane up so it is readable next to the spheres
textNode.scale = SCNVector3(1.5, 1.5, 1.5)
let constraint = SCNBillboardConstraint()
constraint.freeAxes = .all
textNode.constraints = [constraint]
return textNode
}
private func makeLineNode(through positions: [SCNVector3], color: UIColor) -> SCNNode {
guard positions.count >= 2 else { return SCNNode() }
var vertices: [SCNVector3] = positions
var indices: [Int32] = []
for i in 0..<(vertices.count - 1) {
indices.append(Int32(i))
indices.append(Int32(i + 1))
}
let vertexSource = SCNGeometrySource(vertices: vertices)
let element = SCNGeometryElement(
indices: indices,
primitiveType: .line
)
let geometry = SCNGeometry(sources: [vertexSource], elements: [element])
geometry.firstMaterial?.diffuse.contents = color
geometry.firstMaterial?.lightingModel = .constant
return SCNNode(geometry: geometry)
}
private func hourLabel(from date: Date) -> String {
let fmt = DateFormatter()
fmt.dateFormat = "ha"
fmt.amSymbol = "am"
fmt.pmSymbol = "pm"
return fmt.string(from: date)
}
}
}
#endif

140
iOS/velocity/velocity/Features/Inventory/SunseekerViewModel.swift Normal file
View File

@@ -0,0 +1,140 @@
import CoreLocation
import CoreMotion
import Foundation
import Observation
import SceneKit
// MARK: - SunseekerViewModel
/// Owns all sensor state for the Sunseeker AR overlay.
/// Separates CoreLocation / CoreMotion concerns from the ARKit view layer.
@Observable
final class SunseekerViewModel: NSObject, CLLocationManagerDelegate {
// MARK: - Published State
/// True once we have both a GPS fix and a valid heading.
private(set) var isReady = false
/// Latest GPS coordinate. nil until first fix.
private(set) var coordinate: CLLocationCoordinate2D?
/// Latest true heading (0 = North, clockwise).
private(set) var heading: Double = 0
/// Dense hourly arc for today.
private(set) var arc: [(date: Date, position: SunPosition)] = []
/// Current real-time sun position (updated every second).
private(set) var currentPosition: SunPosition?
/// Sunrise and sunset for today.
private(set) var riseSet: (rise: Date?, set: Date?) = (nil, nil)
/// Diagnostic string for the UI when location access is denied.
private(set) var locationError: String?
// MARK: - Private
private let locationManager = CLLocationManager()
private let motionManager = CMMotionManager()
private var updateTimer: Timer?
// MARK: - Init / Deinit
override init() {
super.init()
locationManager.delegate = self
locationManager.desiredAccuracy = kCLLocationAccuracyBest
locationManager.headingFilter = 1.0
locationManager.requestWhenInUseAuthorization()
locationManager.startUpdatingLocation()
locationManager.startUpdatingHeading()
startMotionUpdates()
startRealTimeTick()
}
deinit {
stop()
}
// MARK: - Control
func stop() {
motionManager.stopDeviceMotionUpdates()
locationManager.stopUpdatingLocation()
locationManager.stopUpdatingHeading()
updateTimer?.invalidate()
updateTimer = nil
}
// MARK: - World-Space Transform
/// Converts a solar `SunPosition` into a SceneKit world-space position on a sphere of given `radius`.
/// Orientation is relative to ARWorldTrackingConfiguration(.gravityAndHeading), so north = -Z axis.
func worldPosition(for sun: SunPosition, radius: Float) -> SCNVector3 {
let elev = Float(sun.elevation.radians)
let az = Float(sun.azimuth.radians) // clockwise from north
let x = radius * cos(elev) * sin(az)
let y = radius * sin(elev)
let z = -radius * cos(elev) * cos(az) // -Z = north in ARKit gravity+heading
return SCNVector3(x, y, z)
}
// MARK: - Private helpers
private func startMotionUpdates() {
guard motionManager.isDeviceMotionAvailable else { return }
motionManager.deviceMotionUpdateInterval = 0.05
motionManager.startDeviceMotionUpdates()
}
private func startRealTimeTick() {
updateTimer = Timer.scheduledTimer(withTimeInterval: 1.0, repeats: true) { [weak self] _ in
guard let self, let coord = self.coordinate else { return }
self.currentPosition = SunMath.calculateSunPosition(date: Date(), coordinate: coord)
}
}
private func refreshArc() {
guard let coord = coordinate else { return }
arc = SunMath.sunPathArc(for: Date(), coordinate: coord)
riseSet = SunMath.sunRiseSet(for: Date(), coordinate: coord)
currentPosition = SunMath.calculateSunPosition(date: Date(), coordinate: coord)
isReady = true
}
// MARK: - CLLocationManagerDelegate
func locationManager(_ manager: CLLocationManager, didUpdateLocations locations: [CLLocation]) {
guard coordinate == nil, let loc = locations.last else { return }
coordinate = loc.coordinate
refreshArc()
}
func locationManager(_ manager: CLLocationManager, didUpdateHeading newHeading: CLHeading) {
heading = newHeading.trueHeading > 0 ? newHeading.trueHeading : newHeading.magneticHeading
if coordinate != nil { isReady = true }
}
func locationManager(_ manager: CLLocationManager, didFailWithError error: Error) {
print("[Sunseeker] Location error: \(error.localizedDescription)")
}
func locationManagerDidChangeAuthorization(_ manager: CLLocationManager) {
switch manager.authorizationStatus {
case .denied, .restricted:
locationError = "Location access needed to calculate the sun path. Please enable it in Settings."
case .notDetermined:
manager.requestWhenInUseAuthorization()
default:
locationError = nil
}
}
}
// MARK: - Degree helpers (internal to this file)
private extension Double {
var radians: Double { self * .pi / 180.0 }
}

3
iOS/velocity/velocity/Info.plist
View File

@@ -13,5 +13,8 @@
<string>Velocity would like to save your AI-generated room design to your photo library.</string> <string>Velocity would like to save your AI-generated room design to your photo library.</string>
<key>NSCameraUsageDescription</key> <key>NSCameraUsageDescription</key>
<string>Velocity needs camera access to capture room photos for Dream Weaver.</string> <string>Velocity needs camera access to capture room photos for Dream Weaver.</string>
<key>NSLocationWhenInUseUsageDescription</key>
<string>Velocity uses your location to calculate the accurate sun path for your property.</string>
</dict> </dict>
</plist> </plist>