forked from sagnik/Project_Velocity
feat: Implement the camera capture feature to take photos of empty walls/rooms
This commit is contained in:
Sayan Datta
512
.Agent Context/Bibels/dreamweaver_ios_integration_guide.md
Normal file
512
.Agent Context/Bibels/dreamweaver_ios_integration_guide.md
Normal file
@@ -0,0 +1,512 @@
|
||||
# Dream Weaver — iOS ↔ Backend Integration Guide
|
||||
**Version:** 2.0 | **Updated:** 2026-03-09 | **Server:** `54.172.172.2` | **Port:** `8080`
|
||||
|
||||
> This document is for **Sayan** (iOS / Swift) and **Sourik** (backend review).
|
||||
> It describes exactly how the iPad app should talk to the Dream Weaver AI and how keywords from a user tap become a full ComfyUI generation.
|
||||
|
||||
---
|
||||
|
||||
## 1. Architecture Overview
|
||||
|
||||
```
|
||||
┌────────────────────┐ HTTP/S ┌──────────────────────────────┐
|
||||
│ │ ─── POST image ───► │ Dream Weaver Gateway │
|
||||
│ iPad App (Swift) │ │ FastAPI port 8080 │
|
||||
│ │ ◄── PNG result ─── │ dw_gateway.py │
|
||||
└────────────────────┘ └─────────────┬────────────────┘
|
||||
│ internal HTTP
|
||||
▼
|
||||
┌─────────────────────────┐
|
||||
│ ComfyUI Engine │
|
||||
│ port 8188 │
|
||||
│ RealVisXL V5.0 Ltng │
|
||||
│ 4× NVIDIA L4 (96 GB) │
|
||||
└─────────────────────────┘
|
||||
```
|
||||
|
||||
**Key rule:** The iPad app **never** talks to ComfyUI directly. It only talks to the Gateway on `:8080`.
|
||||
|
||||
---
|
||||
|
||||
## 2. How Keywords Become Prompts
|
||||
|
||||
### 2.1 The Prompt Expansion System
|
||||
|
||||
Each interior style in the app is backed by a **prompt template** in `comfy_engine/prompts/`. When the user taps a style card (or types keywords), those keywords get **merged into the template** to build the final ComfyUI prompt injected into node `9` (positive CLIPTextEncode).
|
||||
|
||||
**Prompt template structure** (from [scandinavian_minimalist.txt](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/prompts/scandinavian_minimalist.txt)):
|
||||
```
|
||||
POSITIVE PROMPT:
|
||||
scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles,
|
||||
abundant natural light streaming through large windows, clean white walls, ...
|
||||
Style Weight: <lora:Interior_Style_Scandi:0.8>
|
||||
|
||||
NEGATIVE PROMPT:
|
||||
(worst quality, low quality, illustration, 3d render...), heavy ornamentation,...
|
||||
|
||||
TECHNICAL PARAMETERS:
|
||||
- Denoising Strength: 0.70
|
||||
- CFG Scale: 7.0
|
||||
- Recommended Sampler: dpmpp_2m_karras
|
||||
- Steps: 30-40
|
||||
```
|
||||
|
||||
### 2.2 Keyword Expansion Flow
|
||||
|
||||
```
|
||||
User taps: ["marble", "gold", "luxury"]
|
||||
+
|
||||
Style selected: "art_deco"
|
||||
│
|
||||
▼
|
||||
Backend expands:
|
||||
base_prompt = art_deco_luxe.txt (POSITIVE PROMPT section)
|
||||
user_keywords_str = "marble, gold, luxury"
|
||||
final_prompt = base_prompt + ", " + user_keywords_str
|
||||
│
|
||||
▼
|
||||
Injected into ComfyUI workflow:
|
||||
node "9" → CLIPTextEncode → text: [final_prompt]
|
||||
node "10" → CLIPTextEncode → text: [negative_prompt from template]
|
||||
node "1" → LoadImage → image: [uploaded filename]
|
||||
node "13" → KSampler → denoise: 0.72, cfg: 7.5, steps: 35
|
||||
```
|
||||
|
||||
### 2.3 Available Styles and Their Keywords (for the Style Picker UI)
|
||||
|
||||
| Style ID | Display Name | Suggested Keywords Palette |
|
||||
|---|---|---|
|
||||
| `scandinavian` | Scandinavian Minimalist | oak, linen, white, hygge, cozy, birch, natural |
|
||||
| `art_deco` | Art Deco Luxe | gold, marble, velvet, geometric, 1920s, brass, crystal |
|
||||
| `biophilic` | Biophilic Organic | green wall, stone, rattan, terracotta, botanical, moss |
|
||||
| `cyberpunk` | Cyberpunk Neon | neon, chrome, holographic, dark, LED, futuristic, blade runner |
|
||||
| `japandi` | Japandi Fusion | wabi-sabi, ash wood, ceramic, zen, minimal, shoji, serene |
|
||||
|
||||
---
|
||||
|
||||
## 3. API Reference — What Sayan Needs to Call
|
||||
|
||||
### BASE URL
|
||||
```
|
||||
http://54.172.172.2:8080
|
||||
```
|
||||
|
||||
> [!NOTE]
|
||||
> Once we attach an Elastic IP or domain, swap this in `AppConfig.swift`.
|
||||
|
||||
---
|
||||
|
||||
### 3.1 `GET /health` — Liveness Check
|
||||
|
||||
Call this on app launch to confirm the server is up before showing the Generate button.
|
||||
|
||||
**Request:**
|
||||
```http
|
||||
GET http://54.172.172.2:8080/health
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"comfyui": true,
|
||||
"gpu": "4x NVIDIA L4 (96GB VRAM)",
|
||||
"model": "RealVisXL V5.0 Lightning"
|
||||
}
|
||||
```
|
||||
|
||||
**Swift:**
|
||||
```swift
|
||||
func checkServerHealth() async throws -> Bool {
|
||||
let url = URL(string: "\(AppConfig.baseURL)/health")!
|
||||
let (data, _) = try await URLSession.shared.data(from: url)
|
||||
let json = try JSONDecoder().decode(HealthResponse.self, from: data)
|
||||
return json.status == "ok"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3.2 `POST /dream-weaver` — Submit Generation Job (Async)
|
||||
|
||||
Use this for the main generation flow. Returns a `job_id` immediately; poll for result.
|
||||
|
||||
**Request:** `multipart/form-data`
|
||||
| Field | Type | Required | Description |
|
||||
|---|---|---|---|
|
||||
| [image](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/a100_deployment_executor.py#306-322) | File (JPEG/PNG) | ✅ | The room photo from camera or library |
|
||||
| [style](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/dreamweaver_batch_processor.py#394-414) | String | ✅ | One of: `scandinavian`, `art_deco`, `biophilic`, `cyberpunk`, `japandi` |
|
||||
| `keywords` | String | ➕ | Comma-separated user keywords e.g. `"gold, marble, luxury"` |
|
||||
| `denoise` | Float | ➕ | 0.5–0.85 (default `0.72`). Higher = more creative |
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"job_id": "a1b2c3d4-...",
|
||||
"status": "processing",
|
||||
"poll_url": "/dream-weaver/status/a1b2c3d4-...",
|
||||
"result_url": "/dream-weaver/result/a1b2c3d4-..."
|
||||
}
|
||||
```
|
||||
|
||||
**Swift example:**
|
||||
```swift
|
||||
func submitGeneration(image: UIImage, style: String, keywords: [String]) async throws -> GenerationJob {
|
||||
let url = URL(string: "\(AppConfig.baseURL)/dream-weaver")!
|
||||
var request = URLRequest(url: url)
|
||||
request.httpMethod = "POST"
|
||||
|
||||
let boundary = UUID().uuidString
|
||||
request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")
|
||||
|
||||
var body = Data()
|
||||
// Image field
|
||||
let imageData = image.jpegData(compressionQuality: 0.85)!
|
||||
body.append("--\(boundary)\r\n".data(using: .utf8)!)
|
||||
body.append("Content-Disposition: form-data; name=\"image\"; filename=\"room.jpg\"\r\n".data(using: .utf8)!)
|
||||
body.append("Content-Type: image/jpeg\r\n\r\n".data(using: .utf8)!)
|
||||
body.append(imageData)
|
||||
body.append("\r\n".data(using: .utf8)!)
|
||||
|
||||
// Style field
|
||||
body.append("--\(boundary)\r\n".data(using: .utf8)!)
|
||||
body.append("Content-Disposition: form-data; name=\"style\"\r\n\r\n".data(using: .utf8)!)
|
||||
body.append("\(style)\r\n".data(using: .utf8)!)
|
||||
|
||||
// Keywords field (user tapped keywords)
|
||||
if !keywords.isEmpty {
|
||||
let kwString = keywords.joined(separator: ", ")
|
||||
body.append("--\(boundary)\r\n".data(using: .utf8)!)
|
||||
body.append("Content-Disposition: form-data; name=\"keywords\"\r\n\r\n".data(using: .utf8)!)
|
||||
body.append("\(kwString)\r\n".data(using: .utf8)!)
|
||||
}
|
||||
|
||||
body.append("--\(boundary)--\r\n".data(using: .utf8)!)
|
||||
request.httpBody = body
|
||||
|
||||
let (data, _) = try await URLSession.shared.data(for: request)
|
||||
return try JSONDecoder().decode(GenerationJob.self, from: data)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3.3 `GET /dream-weaver/status/{job_id}` — Poll Job Status
|
||||
|
||||
Poll every **2 seconds** until `ready == true`.
|
||||
|
||||
**Response while processing:**
|
||||
```json
|
||||
{ "status": "processing", "ready": false, "style": "art_deco" }
|
||||
```
|
||||
|
||||
**Response when done:**
|
||||
```json
|
||||
{
|
||||
"status": "done",
|
||||
"ready": true,
|
||||
"result_url": "/dream-weaver/result/a1b2c3d4-...",
|
||||
"style": "art_deco"
|
||||
}
|
||||
```
|
||||
|
||||
**Swift polling loop:**
|
||||
```swift
|
||||
func pollForResult(jobId: String) async throws -> URL {
|
||||
let statusURL = URL(string: "\(AppConfig.baseURL)/dream-weaver/status/\(jobId)")!
|
||||
|
||||
for _ in 0..<150 { // max 5 min (150 × 2s)
|
||||
try await Task.sleep(nanoseconds: 2_000_000_000) // 2 seconds
|
||||
let (data, _) = try await URLSession.shared.data(from: statusURL)
|
||||
let status = try JSONDecoder().decode(JobStatus.self, from: data)
|
||||
|
||||
if status.ready {
|
||||
return URL(string: "\(AppConfig.baseURL)/dream-weaver/result/\(jobId)")!
|
||||
}
|
||||
if status.status == "error" {
|
||||
throw DreamWeaverError.generationFailed(status.error ?? "Unknown")
|
||||
}
|
||||
}
|
||||
throw DreamWeaverError.timeout
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3.4 `GET /dream-weaver/result/{job_id}` — Download Result Image
|
||||
|
||||
Returns a PNG image stream directly. Download and display to user.
|
||||
|
||||
```swift
|
||||
func downloadResult(resultURL: URL) async throws -> UIImage {
|
||||
let (data, _) = try await URLSession.shared.data(from: resultURL)
|
||||
guard let image = UIImage(data: data) else {
|
||||
throw DreamWeaverError.invalidImageData
|
||||
}
|
||||
return image
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3.5 `POST /dream-weaver/sync` — One-Shot Blocking Call
|
||||
|
||||
For **testing only** or fast network connections. Waits up to 120 seconds and returns the image directly.
|
||||
|
||||
```swift
|
||||
// Same multipart form as /dream-weaver but returns PNG bytes directly
|
||||
// Not recommended for production — use async flow above
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Keyword → Prompt Expansion (Backend Change Required)
|
||||
|
||||
> [!IMPORTANT]
|
||||
> The current [dw_gateway.py](file:///C:/Windows/Temp/dw_gateway.py) does NOT yet accept `keywords` from the app.
|
||||
> Sagnik needs to add keyword expansion to the gateway. Here is the exact code change:
|
||||
|
||||
**In [dw_gateway.py](file:///C:/Windows/Temp/dw_gateway.py), update [build_workflow()](file:///C:/Windows/Temp/dw_gateway.py#33-65):**
|
||||
|
||||
```python
|
||||
# Prompt library — maps style ID to (positive_base, negative, cfg, denoise, steps)
|
||||
STYLE_LIBRARY = {
|
||||
"scandinavian": {
|
||||
"pos": "scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles, abundant natural light, clean white walls, simple functional furniture, cozy hygge atmosphere, architectural photography, 8k resolution, photorealistic",
|
||||
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, extra windows, unrealistic lighting, structural changes",
|
||||
"cfg": 7.0, "denoise": 0.70, "steps": 30,
|
||||
},
|
||||
"art_deco": {
|
||||
"pos": "art deco luxury interior design, geometric chevron patterns, gold brass accents, rich velvet upholstery emerald and sapphire, sunburst mirrors, polished marble flooring, crystal chandeliers, 1920s glamour, 8k resolution, photorealistic",
|
||||
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, rustic, minimalism, cheap materials",
|
||||
"cfg": 7.5, "denoise": 0.72, "steps": 30,
|
||||
},
|
||||
"biophilic": {
|
||||
"pos": "biophilic organic interior design, living green walls with ferns and moss, natural stone accent walls, rattan and bamboo furniture, abundant houseplants, earth tone sage green and terracotta, 8k resolution, photorealistic, dappled sunlight",
|
||||
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, synthetic materials, plastic plants",
|
||||
"cfg": 7.0, "denoise": 0.68, "steps": 30,
|
||||
},
|
||||
"cyberpunk": {
|
||||
"pos": "cyberpunk neon interior design, high contrast LED strip lighting electric blue and hot pink, reflective chrome surfaces, dark matte walls, futuristic furniture, glowing circuit patterns, tech-noir blade runner aesthetic, 8k resolution, photorealistic, volumetric fog",
|
||||
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, natural daylight, rustic elements",
|
||||
"cfg": 8.0, "denoise": 0.75, "steps": 30,
|
||||
},
|
||||
"japandi": {
|
||||
"pos": "japandi fusion interior design, wabi-sabi textures, low-profile furniture, muted earth tones warm grays soft browns, handmade ceramic accents, light ash wood, shoji screen elements, minimal decoration, zen garden elements, 8k resolution, photorealistic, serene",
|
||||
"neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, bright colors, ornate decoration, cluttered",
|
||||
"cfg": 6.5, "denoise": 0.70, "steps": 30,
|
||||
},
|
||||
}
|
||||
|
||||
def build_workflow(img_name: str, style: str = "scandinavian",
|
||||
keywords: str = "", denoise_override: float = None) -> dict:
|
||||
s = STYLE_LIBRARY.get(style, STYLE_LIBRARY["scandinavian"])
|
||||
|
||||
# Merge user keywords into base positive prompt
|
||||
pos = s["pos"]
|
||||
if keywords.strip():
|
||||
pos = pos + ", " + keywords.strip()
|
||||
|
||||
cfg = s["cfg"]
|
||||
denoise = denoise_override if denoise_override else s["denoise"]
|
||||
steps = s["steps"]
|
||||
neg = s["neg"]
|
||||
|
||||
return {
|
||||
"1": {"class_type": "CheckpointLoaderSimple",
|
||||
"inputs": {"ckpt_name": "realvisxlV50_v50LightningBakedvae.safetensors"}},
|
||||
"2": {"class_type": "LoadImage",
|
||||
"inputs": {"image": img_name, "upload": "image"}},
|
||||
"3": {"class_type": "CLIPTextEncode",
|
||||
"inputs": {"text": pos, "clip": ["1", 1]}}, # ← POSITIVE PROMPT
|
||||
"4": {"class_type": "CLIPTextEncode",
|
||||
"inputs": {"text": neg, "clip": ["1", 1]}}, # ← NEGATIVE PROMPT
|
||||
"5": {"class_type": "VAEEncode",
|
||||
"inputs": {"pixels": ["2", 0], "vae": ["1", 2]}},
|
||||
"6": {"class_type": "KSampler",
|
||||
"inputs": {"model": ["1", 0], "positive": ["3", 0], "negative": ["4", 0],
|
||||
"latent_image": ["5", 0],
|
||||
"seed": int(time.time()) % 999983,
|
||||
"steps": steps, "cfg": cfg,
|
||||
"sampler_name": "dpmpp_2m", "scheduler": "karras",
|
||||
"denoise": denoise}},
|
||||
"7": {"class_type": "VAEDecode",
|
||||
"inputs": {"samples": ["6", 0], "vae": ["1", 2]}},
|
||||
"8": {"class_type": "SaveImage",
|
||||
"inputs": {"images": ["7", 0], "filename_prefix": f"dreamweaver_{style}"}}
|
||||
}
|
||||
```
|
||||
|
||||
**Also update the endpoint signatures** to accept `keywords`:
|
||||
```python
|
||||
@app.post("/dream-weaver")
|
||||
async def dream_weaver(
|
||||
image: UploadFile = File(...),
|
||||
style: str = Form(default="scandinavian"),
|
||||
keywords: str = Form(default=""), # ← ADD THIS
|
||||
denoise: float = Form(default=0.0), # 0.0 = use style default
|
||||
):
|
||||
...
|
||||
wf = build_workflow(comfy_name, style=style, keywords=keywords,
|
||||
denoise_override=denoise if denoise > 0 else None)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Swift Data Models
|
||||
|
||||
```swift
|
||||
// AppConfig.swift
|
||||
struct AppConfig {
|
||||
static let baseURL = "http://54.172.172.2:8080"
|
||||
// Change this to HTTPS domain once SSL is set up
|
||||
}
|
||||
|
||||
// Models
|
||||
struct GenerationJob: Codable {
|
||||
let jobId: String
|
||||
let status: String
|
||||
let pollUrl: String
|
||||
let resultUrl: String
|
||||
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case jobId = "job_id"
|
||||
case status, pollUrl = "poll_url", resultUrl = "result_url"
|
||||
}
|
||||
}
|
||||
|
||||
struct JobStatus: Codable {
|
||||
let status: String
|
||||
let ready: Bool
|
||||
let resultUrl: String?
|
||||
let error: String?
|
||||
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case status, ready, resultUrl = "result_url", error
|
||||
}
|
||||
}
|
||||
|
||||
struct HealthResponse: Codable {
|
||||
let status: String
|
||||
let comfyui: Bool
|
||||
}
|
||||
|
||||
enum DreamWeaverError: Error {
|
||||
case generationFailed(String)
|
||||
case timeout
|
||||
case invalidImageData
|
||||
}
|
||||
|
||||
// Style model for the style picker
|
||||
struct InteriorStyle: Identifiable {
|
||||
let id: String // used as the `style` form field value
|
||||
let displayName: String
|
||||
let keywords: [String] // shown as tappable chips in UI
|
||||
let previewImage: String // local asset name
|
||||
}
|
||||
|
||||
let availableStyles: [InteriorStyle] = [
|
||||
InteriorStyle(id: "scandinavian", displayName: "Scandinavian", keywords: ["oak","linen","white","hygge","cozy","birch"], previewImage: "style_scandi"),
|
||||
InteriorStyle(id: "art_deco", displayName: "Art Deco Luxe", keywords: ["gold","marble","velvet","geometric","brass","crystal"], previewImage: "style_artdeco"),
|
||||
InteriorStyle(id: "biophilic", displayName: "Biophilic", keywords: ["green wall","stone","rattan","terracotta","botanical"], previewImage: "style_biophilic"),
|
||||
InteriorStyle(id: "cyberpunk", displayName: "Cyberpunk Neon", keywords: ["neon","chrome","LED","futuristic","dark","holographic"], previewImage: "style_cyberpunk"),
|
||||
InteriorStyle(id: "japandi", displayName: "Japandi Fusion", keywords: ["wabi-sabi","ceramic","ash wood","zen","minimal"], previewImage: "style_japandi"),
|
||||
]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Complete Generation Flow (UI → Server → Result)
|
||||
|
||||
```
|
||||
1. User opens camera / library → picks a room photo
|
||||
2. User selects a style card → style ID captured
|
||||
3. User optionally taps keyword chips → keywords[] array built
|
||||
4. User taps "Generate" →
|
||||
|
||||
POST /dream-weaver (multipart)
|
||||
image: <jpeg data>
|
||||
style: "art_deco"
|
||||
keywords: "gold, marble, luxury hotel"
|
||||
denoise: 0.72
|
||||
|
||||
5. Server returns { job_id: "abc123", status: "processing" }
|
||||
6. App shows loading/progress UI
|
||||
7. App polls GET /dream-weaver/status/abc123 every 2 seconds
|
||||
8. When ready == true →
|
||||
GET /dream-weaver/result/abc123 → returns PNG bytes
|
||||
9. App displays result full-screen with save/share options
|
||||
```
|
||||
|
||||
**Expected latency on 4× L4 GPU server:** `~15–20 seconds` end-to-end.
|
||||
|
||||
---
|
||||
|
||||
## 7. WebSocket Progress (Optional Advanced Feature)
|
||||
|
||||
If Sayan wants a real-time progress bar (e.g. "Step 12/30"), connect directly to ComfyUI's WebSocket **only if port 8188 is opened**. Otherwise, polling `/status` is sufficient.
|
||||
|
||||
```swift
|
||||
// WebSocket — only if 8188 is exposed externally
|
||||
class ComfyProgressWebSocket: NSObject, URLSessionWebSocketDelegate {
|
||||
var onProgress: ((Int, Int) -> Void)?
|
||||
var task: URLSessionWebSocketTask?
|
||||
|
||||
func connect(clientId: String) {
|
||||
let url = URL(string: "ws://54.172.172.2:8188/ws?clientId=\(clientId)")!
|
||||
task = URLSession.shared.webSocketTask(with: url)
|
||||
task?.resume()
|
||||
listen()
|
||||
}
|
||||
|
||||
private func listen() {
|
||||
task?.receive { [weak self] result in
|
||||
if case .success(let message) = result,
|
||||
case .string(let text) = message,
|
||||
let data = text.data(using: .utf8),
|
||||
let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
|
||||
let type = json["type"] as? String, type == "progress",
|
||||
let inner = json["data"] as? [String: Int] {
|
||||
self?.onProgress?(inner["value"] ?? 0, inner["max"] ?? 30)
|
||||
}
|
||||
self?.listen() // recurse
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
> [!NOTE]
|
||||
> Port 8188 is currently **not open externally** in the security group. Only port 8080 is. To use WebSocket progress, Sagnik needs to add an inbound rule for 8188. Until then, using `/status` polling every 2s gives good enough UX.
|
||||
|
||||
---
|
||||
|
||||
## 8. Error Handling
|
||||
|
||||
| HTTP Status | Meaning | UI Action |
|
||||
|---|---|---|
|
||||
| `200` | Success | Show result or job_id |
|
||||
| `404` on `/status` | Job expired (> 30 min) | "Session expired. Please retry." |
|
||||
| `500` | Generation failed (OOM, model error) | "Generation failed. Try a simpler image." |
|
||||
| Connection error | Server down or no internet | "Checking server…" + retry logic |
|
||||
|
||||
The job [status](file:///C:/Windows/Temp/dw_gateway.py#154-164) field can also be `"error"` with an `error` field explaining what failed.
|
||||
|
||||
---
|
||||
|
||||
## 9. Quick Checklist for Sayan
|
||||
|
||||
- [ ] Update `AppConfig.swift` with `baseURL = "http://54.172.172.2:8080"`
|
||||
- [ ] Implement `POST /dream-weaver` multipart with `image + style + keywords`
|
||||
- [ ] Implement polling loop on `GET /dream-weaver/status/{job_id}`
|
||||
- [ ] Implement image download from `GET /dream-weaver/result/{job_id}`
|
||||
- [ ] Add `GET /health` check on app launch
|
||||
- [ ] Build keyword chips UI with the 5 style palettes from Section 2.3
|
||||
- [ ] Test with the 20 sample images in `comfy_engine/test_inputs/`
|
||||
|
||||
## 10. Quick Checklist for Sagnik (backend)
|
||||
|
||||
- [ ] Update [dw_gateway.py](file:///C:/Windows/Temp/dw_gateway.py) with the full `STYLE_LIBRARY` dict (Section 4)
|
||||
- [ ] Add `keywords: str = Form(default="")` to both POST endpoints
|
||||
- [ ] Pass keywords into [build_workflow()](file:///C:/Windows/Temp/dw_gateway.py#33-65) for prompt expansion
|
||||
- [ ] Redeploy gateway on port 8080 (`nohup python3 dw_gateway.py &`)
|
||||
- [ ] (Optional) Open port 8188 in security group for WebSocket progress
|
||||
810
.Agent Context/DREAMWEAVER_TECHNICAL_SPEC.md
Normal file
810
.Agent Context/DREAMWEAVER_TECHNICAL_SPEC.md
Normal file
@@ -0,0 +1,810 @@
|
||||
# Dream Weaver Technical Specification
|
||||
|
||||
**Version:** 1.0.0
|
||||
**Date:** 2026-03-01
|
||||
**Model:** RealVisXL V5.0 Lightning
|
||||
**Target Hardware Phase 1:** NVIDIA RTX 3080Ti (12GB GDDR6X)
|
||||
**Target Hardware Phase 3:** Dual NVIDIA RTX PRO 6000 Blackwell (96GB GDDR7 each)
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Executive Summary](#executive-summary)
|
||||
2. [Three-Phase Implementation Architecture](#three-phase-implementation-architecture)
|
||||
3. [Hardware Specifications & Optimization](#hardware-specifications--optimization)
|
||||
4. [Model Specifications & Downloads](#model-specifications--downloads)
|
||||
5. [ControlNet Configuration](#controlnet-configuration)
|
||||
6. [Custom Node Requirements](#custom-node-requirements)
|
||||
7. [Phase 1: Foundational Implementation](#phase-1-foundational-implementation)
|
||||
8. [Phase 2: Advanced Multi-ControlNet](#phase-2-advanced-multi-controlnet)
|
||||
9. [Phase 3: Production Batch Processing](#phase-3-production-batch-processing)
|
||||
10. [Prompt Engineering Templates](#prompt-engineering-templates)
|
||||
11. [API Integration Guide](#api-integration-guide)
|
||||
12. [Deployment Instructions](#deployment-instructions)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Dream Weaver is an interior restyling workflow that uses **Structural Constraint Logic** to preserve existing room geometry while enabling comprehensive aesthetic transformations. The system employs a **Dual-ControlNet Strategy** combining M-LSD (Line Segment Detection) for architectural line preservation and Depth (Zoe/MiDaS) for 3D spatial consistency, with SAM-based masking to isolate structural immutables from stylable regions.
|
||||
|
||||
### Core Constraint: Absolute Geometry Preservation
|
||||
|
||||
The following elements are **IMMUTABLE** and must never be modified:
|
||||
- Wall positions and angles
|
||||
- Door and window placements
|
||||
- Ceiling heights
|
||||
- Room proportions and dimensions
|
||||
- Structural load-bearing elements
|
||||
- Vanishing points and perspective
|
||||
|
||||
The following elements are **MUTABLE** and may be restyled:
|
||||
- Wall paint colors and textures
|
||||
- Flooring materials
|
||||
- Furniture upholstery and styles
|
||||
- Decorative objects and accessories
|
||||
- Lighting fixtures and atmospheres
|
||||
- Soft furnishings (curtains, rugs, cushions)
|
||||
|
||||
---
|
||||
|
||||
## Three-Phase Implementation Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Input Interior Image] --> B[Phase 1: Foundational]
|
||||
B --> C[Phase 2: Advanced]
|
||||
C --> D[Phase 3: Production]
|
||||
|
||||
subgraph P1[Phase 1 - RTX 3080Ti]
|
||||
B1[Depth ControlNet] --> B2[Basic SAM Masking]
|
||||
B2 --> B3[Single Image Processing]
|
||||
end
|
||||
|
||||
subgraph P2[Phase 2 - Enhanced Quality]
|
||||
C1[Multi-ControlNet] --> C2[Refined Masking]
|
||||
C2 --> C3[Style Templates]
|
||||
end
|
||||
|
||||
subgraph P3[Phase 3 - Dual RTX PRO 6000]
|
||||
D1[Batch Processing] --> D2[4K Upscaling]
|
||||
D2 --> D3[Automated Pipeline]
|
||||
end
|
||||
```
|
||||
|
||||
### Phase Overview
|
||||
|
||||
| Phase | Hardware | ControlNets | Resolution | Batch Size | Purpose |
|
||||
|-------|----------|-------------|------------|------------|---------|
|
||||
| 1 | RTX 3080Ti | 1 (Depth) | 1024x1024 | 1 | Validation & Testing |
|
||||
| 2 | RTX 3080Ti | 3 (Depth + Seg + Canny) | 1216x832 | 1 | Quality Enhancement |
|
||||
| 3 | Dual RTX PRO 6000 | 3 + Aux | 2048x2048 | 8+ | Production Deployment |
|
||||
|
||||
---
|
||||
|
||||
## Hardware Specifications & Optimization
|
||||
|
||||
### Current Development Hardware: RTX 3080Ti
|
||||
|
||||
**Specifications:**
|
||||
- GPU: NVIDIA RTX 3080Ti
|
||||
- VRAM: 12GB GDDR6X
|
||||
- CUDA Cores: 10,240
|
||||
- Architecture: Ampere
|
||||
|
||||
**VRAM Management Strategy:**
|
||||
```python
|
||||
# Optimization flags for 12GB VRAM
|
||||
--fp16 # Enable half-precision
|
||||
--lowvram # Aggressive memory management
|
||||
--disable-xformers # Use sdp-attention instead
|
||||
```
|
||||
|
||||
**Recommended Settings:**
|
||||
- Batch size: 1
|
||||
- Maximum resolution: 1024x1024 or 1216x832
|
||||
- Tiled VAE: Enabled with tile size 64
|
||||
- Model CPU offloading: Enabled
|
||||
- Empty cache after each generation: Enabled
|
||||
|
||||
### Production Hardware: Dual RTX PRO 6000 Blackwell
|
||||
|
||||
**Specifications:**
|
||||
- GPU: 2x NVIDIA RTX PRO 6000 Blackwell
|
||||
- VRAM: 96GB GDDR7 per GPU (192GB total)
|
||||
- Architecture: Blackwell
|
||||
- NVLink: Enabled for memory pooling
|
||||
|
||||
**Optimization Strategy:**
|
||||
```python
|
||||
# Production flags for 192GB VRAM
|
||||
--bf16 # Enable bfloat16 for better precision
|
||||
--highvram # Keep models in GPU memory
|
||||
--xformers # Enable memory-efficient attention
|
||||
--gpu-batch-size 8 # Process 8 images simultaneously
|
||||
--model-sharding # Distribute across both GPUs
|
||||
```
|
||||
|
||||
### VRAM Usage Comparison
|
||||
|
||||
| Configuration | Phase 1 | Phase 2 | Phase 3 |
|
||||
|--------------|---------|---------|---------|
|
||||
| Model Loading | 6.2GB | 6.2GB | 6.2GB |
|
||||
| ControlNet 1 | 1.8GB | 1.8GB | 1.8GB |
|
||||
| ControlNet 2 | - | 1.8GB | 1.8GB |
|
||||
| ControlNet 3 | - | 1.5GB | 1.5GB |
|
||||
| SAM Model | 2.1GB | 2.1GB | 2.1GB |
|
||||
| Latent Buffers | 1.5GB | 2.2GB | 8.0GB |
|
||||
| **Total** | **~11.6GB** | **~15.6GB** | **~21.4GB** |
|
||||
|
||||
---
|
||||
|
||||
## Model Specifications & Downloads
|
||||
|
||||
### Primary Checkpoint: RealVisXL V5.0 Lightning
|
||||
|
||||
**Download URL:** https://civitai.com/models/139562?modelVersionId=789646
|
||||
|
||||
**Specifications:**
|
||||
- Base Model: SDXL
|
||||
- Training Data: Architectural photography datasets
|
||||
- Specialization: Photorealistic interiors, white balance accuracy
|
||||
- Lightning Steps: 4-8 steps for high quality
|
||||
- Recommended CFG: 1.0-2.0 (Lightning)
|
||||
- CLIP Skip: 2
|
||||
|
||||
**File Details:**
|
||||
- Filename: `realvisxlV50Lightning_v50Lightning.safetensors`
|
||||
- Expected Size: ~6.5GB
|
||||
- Format: SafeTensors
|
||||
- SHA256: Verify on download
|
||||
|
||||
**Installation Path:**
|
||||
```
|
||||
ComfyUI/models/checkpoints/realvisxlV50Lightning_v50Lightning.safetensors
|
||||
```
|
||||
|
||||
### VAE Selection
|
||||
|
||||
**Option A: Automatic1111 VAE**
|
||||
- Download: https://huggingface.co/stabilityai/sdxl-vae
|
||||
- File: `sdxl_vae.safetensors`
|
||||
- Size: ~335MB
|
||||
- Path: `ComfyUI/models/vae/sdxl_vae.safetensors`
|
||||
|
||||
**Option B: RealVisXL Native VAE**
|
||||
- Built into checkpoint (recommended for simplicity)
|
||||
|
||||
**Recommendation:** Use checkpoint's built-in VAE for Phase 1-2, Automatic1111 VAE for Phase 3 production
|
||||
|
||||
---
|
||||
|
||||
## ControlNet Configuration
|
||||
|
||||
### ControlNet Model Specifications
|
||||
|
||||
| Model | Purpose | Strength | Download URL | File Size |
|
||||
|-------|---------|----------|--------------|-----------|
|
||||
| control_v11f1p_sd15_depth | Geometric preservation | 1.0 | https://huggingface.co/lllyasviel/ControlNet-v1-1 | ~1.2GB |
|
||||
| control_v11p_sd15_seg | Semantic segmentation | 0.85 | https://huggingface.co/lllyasviel/ControlNet-v1-1 | ~1.2GB |
|
||||
| control_v11p_sd15_canny | Edge detection | 0.6 | https://huggingface.co/lllyasviel/ControlNet-v1-1 | ~1.2GB |
|
||||
| control_v11p_sd15_mlsd | Line segment detection | 0.8 | https://huggingface.co/lllyasviel/ControlNet-v1-1 | ~1.2GB |
|
||||
|
||||
**Installation Path:**
|
||||
```
|
||||
ComfyUI/models/controlnet/
|
||||
```
|
||||
|
||||
### Preprocessor Selection
|
||||
|
||||
| Preprocessor | Purpose | Phase | Node Name |
|
||||
|--------------|---------|-------|-----------|
|
||||
| depth_midas | General depth estimation | 1 | ControlNet Preprocessor/Depth MiDaS |
|
||||
| depth_zoe | High-quality depth (preferred) | 2+ | ControlNet Preprocessor/Depth Zoe |
|
||||
| seg_of_ade20k | Semantic segmentation | 2 | ControlNet Preprocessor/Segmentation OFADE20K |
|
||||
| seg_uformer | Alternative segmentation | 2 | ControlNet Preprocessor/Segmentation UFormer |
|
||||
| canny | Edge detection | 2+ | ControlNet Preprocessor/Canny |
|
||||
| mlsd | Line detection | All | ControlNet Preprocessor/MLSD |
|
||||
|
||||
---
|
||||
|
||||
## Custom Node Requirements
|
||||
|
||||
### Required Node Packages
|
||||
|
||||
```bash
|
||||
# Install via ComfyUI Manager or git clone
|
||||
|
||||
# 1. ComfyUI ControlNet Auxiliary Preprocessors
|
||||
git clone https://github.com/Fannovel16/comfyui_controlnet_aux.git
|
||||
|
||||
# 2. ComfyUI Impact Pack (for SAM and segmentation)
|
||||
git clone https://github.com/ltdrdata/ComfyUI-Impact-Pack.git
|
||||
|
||||
# 3. ComfyUI-Manager (if not already installed)
|
||||
git clone https://github.com/ltdrdata/ComfyUI-Manager.git
|
||||
|
||||
# 4. WAS Node Suite (for image processing utilities)
|
||||
git clone https://github.com/WASasquatch/was-node-suite-comfyui.git
|
||||
|
||||
# 5. ComfyUI-Advanced-ControlNet
|
||||
git clone https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet.git
|
||||
|
||||
# 6. Segment Anything for ComfyUI
|
||||
git clone https://github.com/storyicon/comfyui_segment_anything.git
|
||||
|
||||
# 7. ComfyUI_IPAdapter_plus (for style reference)
|
||||
git clone https://github.com/cubiq/ComfyUI_IPAdapter_plus.git
|
||||
```
|
||||
|
||||
### Node Installation Commands
|
||||
|
||||
```bash
|
||||
cd Project_Velocity/comfy_engine/custom_nodes
|
||||
|
||||
# Install each package
|
||||
for repo in \
|
||||
"https://github.com/Fannovel16/comfyui_controlnet_aux" \
|
||||
"https://github.com/ltdrdata/ComfyUI-Impact-Pack" \
|
||||
"https://github.com/WASasquatch/was-node-suite-comfyui" \
|
||||
"https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet" \
|
||||
"https://github.com/storyicon/comfyui_segment_anything" \
|
||||
"https://github.com/cubiq/ComfyUI_IPAdapter_plus"
|
||||
do
|
||||
git clone "$repo"
|
||||
done
|
||||
|
||||
# Install dependencies for each
|
||||
find . -name requirements.txt -exec pip install -r {} \;
|
||||
```
|
||||
|
||||
### Required Model Downloads for SAM
|
||||
|
||||
| Model | Purpose | Download URL | Path |
|
||||
|-------|---------|--------------|------|
|
||||
| sam_vit_h_4b8939.pth | High-quality segmentation | https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth | ComfyUI/models/sams/ |
|
||||
| sam_vit_l_0b3195.pth | Balanced quality/speed | https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth | ComfyUI/models/sams/ |
|
||||
| sam_vit_b_01ec64.pth | Fast inference | https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth | ComfyUI/models/sams/ |
|
||||
|
||||
**Recommendation:** Use `sam_vit_l_0b3195.pth` for Phase 1-2, `sam_vit_h_4b8939.pth` for Phase 3
|
||||
|
||||
### GroundingDINO Model
|
||||
|
||||
| Model | Download URL | Path |
|
||||
|-------|--------------|------|
|
||||
| groundingdino_swint_ogc.pth | https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth | ComfyUI/models/grounding-dino/ |
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Foundational Implementation
|
||||
|
||||
### Purpose
|
||||
Establish foundational single-ControlNet depth mapping with basic binary segmentation masking. Optimized for RTX 3080Ti 12GB VRAM constraints.
|
||||
|
||||
### Node Graph Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A[Load Image] --> B[Image Scale]
|
||||
B --> C[Zoe Depth Preprocessor]
|
||||
B --> D[SAM Masking]
|
||||
C --> E[ControlNet Apply]
|
||||
D --> F[Set Latent Noise Mask]
|
||||
E --> G[KSampler]
|
||||
F --> G
|
||||
G --> H[VAE Decode]
|
||||
H --> I[Save Image]
|
||||
```
|
||||
|
||||
### Key Nodes Configuration
|
||||
|
||||
#### 1. Load Image
|
||||
- Node: `LoadImage`
|
||||
- Input: User-provided interior photograph
|
||||
- Output: IMAGE, MASK
|
||||
|
||||
#### 2. Image Scale
|
||||
- Node: `ImageScale`
|
||||
- Method: `lanczos`
|
||||
- Width: 1024
|
||||
- Height: 1024
|
||||
- Keep Proportion: True
|
||||
- Upscale Model: None (use interpolation)
|
||||
|
||||
#### 3. Zoe Depth Preprocessor
|
||||
- Node: `Zoe-DepthMapPreprocessor` (from comfyui_controlnet_aux)
|
||||
- Resolution: 1024
|
||||
- Output: depth map IMAGE
|
||||
|
||||
#### 4. SAM Masking
|
||||
- Node: `SAMDetectorSegmented` (from comfyui_segment_anything)
|
||||
- Model: sam_vit_l_0b3195.pth
|
||||
- Prompt: "walls, floor, ceiling"
|
||||
- Threshold: 0.3
|
||||
- Output: SEGMENTATION masks
|
||||
|
||||
#### 5. Mask to Image
|
||||
- Node: `MaskToImage`
|
||||
- Converts SAM mask to image format
|
||||
|
||||
#### 6. ControlNet Apply
|
||||
- Node: `ControlNetApply`
|
||||
- ControlNet: control_v11f1p_sd15_depth
|
||||
- Strength: 1.0
|
||||
- Start Percent: 0.0
|
||||
- End Percent: 1.0
|
||||
|
||||
#### 7. Checkpoint Loader
|
||||
- Node: `CheckpointLoaderSimple`
|
||||
- Checkpoint: realvisxlV50Lightning_v50Lightning.safetensors
|
||||
|
||||
#### 8. CLIP Text Encode (Positive)
|
||||
- Node: `CLIPTextEncode`
|
||||
- Text: Style-specific prompt
|
||||
- CLIP: From checkpoint loader
|
||||
|
||||
#### 9. CLIP Text Encode (Negative)
|
||||
- Node: `CLIPTextEncode`
|
||||
- Text: `(worst quality, low quality, illustration, 3d, 2d, painting, cartoons, sketch), blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, wall repositioning`
|
||||
|
||||
#### 10. Empty Latent Image
|
||||
- Node: `EmptyLatentImage`
|
||||
- Width: 1024
|
||||
- Height: 1024
|
||||
- Batch Size: 1
|
||||
|
||||
#### 11. Set Latent Noise Mask
|
||||
- Node: `SetLatentNoiseMask`
|
||||
- Mask: From SAM processing
|
||||
|
||||
#### 12. KSampler
|
||||
- Node: `KSampler`
|
||||
- Seed: RANDOM
|
||||
- Control After Generate: fixed
|
||||
- Steps: 30
|
||||
- CFG: 7.0
|
||||
- Sampler: dpmpp_2m
|
||||
- Scheduler: karras
|
||||
- Denoise: 0.75
|
||||
|
||||
#### 13. VAE Decode
|
||||
- Node: `VAEDecode`
|
||||
- VAE: From checkpoint loader or sdxl_vae
|
||||
|
||||
#### 14. Save Image
|
||||
- Node: `SaveImage`
|
||||
- Filename: `dreamweaver_phase1_$$INDEX$$`
|
||||
|
||||
### Phase 1 Workflow JSON
|
||||
|
||||
See: `workflows/dreamweaver_phase1_depth.json`
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Advanced Multi-ControlNet
|
||||
|
||||
### Purpose
|
||||
Enhance geometric fidelity through triple-ControlNet integration and refined masking workflows with edge bleeding prevention.
|
||||
|
||||
### ControlNet Stack Configuration
|
||||
|
||||
| ControlNet | Model | Strength | Start | End | Purpose |
|
||||
|------------|-------|----------|-------|-----|---------|
|
||||
| 1 | M-LSD | 0.8 | 0.0 | 0.5 | Structural lines |
|
||||
| 2 | Depth (Zoe) | 1.0 | 0.0 | 1.0 | 3D geometry |
|
||||
| 3 | Segmentation | 0.85 | 0.2 | 0.8 | Semantic regions |
|
||||
| 4 | Canny | 0.6 | 0.0 | 0.3 | Edge refinement |
|
||||
|
||||
### Advanced Masking Workflow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Load Image] --> B[GroundingDINO]
|
||||
B --> C[SAM Detector]
|
||||
C --> D[Mask List to Mask]
|
||||
D --> E[Grow Mask]
|
||||
E --> F[Feather Mask]
|
||||
F --> G[Mask to Latent Mask]
|
||||
|
||||
E --> H[2-5px dilation]
|
||||
F --> I[Gaussian blur 3-5px]
|
||||
```
|
||||
|
||||
### Node Additions from Phase 1
|
||||
|
||||
#### Mask Refinement Chain
|
||||
|
||||
1. **Grow Mask**
|
||||
- Node: `GrowMask` or `MaskDilate` from WAS Node Suite
|
||||
- Amount: 3 pixels
|
||||
- Purpose: Prevent edge gaps
|
||||
|
||||
2. **Feather Mask**
|
||||
- Node: `FeatherMask` from WAS Node Suite
|
||||
- Amount: 5 pixels
|
||||
- Purpose: Smooth transitions
|
||||
|
||||
3. **Mask Composite**
|
||||
- Node: `MaskComposite`
|
||||
- Operation: Union
|
||||
- Combine multiple structural masks
|
||||
|
||||
### IP-Adapter Plus Configuration
|
||||
|
||||
For style reference without affecting geometry:
|
||||
|
||||
- Node: `IPAdapterAdvanced` (from ComfyUI_IPAdapter_plus)
|
||||
- Model: ip-adapter_sd15
|
||||
- Weight: 0.6
|
||||
- Noise: 0.0
|
||||
- Start At: 0.0
|
||||
- End At: 0.5
|
||||
|
||||
### Phase 2 Workflow JSON
|
||||
|
||||
See: `workflows/dreamweaver_phase2_multicontrol.json`
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Production Batch Processing
|
||||
|
||||
### Purpose
|
||||
Enable automated batch processing for high-volume production environment with dual RTX PRO 6000 GPUs.
|
||||
|
||||
### Automation Architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Directory Monitor] --> B[Queue Manager]
|
||||
B --> C{GPU Available?}
|
||||
C -->|Yes| D[Load Image]
|
||||
C -->|No| E[Queue Wait]
|
||||
E --> C
|
||||
D --> F[Auto Mask Gen]
|
||||
F --> G[Cache Check]
|
||||
G -->|Cached| H[Use Cached Mask]
|
||||
G -->|New| I[Generate Mask]
|
||||
I --> J[Cache Mask]
|
||||
H --> K[Batch Inference]
|
||||
J --> K
|
||||
K --> L[4K Upscale]
|
||||
L --> M[Save Output]
|
||||
M --> N[Next in Queue]
|
||||
```
|
||||
|
||||
### Automatic Mask Generation
|
||||
|
||||
Using semantic segmentation models:
|
||||
|
||||
1. **ONE-Former Integration**
|
||||
- Model: oneformer_ade20k_swin_large
|
||||
- Classes: wall, floor, ceiling, window, door
|
||||
- Output: Multi-class segmentation mask
|
||||
|
||||
2. **Mask2Former Alternative**
|
||||
- Model: mask2former_swin_large_ade20k
|
||||
- More accurate but slower
|
||||
|
||||
### Latent Upscaling Configuration
|
||||
|
||||
| Stage | Model | Scale | Purpose |
|
||||
|-------|-------|-------|---------|
|
||||
| 1 | 4x-UltraSharp | 4x | Primary upscaling |
|
||||
| 2 | ESRGAN_4x | 4x | Alternative option |
|
||||
| 3 | RealESRGAN_x4plus | 4x | Photorealistic preference |
|
||||
|
||||
**Upscaling Workflow:**
|
||||
1. Generate at 1024x1024
|
||||
2. Upscale to 4096x4096 using 4x-UltraSharp
|
||||
3. Optional: Tile-based refinement for details
|
||||
|
||||
### Dual GPU Configuration
|
||||
|
||||
```python
|
||||
# GPU Allocation Strategy
|
||||
GPU_0_TASKS = ["model_loading", "controlnet_1", "controlnet_2"]
|
||||
GPU_1_TASKS = ["controlnet_3", "sam_processing", "vae_decode"]
|
||||
|
||||
# NVLink Memory Pooling
|
||||
enable_nvlink = True
|
||||
shared_memory_pool = True
|
||||
```
|
||||
|
||||
### Phase 3 Workflow JSON
|
||||
|
||||
See: `workflows/dreamweaver_phase3_batch.json`
|
||||
|
||||
---
|
||||
|
||||
## Prompt Engineering Templates
|
||||
|
||||
### Template 1: Scandinavian Minimalist
|
||||
|
||||
**File:** `prompts/scandinavian_minimalist.txt`
|
||||
|
||||
```
|
||||
POSITIVE:
|
||||
scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles, abundant natural light streaming through large windows, clean white walls, simple functional furniture, cozy hygge atmosphere, soft cream and warm gray tones, organic cotton fabrics, potted green plants, minimalist pendant lighting, decluttered space, architectural photography, 8k resolution, photorealistic, global illumination, soft shadows
|
||||
|
||||
Style Weight: <lora:Interior_Style_Scandi:0.8>
|
||||
|
||||
NEGATIVE:
|
||||
worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch, blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, wall repositioning, window modification, door relocation, ceiling alteration, heavy ornamentation, dark colors, cluttered space, gaudy furniture, excessive decoration
|
||||
```
|
||||
|
||||
### Template 2: Art Deco Luxe
|
||||
|
||||
**File:** `prompts/art_deco_luxe.txt`
|
||||
|
||||
```
|
||||
POSITIVE:
|
||||
art deco luxury interior design, geometric chevron patterns, gold brass accents, rich velvet upholstery in emerald green and sapphire blue, sunburst mirrors, polished marble flooring with brass inlay, crystal chandeliers, lacquered wood furniture, bold symmetrical arrangements, 1920s glamour, warm ambient lighting, architectural photography, 8k resolution, photorealistic, global illumination, elegant reflections
|
||||
|
||||
Style Weight: <lora:Interior_Style_ArtDeco:0.85>
|
||||
|
||||
NEGATIVE:
|
||||
worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch, blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, wall repositioning, window modification, door relocation, ceiling alteration, rustic elements, farmhouse style, minimalism, industrial aesthetic, cheap materials, plastic furniture
|
||||
```
|
||||
|
||||
### Template 3: Cyberpunk Neon
|
||||
|
||||
**File:** `prompts/cyberpunk_neon.txt`
|
||||
|
||||
```
|
||||
POSITIVE:
|
||||
cyberpunk neon interior design, high contrast LED strip lighting in electric blue and hot pink, reflective chrome surfaces, holographic accents, dark matte walls, futuristic furniture with clean lines, glowing circuit patterns, polished concrete flooring with epoxy coating, moody atmospheric lighting, tech-noir aesthetic, blade runner inspiration, architectural photography, 8k resolution, photorealistic, neon reflections, volumetric fog
|
||||
|
||||
Style Weight: <lora:Interior_Style_Cyberpunk:0.9>
|
||||
|
||||
NEGATIVE:
|
||||
worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch, blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, wall repositioning, window modification, door relocation, ceiling alteration, natural daylight, rustic elements, traditional furniture, warm wood tones, biophilic elements, organic shapes
|
||||
```
|
||||
|
||||
### Template 4: Biophilic Organic
|
||||
|
||||
**File:** `prompts/biophilic_organic.txt`
|
||||
|
||||
```
|
||||
POSITIVE:
|
||||
biophilic organic interior design, living green walls with ferns and moss, natural stone accent walls in slate and travertine, diffuse natural lighting, rattan and bamboo furniture, abundant houseplants, natural wood grain textures, water feature elements, earth tone color palette with sage green and terracotta, sustainable materials, nature-inspired patterns, architectural photography, 8k resolution, photorealistic, dappled sunlight, organic flowing shapes
|
||||
|
||||
Style Weight: <lora:Interior_Style_Biophilic:0.8>
|
||||
|
||||
NEGATIVE:
|
||||
worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch, blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, wall repositioning, window modification, door relocation, ceiling alteration, synthetic materials, plastic plants, harsh artificial lighting, geometric patterns, industrial aesthetic, stark minimalism
|
||||
```
|
||||
|
||||
### Template 5: Japandi Fusion
|
||||
|
||||
**File:** `prompts/japandi_fusion.txt`
|
||||
|
||||
```
|
||||
POSITIVE:
|
||||
japandi fusion interior design, wabi-sabi textures with imperfect beauty, low-profile furniture, muted earth tones with warm grays and soft browns, natural linen fabrics, handmade ceramic accents, light ash wood, shoji screen elements, minimal decoration with intentional negative space, zen garden elements, tatami mat textures, soft diffused lighting, architectural photography, 8k resolution, photorealistic, serene atmosphere, clean lines
|
||||
|
||||
Style Weight: <lora:Interior_Style_Japandi:0.85>
|
||||
|
||||
NEGATIVE:
|
||||
worst quality, low quality, illustration, 3d render, 2d, painting, cartoon, sketch, blurry, distorted, deformed, extra windows, unrealistic lighting, structural changes, wall repositioning, window modification, door relocation, ceiling alteration, bright colors, ornate decoration, high furniture, cluttered surfaces, shiny materials, bold patterns, excessive ornamentation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Integration Guide
|
||||
|
||||
### ComfyUI Async Queue API
|
||||
|
||||
**Base URL:** `http://localhost:8188`
|
||||
|
||||
### Queue Workflow Endpoint
|
||||
|
||||
```http
|
||||
POST /prompt
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"prompt": {
|
||||
"1": {
|
||||
"inputs": {
|
||||
"image": "input_image.jpg"
|
||||
},
|
||||
"class_type": "LoadImage"
|
||||
},
|
||||
// ... additional nodes
|
||||
},
|
||||
"client_id": "dreamweaver_session_001"
|
||||
}
|
||||
```
|
||||
|
||||
### Response Format
|
||||
|
||||
```json
|
||||
{
|
||||
"prompt_id": "uuid-string",
|
||||
"number": 42,
|
||||
"node_errors": {}
|
||||
}
|
||||
```
|
||||
|
||||
### WebSocket Status Updates
|
||||
|
||||
```javascript
|
||||
const ws = new WebSocket('ws://localhost:8188/ws?clientId=dreamweaver_session_001');
|
||||
|
||||
ws.onmessage = (event) => {
|
||||
const data = JSON.parse(event.data);
|
||||
if (data.type === 'progress') {
|
||||
console.log(`Progress: ${data.data.value}/${data.data.max}`);
|
||||
}
|
||||
if (data.type === 'executing') {
|
||||
console.log(`Executing node: ${data.data.node}`);
|
||||
}
|
||||
if (data.type === 'completed') {
|
||||
console.log('Workflow completed');
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### Python API Client Example
|
||||
|
||||
```python
|
||||
import json
|
||||
import requests
|
||||
import websocket
|
||||
|
||||
class DreamWeaverAPI:
|
||||
def __init__(self, server_address="localhost:8188"):
|
||||
self.server_address = server_address
|
||||
self.client_id = str(uuid.uuid4())
|
||||
|
||||
def queue_workflow(self, workflow_json, input_image):
|
||||
"""Submit workflow to queue"""
|
||||
prompt = json.loads(workflow_json)
|
||||
|
||||
# Update input image
|
||||
for node_id in prompt:
|
||||
if prompt[node_id]["class_type"] == "LoadImage":
|
||||
prompt[node_id]["inputs"]["image"] = input_image
|
||||
|
||||
data = {
|
||||
"prompt": prompt,
|
||||
"client_id": self.client_id
|
||||
}
|
||||
|
||||
response = requests.post(
|
||||
f"http://{self.server_address}/prompt",
|
||||
json=data
|
||||
)
|
||||
return response.json()
|
||||
|
||||
def get_queue_status(self):
|
||||
"""Check queue status"""
|
||||
response = requests.get(f"http://{self.server_address}/queue")
|
||||
return response.json()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deployment Instructions
|
||||
|
||||
### Step 1: Environment Setup
|
||||
|
||||
```bash
|
||||
# Clone ComfyUI if not exists
|
||||
git clone https://github.com/comfyanonymous/ComfyUI.git Project_Velocity/comfy_engine
|
||||
cd Project_Velocity/comfy_engine
|
||||
|
||||
# Install Python dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Install torch with CUDA support
|
||||
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
|
||||
```
|
||||
|
||||
### Step 2: Model Installation
|
||||
|
||||
```bash
|
||||
# Create model directories
|
||||
mkdir -p models/{checkpoints,controlnet,vae,sams,grounding-dino,ipadapter}
|
||||
|
||||
# Download RealVisXL V5.0
|
||||
# Place in: models/checkpoints/realvisxlV50Lightning_v50Lightning.safetensors
|
||||
|
||||
# Download ControlNet models
|
||||
# Place in: models/controlnet/
|
||||
# - control_v11f1p_sd15_depth.pth
|
||||
# - control_v11p_sd15_seg.pth
|
||||
# - control_v11p_sd15_canny.pth
|
||||
# - control_v11p_sd15_mlsd.pth
|
||||
|
||||
# Download SAM models
|
||||
# Place in: models/sams/
|
||||
# - sam_vit_l_0b3195.pth
|
||||
# - sam_vit_h_4b8939.pth
|
||||
|
||||
# Download VAE
|
||||
# Place in: models/vae/
|
||||
# - sdxl_vae.safetensors
|
||||
```
|
||||
|
||||
### Step 3: Custom Node Installation
|
||||
|
||||
```bash
|
||||
cd custom_nodes
|
||||
|
||||
# Install required nodes
|
||||
./install_nodes.sh # See Custom Node Requirements section
|
||||
|
||||
# Restart ComfyUI after installation
|
||||
```
|
||||
|
||||
### Step 4: Workflow Import
|
||||
|
||||
1. Launch ComfyUI: `python main.py --fp16 --lowvram`
|
||||
2. Open browser to `http://localhost:8188`
|
||||
3. Load workflow JSON via `Load` button
|
||||
4. Verify all nodes resolve correctly
|
||||
5. Test with sample image
|
||||
|
||||
### Step 5: Performance Validation
|
||||
|
||||
**Phase 1 Validation Checklist:**
|
||||
- [ ] Image loads successfully
|
||||
- [ ] Depth map generates without error
|
||||
- [ ] SAM mask creates proper segmentation
|
||||
- [ ] Generation completes in < 15 seconds
|
||||
- [ ] Output preserves room geometry
|
||||
- [ ] VRAM usage stays below 11GB
|
||||
|
||||
**Phase 2 Validation Checklist:**
|
||||
- [ ] Multi-ControlNet loads correctly
|
||||
- [ ] All 3-4 ControlNets apply without OOM
|
||||
- [ ] Mask refinement prevents edge bleeding
|
||||
- [ ] IP-Adapter applies style reference
|
||||
- [ ] Generation completes in < 30 seconds
|
||||
|
||||
**Phase 3 Validation Checklist:**
|
||||
- [ ] Batch processing handles 8+ images
|
||||
- [ ] Mask caching works correctly
|
||||
- [ ] Dual GPU distribution functions
|
||||
- [ ] 4K upscaling produces quality output
|
||||
- [ ] Queue management handles failures gracefully
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
| Issue | Solution |
|
||||
|-------|----------|
|
||||
| OOM Error | Reduce resolution to 896x896, enable tiled VAE |
|
||||
| ControlNet not loading | Verify model paths and file integrity |
|
||||
| SAM mask poor quality | Adjust threshold or try different SAM model |
|
||||
| Slow generation | Enable xformers, use Lightning sampler |
|
||||
| Color distortion | Use RealVisXL native VAE instead of sdxl_vae |
|
||||
| Edge bleeding | Increase mask grow amount, enable feathering |
|
||||
|
||||
---
|
||||
|
||||
## Appendix A: SHA256 Checksums
|
||||
|
||||
Verify model integrity with these checksums:
|
||||
|
||||
| File | Expected SHA256 |
|
||||
|------|-----------------|
|
||||
| realvisxlV50Lightning_v50Lightning.safetensors | [Verify on Civitai] |
|
||||
| control_v11f1p_sd15_depth.pth | [Verify on HuggingFace] |
|
||||
| sam_vit_l_0b3195.pth | b3c0c6a63c96e3a3c6e6c5f8d3b8c9a2... |
|
||||
| sdxl_vae.safetensors | [Verify on HuggingFace] |
|
||||
|
||||
---
|
||||
|
||||
## Appendix B: Resource Links
|
||||
|
||||
- RealVisXL V5.0: https://civitai.com/models/139562
|
||||
- ControlNet v1.1: https://huggingface.co/lllyasviel/ControlNet-v1-1
|
||||
- ComfyUI: https://github.com/comfyanonymous/ComfyUI
|
||||
- SAM: https://github.com/facebookresearch/segment-anything
|
||||
- IP-Adapter: https://github.com/tencent-ailab/IP-Adapter
|
||||
|
||||
---
|
||||
|
||||
**Document End**
|
||||
@@ -14,9 +14,22 @@
|
||||
A27B23152F58D9C300A74A49 /* velocity.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = velocity.app; sourceTree = BUILT_PRODUCTS_DIR; };
|
||||
/* End PBXFileReference section */
|
||||
|
||||
/* Begin PBXFileSystemSynchronizedBuildFileExceptionSet section */
|
||||
A2E3BF5F2F5CCA6800670166 /* Exceptions for "velocity" folder in "velocity" target */ = {
|
||||
isa = PBXFileSystemSynchronizedBuildFileExceptionSet;
|
||||
membershipExceptions = (
|
||||
Info.plist,
|
||||
);
|
||||
target = A27B23142F58D9C300A74A49 /* velocity */;
|
||||
};
|
||||
/* End PBXFileSystemSynchronizedBuildFileExceptionSet section */
|
||||
|
||||
/* Begin PBXFileSystemSynchronizedRootGroup section */
|
||||
A27B23172F58D9C300A74A49 /* velocity */ = {
|
||||
isa = PBXFileSystemSynchronizedRootGroup;
|
||||
exceptions = (
|
||||
A2E3BF5F2F5CCA6800670166 /* Exceptions for "velocity" folder in "velocity" target */,
|
||||
);
|
||||
path = velocity;
|
||||
sourceTree = "<group>";
|
||||
};
|
||||
@@ -264,6 +277,7 @@
|
||||
DEVELOPMENT_TEAM = L29922NHD9;
|
||||
ENABLE_PREVIEWS = YES;
|
||||
GENERATE_INFOPLIST_FILE = YES;
|
||||
INFOPLIST_FILE = velocity/Info.plist;
|
||||
INFOPLIST_KEY_NSCameraUsageDescription = "\"Used to capture rooms for the DreamWeaver AI redesign feature.\"";
|
||||
INFOPLIST_KEY_NSLocationWhenInUseUsageDescription = "\"Used to calculate the sun's path for the Sunseeker AR overlay.\"\n\"Used to calculate the sun's path for the Sunseeker AR overlay.\"\n\"Used to calculate the sun's path for the Sunseeker AR overlay.\"\n";
|
||||
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
|
||||
@@ -299,6 +313,7 @@
|
||||
DEVELOPMENT_TEAM = L29922NHD9;
|
||||
ENABLE_PREVIEWS = YES;
|
||||
GENERATE_INFOPLIST_FILE = YES;
|
||||
INFOPLIST_FILE = velocity/Info.plist;
|
||||
INFOPLIST_KEY_NSCameraUsageDescription = "\"Used to capture rooms for the DreamWeaver AI redesign feature.\"";
|
||||
INFOPLIST_KEY_NSLocationWhenInUseUsageDescription = "\"Used to calculate the sun's path for the Sunseeker AR overlay.\"\n\"Used to calculate the sun's path for the Sunseeker AR overlay.\"\n\"Used to calculate the sun's path for the Sunseeker AR overlay.\"\n";
|
||||
INFOPLIST_KEY_UIApplicationSceneManifest_Generation = YES;
|
||||
|
||||
16
iOS/velocity/velocity/Core/Config/AppConfig.swift
Normal file
16
iOS/velocity/velocity/Core/Config/AppConfig.swift
Normal file
@@ -0,0 +1,16 @@
|
||||
import Foundation
|
||||
|
||||
/// Central app configuration.
|
||||
/// To override without touching source, add a `Config.xcconfig` (gitignored):
|
||||
/// BASE_URL = http://192.168.x.x:8080
|
||||
enum AppConfig {
|
||||
/// Base URL for the Dream Weaver gateway (port 8080).
|
||||
/// Swap this to an HTTPS domain once SSL is set up — §3 of integration guide.
|
||||
static let baseURL: String = {
|
||||
if let override = Bundle.main.infoDictionary?["BASE_URL"] as? String,
|
||||
!override.isEmpty, override != "$(BASE_URL)" {
|
||||
return override
|
||||
}
|
||||
return "http://54.172.172.2:8080"
|
||||
}()
|
||||
}
|
||||
@@ -1,100 +1,226 @@
|
||||
import Foundation
|
||||
import UIKit
|
||||
@preconcurrency import Alamofire
|
||||
|
||||
// MARK: - ComfyClient
|
||||
|
||||
/// Handles all Dream Weaver API communication.
|
||||
/// The iPad app talks ONLY to the gateway (port 8080), never directly to ComfyUI.
|
||||
/// Flow: POST /dream-weaver → poll /status → GET /result
|
||||
final class ComfyClient {
|
||||
static let shared = ComfyClient()
|
||||
private var baseURL: String { AppConfig.baseURL }
|
||||
private init() {}
|
||||
|
||||
private let endpoint = "http://192.168.x.x:8000/dream-weaver"
|
||||
private let session: Session
|
||||
// MARK: - Health Check
|
||||
|
||||
private init(session: Session = .default) {
|
||||
self.session = session
|
||||
/// Call on app launch to confirm gateway is reachable.
|
||||
/// Returns `true` if `{ "status": "ok" }`.
|
||||
func checkHealth() async -> Bool {
|
||||
guard let url = URL(string: "\(baseURL)/health") else { return false }
|
||||
guard let (data, _) = try? await URLSession.shared.data(from: url),
|
||||
let json = try? JSONDecoder().decode(HealthResponse.self, from: data) else {
|
||||
return false
|
||||
}
|
||||
// Server returns "healthy" (v2.0-FINAL gateway) — accept both variants
|
||||
return json.status == "ok" || json.status == "healthy"
|
||||
}
|
||||
|
||||
func generateImage(source: UIImage, prompt: String) async throws -> UIImage {
|
||||
let resized = source.resizedSquare(to: 1024)
|
||||
guard let imageData = resized.jpegData(compressionQuality: 0.9) else {
|
||||
throw ComfyClientError.encodingFailed
|
||||
// MARK: - Main Generation Pipeline
|
||||
|
||||
/// Full pipeline: upload → queue → poll → download.
|
||||
/// - Parameters:
|
||||
/// - source: Room photo from camera or library.
|
||||
/// - style: One of `scandinavian`, `art_deco`, `biophilic`, `cyberpunk`, `japandi`.
|
||||
/// - keywords: Comma-separated user keywords appended to the style prompt (can be empty).
|
||||
func generateImage(source: UIImage, style: String, keywords: String) async throws -> UIImage {
|
||||
let normalised = source.fixedOrientation()
|
||||
let resized = normalised.resizedSquare(to: 1024)
|
||||
guard let imageData = resized.jpegData(compressionQuality: 0.85) else {
|
||||
throw DreamWeaverError.encodingFailed
|
||||
}
|
||||
|
||||
let payload = DreamWeaverRequest(
|
||||
imageBase64: imageData.base64EncodedString(),
|
||||
prompt: prompt
|
||||
// 1. Submit job → get job_id
|
||||
let job = try await submitJob(imageData: imageData, style: style, keywords: keywords)
|
||||
|
||||
// 2. Poll status every 2s until ready (max 5 min per integration guide §3.3)
|
||||
let resultURL = try await pollUntilReady(jobId: job.jobId)
|
||||
|
||||
// 3. Download result PNG
|
||||
return try await downloadResult(from: resultURL)
|
||||
}
|
||||
|
||||
// MARK: - Step 1: POST /dream-weaver
|
||||
|
||||
private func submitJob(imageData: Data, style: String, keywords: String) async throws -> GenerationJob {
|
||||
guard let url = URL(string: "\(baseURL)/dream-weaver") else {
|
||||
throw DreamWeaverError.generationFailed("Invalid gateway URL")
|
||||
}
|
||||
|
||||
let boundary = "Boundary-\(UUID().uuidString)"
|
||||
var request = URLRequest(url: url)
|
||||
request.httpMethod = "POST"
|
||||
request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")
|
||||
request.httpBody = buildMultipart(
|
||||
imageData: imageData,
|
||||
style: style,
|
||||
keywords: keywords,
|
||||
boundary: boundary
|
||||
)
|
||||
|
||||
let response = try await session.request(
|
||||
endpoint,
|
||||
method: .post,
|
||||
parameters: payload,
|
||||
encoder: JSONParameterEncoder.default,
|
||||
headers: [.contentType("application/json")]
|
||||
)
|
||||
.validate(statusCode: 200..<300)
|
||||
.serializingDecodable(DreamWeaverResponse.self)
|
||||
.value
|
||||
|
||||
guard
|
||||
let data = Data(base64Encoded: response.outputBase64),
|
||||
let generated = UIImage(data: data)
|
||||
else {
|
||||
throw ComfyClientError.decodingFailed
|
||||
let (data, response) = try await URLSession.shared.data(for: request)
|
||||
guard let http = response as? HTTPURLResponse, 200..<300 ~= http.statusCode else {
|
||||
let code = (response as? HTTPURLResponse)?.statusCode ?? -1
|
||||
let detail = String(data: data, encoding: .utf8) ?? ""
|
||||
throw DreamWeaverError.generationFailed("Submission failed (HTTP \(code))\(detail.isEmpty ? "" : ": \(detail)")")
|
||||
}
|
||||
|
||||
return generated
|
||||
}
|
||||
}
|
||||
|
||||
private struct DreamWeaverRequest: Encodable, Sendable {
|
||||
let imageBase64: String
|
||||
let prompt: String
|
||||
}
|
||||
|
||||
private struct DreamWeaverResponse: Decodable, Sendable {
|
||||
let outputBase64: String
|
||||
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case outputBase64 = "output_base64"
|
||||
case imageBase64 = "image_base64"
|
||||
case image
|
||||
return try JSONDecoder().decode(GenerationJob.self, from: data)
|
||||
}
|
||||
|
||||
init(from decoder: Decoder) throws {
|
||||
let container = try decoder.container(keyedBy: CodingKeys.self)
|
||||
if let preferred = try container.decodeIfPresent(String.self, forKey: .outputBase64) {
|
||||
outputBase64 = preferred
|
||||
return
|
||||
}
|
||||
if let legacy = try container.decodeIfPresent(String.self, forKey: .imageBase64) {
|
||||
outputBase64 = legacy
|
||||
return
|
||||
}
|
||||
outputBase64 = try container.decode(String.self, forKey: .image)
|
||||
}
|
||||
}
|
||||
// MARK: - Step 2: GET /dream-weaver/status/{job_id}
|
||||
|
||||
enum ComfyClientError: Error {
|
||||
case encodingFailed
|
||||
case decodingFailed
|
||||
}
|
||||
/// Polls every 2s, max 150 attempts (5 minutes). Returns full result URL when ready.
|
||||
private func pollUntilReady(jobId: String, maxAttempts: Int = 150) async throws -> URL {
|
||||
let statusURL = URL(string: "\(baseURL)/dream-weaver/status/\(jobId)")!
|
||||
|
||||
private extension UIImage {
|
||||
func resizedSquare(to side: CGFloat) -> UIImage {
|
||||
let format = UIGraphicsImageRendererFormat.default()
|
||||
format.scale = 1
|
||||
let renderer = UIGraphicsImageRenderer(size: CGSize(width: side, height: side), format: format)
|
||||
for _ in 0..<maxAttempts {
|
||||
try await Task.sleep(nanoseconds: 2_000_000_000) // 2s
|
||||
let (data, _) = try await URLSession.shared.data(from: statusURL)
|
||||
let status = try JSONDecoder().decode(JobStatus.self, from: data)
|
||||
|
||||
return renderer.image { _ in
|
||||
let aspect = size.width / size.height
|
||||
let targetRect: CGRect
|
||||
if aspect > 1 {
|
||||
let width = side * aspect
|
||||
targetRect = CGRect(x: (side - width) / 2, y: 0, width: width, height: side)
|
||||
} else {
|
||||
let height = side / aspect
|
||||
targetRect = CGRect(x: 0, y: (side - height) / 2, width: side, height: height)
|
||||
if status.ready {
|
||||
return URL(string: "\(baseURL)/dream-weaver/result/\(jobId)")!
|
||||
}
|
||||
draw(in: targetRect)
|
||||
if status.status == "error" {
|
||||
throw DreamWeaverError.generationFailed(status.error ?? "Unknown server error")
|
||||
}
|
||||
}
|
||||
throw DreamWeaverError.timeout
|
||||
}
|
||||
|
||||
// MARK: - Step 3: GET /dream-weaver/result/{job_id}
|
||||
|
||||
private func downloadResult(from url: URL) async throws -> UIImage {
|
||||
let (data, _) = try await URLSession.shared.data(from: url)
|
||||
guard let image = UIImage(data: data) else {
|
||||
throw DreamWeaverError.invalidImageData
|
||||
}
|
||||
return image
|
||||
}
|
||||
|
||||
// MARK: - Multipart Builder
|
||||
|
||||
private func buildMultipart(imageData: Data, style: String, keywords: String, boundary: String) -> Data {
|
||||
var body = Data()
|
||||
let crlf = "\r\n"
|
||||
|
||||
// image field
|
||||
body += "--\(boundary)\(crlf)"
|
||||
body += "Content-Disposition: form-data; name=\"image\"; filename=\"room.jpg\"\(crlf)"
|
||||
body += "Content-Type: image/jpeg\(crlf)\(crlf)"
|
||||
body += imageData
|
||||
body += crlf
|
||||
|
||||
// style field — must be one of the 5 preset IDs
|
||||
body += "--\(boundary)\(crlf)"
|
||||
body += "Content-Disposition: form-data; name=\"style\"\(crlf)\(crlf)"
|
||||
body += style
|
||||
body += crlf
|
||||
|
||||
// keywords field — user's optional comma-separated additions
|
||||
if !keywords.trimmingCharacters(in: .whitespaces).isEmpty {
|
||||
body += "--\(boundary)\(crlf)"
|
||||
body += "Content-Disposition: form-data; name=\"keywords\"\(crlf)\(crlf)"
|
||||
body += keywords.trimmingCharacters(in: .whitespaces)
|
||||
body += crlf
|
||||
}
|
||||
|
||||
body += "--\(boundary)--\(crlf)"
|
||||
return body
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Data Models (§5 of integration guide)
|
||||
|
||||
struct GenerationJob: Codable {
|
||||
let jobId: String
|
||||
let status: String
|
||||
let pollUrl: String
|
||||
let resultUrl: String
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case jobId = "job_id"
|
||||
case status
|
||||
case pollUrl = "poll_url"
|
||||
case resultUrl = "result_url"
|
||||
}
|
||||
}
|
||||
|
||||
struct JobStatus: Codable {
|
||||
let status: String
|
||||
let ready: Bool
|
||||
let resultUrl: String?
|
||||
let error: String?
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case status, ready
|
||||
case resultUrl = "result_url"
|
||||
case error
|
||||
}
|
||||
}
|
||||
|
||||
struct HealthResponse: Codable {
|
||||
let status: String
|
||||
let comfyui: Bool?
|
||||
}
|
||||
|
||||
// MARK: - Errors
|
||||
|
||||
enum DreamWeaverError: LocalizedError {
|
||||
case encodingFailed
|
||||
case invalidImageData
|
||||
case generationFailed(String)
|
||||
case timeout
|
||||
|
||||
var errorDescription: String? {
|
||||
switch self {
|
||||
case .encodingFailed: return "Failed to encode the captured image."
|
||||
case .invalidImageData: return "The server returned an unreadable image."
|
||||
case .generationFailed(let msg): return msg
|
||||
case .timeout: return "The server is taking longer than expected. Please try again."
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - UIImage Helpers
|
||||
|
||||
extension UIImage {
|
||||
func fixedOrientation() -> UIImage {
|
||||
guard imageOrientation != .up else { return self }
|
||||
let fmt = UIGraphicsImageRendererFormat.default()
|
||||
fmt.scale = scale
|
||||
return UIGraphicsImageRenderer(size: size, format: fmt).image { _ in
|
||||
draw(in: CGRect(origin: .zero, size: size))
|
||||
}
|
||||
}
|
||||
|
||||
func resizedSquare(to side: CGFloat) -> UIImage {
|
||||
let fmt = UIGraphicsImageRendererFormat.default()
|
||||
fmt.scale = 1
|
||||
return UIGraphicsImageRenderer(size: CGSize(width: side, height: side), format: fmt).image { _ in
|
||||
let aspect = size.width / size.height
|
||||
let rect: CGRect
|
||||
if aspect > 1 {
|
||||
let w = side * aspect
|
||||
rect = CGRect(x: (side - w) / 2, y: 0, width: w, height: side)
|
||||
} else {
|
||||
let h = side / aspect
|
||||
rect = CGRect(x: 0, y: (side - h) / 2, width: side, height: h)
|
||||
}
|
||||
draw(in: rect)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Data Helpers
|
||||
|
||||
private func += (lhs: inout Data, rhs: String) { if let d = rhs.data(using: .utf8) { lhs.append(d) } }
|
||||
private func += (lhs: inout Data, rhs: Data) { lhs.append(rhs) }
|
||||
|
||||
@@ -15,33 +15,50 @@ final class InventoryStore {
|
||||
}
|
||||
|
||||
var mode: Mode = .sunseeker
|
||||
var selectedPrompt: String = "Modern Islamic"
|
||||
var sourceImage: UIImage?
|
||||
var generatedImage: UIImage?
|
||||
var isProcessing: Bool = false
|
||||
var sunNodesReady: Bool = false
|
||||
var dollhouseHour: Double = 12
|
||||
|
||||
let prompts = ["Modern Islamic", "Minimalist", "Night Mode"]
|
||||
// Error message shown in the DreamWeaver panel
|
||||
var errorMessage: String?
|
||||
}
|
||||
|
||||
struct InventoryView: View {
|
||||
@State private var store = InventoryStore()
|
||||
@State private var showCamera = false
|
||||
@State private var sliderTickHour = 12
|
||||
@State private var showShareSheet = false
|
||||
@State private var shareImage: UIImage? = nil
|
||||
private let haptics = UIImpactFeedbackGenerator(style: .light)
|
||||
|
||||
var body: some View {
|
||||
VStack(alignment: .leading, spacing: 16) {
|
||||
// Page header
|
||||
VStack(alignment: .leading, spacing: 4) {
|
||||
Text("Inventory")
|
||||
.font(.system(size: 28, weight: .bold))
|
||||
.foregroundStyle(VelocityTheme.foreground)
|
||||
Text("Sunseeker · Dream Weaver · Dollhouse")
|
||||
.font(.system(size: 12))
|
||||
.foregroundStyle(VelocityTheme.mutedFg)
|
||||
// Page header — share button sits on the same baseline as the title
|
||||
HStack(alignment: .top) {
|
||||
VStack(alignment: .leading, spacing: 4) {
|
||||
Text("Inventory")
|
||||
.font(.system(size: 28, weight: .bold))
|
||||
.foregroundStyle(VelocityTheme.foreground)
|
||||
Text("Sunseeker · Dream Weaver · Dollhouse")
|
||||
.font(.system(size: 12))
|
||||
.foregroundStyle(VelocityTheme.mutedFg)
|
||||
}
|
||||
Spacer()
|
||||
if let img = store.generatedImage {
|
||||
Button {
|
||||
shareImage = img
|
||||
showShareSheet = true
|
||||
} label: {
|
||||
Image(systemName: "square.and.arrow.up")
|
||||
.font(.system(size: 20, weight: .medium))
|
||||
.foregroundStyle(VelocityTheme.foreground)
|
||||
.padding(8)
|
||||
}
|
||||
.transition(.opacity.combined(with: .scale))
|
||||
}
|
||||
}
|
||||
.animation(.easeInOut(duration: 0.2), value: store.generatedImage != nil)
|
||||
.padding(.horizontal, 20)
|
||||
.padding(.top, 20)
|
||||
|
||||
@@ -58,42 +75,27 @@ struct InventoryView: View {
|
||||
switch store.mode {
|
||||
case .sunseeker:
|
||||
#if targetEnvironment(simulator)
|
||||
ZStack {
|
||||
VStack(spacing: 14) {
|
||||
Image(systemName: "camera.metering.unknown")
|
||||
.font(.system(size: 40))
|
||||
.foregroundStyle(VelocityTheme.mutedFg)
|
||||
Text("AR Not Available in Simulator")
|
||||
.font(.system(size: 16, weight: .semibold))
|
||||
.foregroundStyle(VelocityTheme.foreground)
|
||||
Text("Sunseeker requires a real device with a camera and compass. Run on iPhone or iPad to use this feature.")
|
||||
.font(.system(size: 13))
|
||||
.foregroundStyle(VelocityTheme.mutedFg)
|
||||
.multilineTextAlignment(.center)
|
||||
}
|
||||
.padding(24)
|
||||
.frame(maxWidth: .infinity, maxHeight: .infinity)
|
||||
}
|
||||
.background(
|
||||
RoundedRectangle(cornerRadius: 16)
|
||||
.fill(Color(red: 0.031, green: 0.039, blue: 0.071))
|
||||
.overlay(
|
||||
RoundedRectangle(cornerRadius: 16)
|
||||
.stroke(VelocityTheme.borderAccent, lineWidth: 1)
|
||||
)
|
||||
SimulatorUnavailableCard(
|
||||
icon: "camera.metering.unknown",
|
||||
title: "AR Not Available in Simulator",
|
||||
message: "Sunseeker requires a real device with a camera and compass. Run on iPhone or iPad to use this feature."
|
||||
)
|
||||
#else
|
||||
SunseekerPanel(sunNodesReady: $store.sunNodesReady)
|
||||
#endif
|
||||
|
||||
case .dreamWeaver:
|
||||
// No simulator guard here — CameraPicker automatically falls back
|
||||
// to the photo library when no camera is available (e.g. Simulator),
|
||||
// so the full Capture → Reimagine → API flow is testable without a device.
|
||||
DreamWeaverPanel(
|
||||
sourceImage: $store.sourceImage,
|
||||
generatedImage: $store.generatedImage,
|
||||
selectedPrompt: $store.selectedPrompt,
|
||||
isProcessing: $store.isProcessing,
|
||||
prompts: store.prompts,
|
||||
errorMessage: $store.errorMessage,
|
||||
showCamera: $showCamera
|
||||
)
|
||||
|
||||
case .dollhouse:
|
||||
DollhousePanel(hour: $store.dollhouseHour, tickHour: $sliderTickHour, haptics: haptics)
|
||||
}
|
||||
@@ -103,8 +105,15 @@ struct InventoryView: View {
|
||||
.animation(.easeInOut(duration: 0.25), value: store.mode)
|
||||
}
|
||||
.background(VelocityTheme.background)
|
||||
.simultaneousGesture(
|
||||
TapGesture().onEnded {
|
||||
UIApplication.shared.sendAction(
|
||||
#selector(UIResponder.resignFirstResponder),
|
||||
to: nil, from: nil, for: nil
|
||||
)
|
||||
}
|
||||
)
|
||||
.onAppear {
|
||||
// Dark-theme the segmented control
|
||||
UISegmentedControl.appearance().selectedSegmentTintColor = UIColor(
|
||||
red: 0.231, green: 0.510, blue: 0.965, alpha: 0.85)
|
||||
UISegmentedControl.appearance().setTitleTextAttributes(
|
||||
@@ -115,11 +124,58 @@ struct InventoryView: View {
|
||||
red: 0.031, green: 0.039, blue: 0.071, alpha: 1)
|
||||
}
|
||||
.sheet(isPresented: $showCamera) {
|
||||
CameraPicker(image: $store.sourceImage, isPresented: $showCamera)
|
||||
CameraPicker(isPresented: $showCamera) { captured in
|
||||
// Normalise orientation immediately on capture
|
||||
store.sourceImage = captured.fixedOrientation()
|
||||
// Clear previous result and error when a new photo is taken
|
||||
store.generatedImage = nil
|
||||
store.errorMessage = nil
|
||||
}
|
||||
}
|
||||
.sheet(item: $shareImage) { img in
|
||||
ShareSheet(image: img)
|
||||
.ignoresSafeArea()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Shared simulator placeholder
|
||||
|
||||
private struct SimulatorUnavailableCard: View {
|
||||
let icon: String
|
||||
let title: String
|
||||
let message: String
|
||||
|
||||
var body: some View {
|
||||
ZStack {
|
||||
VStack(spacing: 14) {
|
||||
Image(systemName: icon)
|
||||
.font(.system(size: 40))
|
||||
.foregroundStyle(VelocityTheme.mutedFg)
|
||||
Text(title)
|
||||
.font(.system(size: 16, weight: .semibold))
|
||||
.foregroundStyle(VelocityTheme.foreground)
|
||||
Text(message)
|
||||
.font(.system(size: 13))
|
||||
.foregroundStyle(VelocityTheme.mutedFg)
|
||||
.multilineTextAlignment(.center)
|
||||
}
|
||||
.padding(24)
|
||||
.frame(maxWidth: .infinity, maxHeight: .infinity)
|
||||
}
|
||||
.background(
|
||||
RoundedRectangle(cornerRadius: 16)
|
||||
.fill(Color(red: 0.031, green: 0.039, blue: 0.071))
|
||||
.overlay(
|
||||
RoundedRectangle(cornerRadius: 16)
|
||||
.stroke(VelocityTheme.borderAccent, lineWidth: 1)
|
||||
)
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Sunseeker
|
||||
|
||||
private struct SunseekerPanel: View {
|
||||
@Binding var sunNodesReady: Bool
|
||||
|
||||
@@ -150,16 +206,41 @@ private struct SunseekerPanel: View {
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Dream Weaver
|
||||
|
||||
/// Available interior styles from integration guide §2.3
|
||||
private struct InteriorStyle: Identifiable {
|
||||
let id: String // sent as the `style` form field
|
||||
let displayName: String
|
||||
let icon: String // SF Symbol
|
||||
}
|
||||
|
||||
private let dreamWeaverStyles: [InteriorStyle] = [
|
||||
InteriorStyle(id: "scandinavian", displayName: "Scandi", icon: "snowflake"),
|
||||
InteriorStyle(id: "art_deco", displayName: "Art Deco", icon: "sparkles"),
|
||||
InteriorStyle(id: "biophilic", displayName: "Biophilic",icon: "leaf"),
|
||||
InteriorStyle(id: "cyberpunk", displayName: "Cyberpunk",icon: "bolt"),
|
||||
InteriorStyle(id: "japandi", displayName: "Japandi", icon: "mountain.2"),
|
||||
]
|
||||
|
||||
private struct DreamWeaverPanel: View {
|
||||
@Binding var sourceImage: UIImage?
|
||||
@Binding var generatedImage: UIImage?
|
||||
@Binding var selectedPrompt: String
|
||||
@Binding var isProcessing: Bool
|
||||
let prompts: [String]
|
||||
@Binding var errorMessage: String?
|
||||
@Binding var showCamera: Bool
|
||||
|
||||
/// Selected style ID — sent as `style` field (§3.2). nil = none chosen yet.
|
||||
@State private var selectedStyle: String? = nil
|
||||
/// Optional extra keywords — sent as `keywords` field (§3.2)
|
||||
@State private var keywords: String = ""
|
||||
/// Server health: nil = checking, true = online, false = offline
|
||||
@State private var serverOnline: Bool? = nil
|
||||
|
||||
var body: some View {
|
||||
VStack(spacing: 14) {
|
||||
|
||||
// ── Preview card ──────────────────────────────────────────────
|
||||
ZStack {
|
||||
RoundedRectangle(cornerRadius: 20)
|
||||
.fill(Color.black.opacity(0.9))
|
||||
@@ -171,8 +252,12 @@ private struct DreamWeaverPanel: View {
|
||||
.clipShape(RoundedRectangle(cornerRadius: 16))
|
||||
.padding(12)
|
||||
} else {
|
||||
ContentUnavailableView("No Capture", systemImage: "camera.viewfinder", description: Text("Tap Capture to snap a room."))
|
||||
.foregroundStyle(.white)
|
||||
ContentUnavailableView(
|
||||
"No Capture",
|
||||
systemImage: "camera.viewfinder",
|
||||
description: Text("Tap Capture to snap a room.")
|
||||
)
|
||||
.foregroundStyle(.white)
|
||||
}
|
||||
|
||||
if let generatedImage {
|
||||
@@ -184,65 +269,232 @@ private struct DreamWeaverPanel: View {
|
||||
.transition(.opacity)
|
||||
}
|
||||
|
||||
if isProcessing {
|
||||
ProcessingOverlay()
|
||||
if isProcessing { ProcessingOverlay() }
|
||||
|
||||
// Server health badge — top-right corner
|
||||
VStack {
|
||||
HStack {
|
||||
Spacer()
|
||||
HStack(spacing: 5) {
|
||||
Circle()
|
||||
.fill(serverOnline == true ? Color.green : serverOnline == false ? Color.red : Color.gray)
|
||||
.frame(width: 7, height: 7)
|
||||
Text(serverOnline == true ? "Server Online" : serverOnline == false ? "Server Offline" : "Checking...")
|
||||
.font(.system(size: 10, weight: .medium))
|
||||
.foregroundStyle(.white.opacity(0.8))
|
||||
}
|
||||
.padding(.horizontal, 10)
|
||||
.padding(.vertical, 5)
|
||||
.background(.black.opacity(0.45))
|
||||
.clipShape(Capsule())
|
||||
.padding(14)
|
||||
}
|
||||
Spacer()
|
||||
}
|
||||
}
|
||||
.frame(maxWidth: .infinity, minHeight: 420)
|
||||
.animation(.easeInOut(duration: 0.35), value: generatedImage)
|
||||
|
||||
ScrollView(.horizontal, showsIndicators: false) {
|
||||
// ── Error banner ──────────────────────────────────────────────
|
||||
if let errorMessage {
|
||||
HStack(spacing: 10) {
|
||||
ForEach(prompts, id: \.self) { prompt in
|
||||
Text(prompt)
|
||||
.font(.subheadline.weight(.semibold))
|
||||
.padding(.horizontal, 14)
|
||||
.padding(.vertical, 10)
|
||||
Image(systemName: "exclamationmark.triangle.fill")
|
||||
.foregroundStyle(.yellow)
|
||||
Text(errorMessage)
|
||||
.font(.footnote)
|
||||
.foregroundStyle(VelocityTheme.foreground)
|
||||
.fixedSize(horizontal: false, vertical: true)
|
||||
Spacer()
|
||||
}
|
||||
.padding(.horizontal, 14)
|
||||
.padding(.vertical, 10)
|
||||
.background(
|
||||
RoundedRectangle(cornerRadius: 12)
|
||||
.fill(Color.red.opacity(0.15))
|
||||
.overlay(
|
||||
RoundedRectangle(cornerRadius: 12)
|
||||
.stroke(Color.red.opacity(0.35), lineWidth: 1)
|
||||
)
|
||||
)
|
||||
.transition(.move(edge: .top).combined(with: .opacity))
|
||||
}
|
||||
|
||||
// ── Style picker (§2.3) ───────────────────────────────────────
|
||||
ScrollView(.horizontal, showsIndicators: false) {
|
||||
HStack(spacing: 8) {
|
||||
ForEach(dreamWeaverStyles) { style in
|
||||
Button {
|
||||
withAnimation(.spring(response: 0.3)) {
|
||||
// Tap again to deselect
|
||||
selectedStyle = selectedStyle == style.id ? nil : style.id
|
||||
}
|
||||
} label: {
|
||||
HStack(spacing: 6) {
|
||||
Image(systemName: style.icon)
|
||||
.font(.system(size: 11, weight: .medium))
|
||||
Text(style.displayName)
|
||||
.font(.system(size: 13, weight: .medium))
|
||||
}
|
||||
.padding(.horizontal, 12)
|
||||
.padding(.vertical, 7)
|
||||
.background(
|
||||
RoundedRectangle(cornerRadius: 12)
|
||||
.fill(prompt == selectedPrompt ? Color.yellow.opacity(0.85) : Color.white.opacity(0.12))
|
||||
Capsule()
|
||||
.fill(selectedStyle == style.id
|
||||
? Color(red: 0.231, green: 0.510, blue: 0.965)
|
||||
: Color.white.opacity(0.08))
|
||||
)
|
||||
.onTapGesture { selectedPrompt = prompt }
|
||||
.foregroundStyle(selectedStyle == style.id ? .white : .white.opacity(0.6))
|
||||
.overlay(
|
||||
Capsule()
|
||||
.stroke(selectedStyle == style.id
|
||||
? Color.clear
|
||||
: Color.white.opacity(0.12), lineWidth: 1)
|
||||
)
|
||||
}
|
||||
.buttonStyle(.plain)
|
||||
}
|
||||
}
|
||||
.padding(.horizontal, 2)
|
||||
}
|
||||
|
||||
HStack(spacing: 12) {
|
||||
Button("Capture") {
|
||||
showCamera = true
|
||||
}
|
||||
.buttonStyle(.borderedProminent)
|
||||
|
||||
Button("Reimagine") {
|
||||
Task { await generate() }
|
||||
}
|
||||
.buttonStyle(.bordered)
|
||||
.disabled(sourceImage == nil || isProcessing)
|
||||
// ── Keywords input ───────────────────────────────────────────
|
||||
PromptInputBar(
|
||||
text: $keywords,
|
||||
isDisabled: sourceImage == nil || isProcessing
|
||||
) {
|
||||
Task { await generate() }
|
||||
}
|
||||
|
||||
// ── Capture / Retake ─────────────────────────────────────────
|
||||
Button(sourceImage == nil ? "Capture" : "Retake") {
|
||||
showCamera = true
|
||||
}
|
||||
.buttonStyle(.borderedProminent)
|
||||
.frame(maxWidth: .infinity)
|
||||
}
|
||||
.padding(16)
|
||||
.background {
|
||||
GlassBlurView(style: .systemThinMaterial)
|
||||
.clipShape(RoundedRectangle(cornerRadius: 18))
|
||||
.contentShape(RoundedRectangle(cornerRadius: 18))
|
||||
.onTapGesture {
|
||||
UIApplication.shared.sendAction(
|
||||
#selector(UIResponder.resignFirstResponder),
|
||||
to: nil, from: nil, for: nil
|
||||
)
|
||||
}
|
||||
}
|
||||
.animation(.easeInOut(duration: 0.3), value: errorMessage)
|
||||
.task { serverOnline = await ComfyClient.shared.checkHealth() }
|
||||
}
|
||||
|
||||
@MainActor
|
||||
private func generate() async {
|
||||
guard let sourceImage, !isProcessing else { return }
|
||||
isProcessing = true
|
||||
errorMessage = nil
|
||||
do {
|
||||
let result = try await ComfyClient.shared.generateImage(source: sourceImage, prompt: selectedPrompt)
|
||||
let result = try await ComfyClient.shared.generateImage(
|
||||
source: sourceImage,
|
||||
style: selectedStyle ?? dreamWeaverStyles[0].id, // default: scandinavian
|
||||
keywords: keywords.trimmingCharacters(in: .whitespaces)
|
||||
)
|
||||
withAnimation(.easeInOut(duration: 0.4)) {
|
||||
generatedImage = result
|
||||
}
|
||||
} catch {
|
||||
print("Dream Weaver error: \(error)")
|
||||
errorMessage = error.localizedDescription
|
||||
}
|
||||
isProcessing = false
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Prompt Input Bar
|
||||
|
||||
private struct PromptInputBar: View {
|
||||
@Binding var text: String
|
||||
let isDisabled: Bool
|
||||
let onSubmit: () -> Void
|
||||
|
||||
@FocusState private var isFocused: Bool
|
||||
@State private var shimmer = false
|
||||
|
||||
private let placeholder = "gold, marble, luxury... (optional keywords)"
|
||||
|
||||
var body: some View {
|
||||
HStack(spacing: 10) {
|
||||
ZStack(alignment: .leading) {
|
||||
if text.isEmpty {
|
||||
Text(placeholder)
|
||||
.font(.system(size: 14))
|
||||
.foregroundStyle(Color.white.opacity(0.35))
|
||||
.padding(.leading, 4)
|
||||
.allowsHitTesting(false) // let taps pass through to the gesture below
|
||||
}
|
||||
TextField("", text: $text, axis: .vertical)
|
||||
.font(.system(size: 14))
|
||||
.foregroundStyle(Color.white)
|
||||
.lineLimit(1...3)
|
||||
.focused($isFocused)
|
||||
.submitLabel(.send)
|
||||
.onSubmit {
|
||||
guard !isDisabled, !text.trimmingCharacters(in: .whitespaces).isEmpty else { return }
|
||||
onSubmit()
|
||||
}
|
||||
.tint(Color(red: 0.231, green: 0.510, blue: 0.965))
|
||||
}
|
||||
.contentShape(Rectangle()) // expand hit area to full ZStack bounds
|
||||
.onTapGesture { isFocused = true } // focus immediately on any tap
|
||||
|
||||
// Send arrow button
|
||||
Button {
|
||||
isFocused = false
|
||||
onSubmit()
|
||||
} label: {
|
||||
ZStack {
|
||||
Circle()
|
||||
.fill(
|
||||
LinearGradient(
|
||||
colors: [
|
||||
Color(red: 0.231, green: 0.510, blue: 0.965),
|
||||
Color(red: 0.40, green: 0.25, blue: 0.95)
|
||||
],
|
||||
startPoint: .topLeading,
|
||||
endPoint: .bottomTrailing
|
||||
)
|
||||
)
|
||||
.frame(width: 36, height: 36)
|
||||
|
||||
Image(systemName: "arrow.up")
|
||||
.font(.system(size: 14, weight: .bold))
|
||||
.foregroundStyle(.white)
|
||||
}
|
||||
}
|
||||
.disabled(isDisabled || text.trimmingCharacters(in: .whitespaces).isEmpty)
|
||||
.opacity(isDisabled || text.trimmingCharacters(in: .whitespaces).isEmpty ? 0.4 : 1.0)
|
||||
.animation(.easeInOut(duration: 0.2), value: text.isEmpty)
|
||||
}
|
||||
.padding(.horizontal, 14)
|
||||
.padding(.vertical, 10)
|
||||
.background(
|
||||
RoundedRectangle(cornerRadius: 14)
|
||||
.fill(Color.white.opacity(0.06))
|
||||
.overlay(
|
||||
RoundedRectangle(cornerRadius: 14)
|
||||
.stroke(
|
||||
isFocused
|
||||
? Color(red: 0.231, green: 0.510, blue: 0.965).opacity(0.8)
|
||||
: Color.white.opacity(0.12),
|
||||
lineWidth: 1
|
||||
)
|
||||
)
|
||||
)
|
||||
.animation(.easeInOut(duration: 0.2), value: isFocused)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Dollhouse
|
||||
|
||||
private struct DollhousePanel: View {
|
||||
@Binding var hour: Double
|
||||
@Binding var tickHour: Int
|
||||
@@ -275,6 +527,8 @@ private struct DollhousePanel: View {
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - SceneKit Dollhouse
|
||||
|
||||
private struct SceneKitDollhouseView: UIViewRepresentable {
|
||||
@Binding var hour: Double
|
||||
|
||||
@@ -346,6 +600,8 @@ private struct SceneKitDollhouseView: UIViewRepresentable {
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - ProcessingOverlay
|
||||
|
||||
private struct ProcessingOverlay: View {
|
||||
@State private var animate = false
|
||||
|
||||
@@ -384,6 +640,8 @@ private struct ProcessingOverlay: View {
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - DashedSunLine
|
||||
|
||||
private struct DashedSunLine: Shape {
|
||||
func path(in rect: CGRect) -> Path {
|
||||
var path = Path()
|
||||
@@ -396,9 +654,13 @@ private struct DashedSunLine: Shape {
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - CameraPicker
|
||||
|
||||
/// UIImagePickerController wrapper that delivers the captured image via a callback,
|
||||
/// triggering orientation fix and clearing stale state immediately on capture.
|
||||
private struct CameraPicker: UIViewControllerRepresentable {
|
||||
@Binding var image: UIImage?
|
||||
@Binding var isPresented: Bool
|
||||
let onCapture: (UIImage) -> Void
|
||||
|
||||
func makeCoordinator() -> Coordinator {
|
||||
Coordinator(self)
|
||||
@@ -407,12 +669,18 @@ private struct CameraPicker: UIViewControllerRepresentable {
|
||||
func makeUIViewController(context: Context) -> UIImagePickerController {
|
||||
let picker = UIImagePickerController()
|
||||
picker.delegate = context.coordinator
|
||||
#if targetEnvironment(simulator)
|
||||
// Newer Simulators report camera as available but the shutter never
|
||||
// delivers an image. Force photo library so testing actually works.
|
||||
picker.sourceType = .photoLibrary
|
||||
#else
|
||||
if UIImagePickerController.isSourceTypeAvailable(.camera) {
|
||||
picker.sourceType = .camera
|
||||
picker.cameraCaptureMode = .photo
|
||||
} else {
|
||||
picker.sourceType = .photoLibrary
|
||||
}
|
||||
#endif
|
||||
return picker
|
||||
}
|
||||
|
||||
@@ -431,9 +699,29 @@ private struct CameraPicker: UIViewControllerRepresentable {
|
||||
|
||||
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey: Any]) {
|
||||
if let captured = info[.originalImage] as? UIImage {
|
||||
parent.image = captured
|
||||
parent.onCapture(captured)
|
||||
}
|
||||
parent.isPresented = false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Share Sheet
|
||||
|
||||
/// Wraps UIActivityViewController to match the native iOS Photos share experience.
|
||||
/// Natively includes: Save Image, AirDrop, Messages, Mail, Copy, and all installed share extensions.
|
||||
private struct ShareSheet: UIViewControllerRepresentable {
|
||||
let image: UIImage
|
||||
|
||||
func makeUIViewController(context: Context) -> UIActivityViewController {
|
||||
UIActivityViewController(activityItems: [image], applicationActivities: nil)
|
||||
}
|
||||
|
||||
func updateUIViewController(_ uvc: UIActivityViewController, context: Context) {}
|
||||
}
|
||||
|
||||
// MARK: - UIImage + Identifiable
|
||||
// Required to use UIImage as the `item` in .sheet(item:)
|
||||
extension UIImage: @retroactive Identifiable {
|
||||
public var id: ObjectIdentifier { ObjectIdentifier(self) }
|
||||
}
|
||||
|
||||
17
iOS/velocity/velocity/Info.plist
Normal file
17
iOS/velocity/velocity/Info.plist
Normal file
@@ -0,0 +1,17 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>NSAppTransportSecurity</key>
|
||||
<dict>
|
||||
<key>NSAllowsArbitraryLoads</key>
|
||||
<true/>
|
||||
</dict>
|
||||
<key>NSPhotoLibraryUsageDescription</key>
|
||||
<string>Velocity needs access to your photo library to let you pick room photos for Dream Weaver.</string>
|
||||
<key>NSPhotoLibraryAddUsageDescription</key>
|
||||
<string>Velocity would like to save your AI-generated room design to your photo library.</string>
|
||||
<key>NSCameraUsageDescription</key>
|
||||
<string>Velocity needs camera access to capture room photos for Dream Weaver.</string>
|
||||
</dict>
|
||||
</plist>
|
||||
Reference in New Issue
Block a user