# Dream Weaver — iOS ↔ Backend Integration Guide
**Version:** 2.0 | **Updated:** 2026-03-09 | **Server:** `54.172.172.2` | **Port:** `8080`

> This document is for **Sayan** (iOS / Swift) and **Sourik** (backend review).  
> It describes exactly how the iPad app should talk to the Dream Weaver AI and how keywords from a user tap become a full ComfyUI generation.

---

## 1. Architecture Overview

```
┌────────────────────┐       HTTP/S        ┌──────────────────────────────┐
│                    │ ─── POST image ───► │  Dream Weaver Gateway        │
│  iPad App (Swift)  │                     │  FastAPI  port 8080          │
│                    │ ◄── PNG result ───  │  dw_gateway.py               │
└────────────────────┘                     └─────────────┬────────────────┘
                                                         │ internal HTTP
                                                         ▼
                                            ┌─────────────────────────┐
                                            │  ComfyUI Engine         │
                                            │  port 8188              │
                                            │  RealVisXL V5.0 Ltng    │
                                            │  4× NVIDIA L4 (96 GB)   │
                                            └─────────────────────────┘
```

**Key rule:** The iPad app **never** talks to ComfyUI directly. It only talks to the Gateway on `:8080`.

---

## 2. How Keywords Become Prompts

### 2.1 The Prompt Expansion System

Each interior style in the app is backed by a **prompt template** in `comfy_engine/prompts/`. When the user taps a style card (or types keywords), those keywords get **merged into the template** to build the final ComfyUI prompt injected into node `9` (positive CLIPTextEncode).

**Prompt template structure** (from [scandinavian_minimalist.txt](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/prompts/scandinavian_minimalist.txt)):
```
POSITIVE PROMPT:
scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles,
abundant natural light streaming through large windows, clean white walls, ...
Style Weight: <lora:Interior_Style_Scandi:0.8>

NEGATIVE PROMPT:
(worst quality, low quality, illustration, 3d render...), heavy ornamentation,...

TECHNICAL PARAMETERS:
- Denoising Strength: 0.70
- CFG Scale: 7.0
- Recommended Sampler: dpmpp_2m_karras
- Steps: 30-40
```

### 2.2 Keyword Expansion Flow

```
User taps: ["marble", "gold", "luxury"]
           +
Style selected: "art_deco"
       │
       ▼
Backend expands:
  base_prompt = art_deco_luxe.txt (POSITIVE PROMPT section)
  user_keywords_str = "marble, gold, luxury"
  final_prompt = base_prompt + ", " + user_keywords_str
       │
       ▼
Injected into ComfyUI workflow:
  node "9" → CLIPTextEncode → text: [final_prompt]
  node "10" → CLIPTextEncode → text: [negative_prompt from template]
  node "1"  → LoadImage      → image: [uploaded filename]
  node "13" → KSampler       →  denoise: 0.72, cfg: 7.5, steps: 35
```

### 2.3 Available Styles and Their Keywords (for the Style Picker UI)

| Style ID | Display Name | Suggested Keywords Palette |
|---|---|---|
| `scandinavian` | Scandinavian Minimalist | oak, linen, white, hygge, cozy, birch, natural |
| `art_deco` | Art Deco Luxe | gold, marble, velvet, geometric, 1920s, brass, crystal |
| `biophilic` | Biophilic Organic | green wall, stone, rattan, terracotta, botanical, moss |
| `cyberpunk` | Cyberpunk Neon | neon, chrome, holographic, dark, LED, futuristic, blade runner |
| `japandi` | Japandi Fusion | wabi-sabi, ash wood, ceramic, zen, minimal, shoji, serene |

---

## 3. API Reference — What Sayan Needs to Call

### BASE URL
```
http://54.172.172.2:8080
```

> [!NOTE]
> Once we attach an Elastic IP or domain, swap this in `AppConfig.swift`.

---

### 3.1 `GET /health` — Liveness Check

Call this on app launch to confirm the server is up before showing the Generate button.

**Request:**
```http
GET http://54.172.172.2:8080/health
```

**Response:**
```json
{
  "status": "ok",
  "comfyui": true,
  "gpu": "4x NVIDIA L4 (96GB VRAM)",
  "model": "RealVisXL V5.0 Lightning"
}
```

**Swift:**
```swift
func checkServerHealth() async throws -> Bool {
    let url = URL(string: "\(AppConfig.baseURL)/health")!
    let (data, _) = try await URLSession.shared.data(from: url)
    let json = try JSONDecoder().decode(HealthResponse.self, from: data)
    return json.status == "ok"
}
```

---

### 3.2 `POST /dream-weaver` — Submit Generation Job (Async)

Use this for the main generation flow. Returns a `job_id` immediately; poll for result.

**Request:** `multipart/form-data`
| Field | Type | Required | Description |
|---|---|---|---|
| [image](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/a100_deployment_executor.py#306-322) | File (JPEG/PNG) | ✅ | The room photo from camera or library |
| [style](file:///F:/Workin%20In%20Progress/DESINEURON/GITLAB/Project_Velocity/comfy_engine/scripts/dreamweaver_batch_processor.py#394-414) | String | ✅ | One of: `scandinavian`, `art_deco`, `biophilic`, `cyberpunk`, `japandi` |
| `keywords` | String | ➕ | Comma-separated user keywords e.g. `"gold, marble, luxury"` |
| `denoise` | Float | ➕ | 0.5–0.85 (default `0.72`). Higher = more creative |

**Response:**
```json
{
  "job_id": "a1b2c3d4-...",
  "status": "processing",
  "poll_url": "/dream-weaver/status/a1b2c3d4-...",
  "result_url": "/dream-weaver/result/a1b2c3d4-..."
}
```

**Swift example:**
```swift
func submitGeneration(image: UIImage, style: String, keywords: [String]) async throws -> GenerationJob {
    let url = URL(string: "\(AppConfig.baseURL)/dream-weaver")!
    var request = URLRequest(url: url)
    request.httpMethod = "POST"

    let boundary = UUID().uuidString
    request.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")

    var body = Data()
    // Image field
    let imageData = image.jpegData(compressionQuality: 0.85)!
    body.append("--\(boundary)\r\n".data(using: .utf8)!)
    body.append("Content-Disposition: form-data; name=\"image\"; filename=\"room.jpg\"\r\n".data(using: .utf8)!)
    body.append("Content-Type: image/jpeg\r\n\r\n".data(using: .utf8)!)
    body.append(imageData)
    body.append("\r\n".data(using: .utf8)!)

    // Style field
    body.append("--\(boundary)\r\n".data(using: .utf8)!)
    body.append("Content-Disposition: form-data; name=\"style\"\r\n\r\n".data(using: .utf8)!)
    body.append("\(style)\r\n".data(using: .utf8)!)

    // Keywords field (user tapped keywords)
    if !keywords.isEmpty {
        let kwString = keywords.joined(separator: ", ")
        body.append("--\(boundary)\r\n".data(using: .utf8)!)
        body.append("Content-Disposition: form-data; name=\"keywords\"\r\n\r\n".data(using: .utf8)!)
        body.append("\(kwString)\r\n".data(using: .utf8)!)
    }

    body.append("--\(boundary)--\r\n".data(using: .utf8)!)
    request.httpBody = body

    let (data, _) = try await URLSession.shared.data(for: request)
    return try JSONDecoder().decode(GenerationJob.self, from: data)
}
```

---

### 3.3 `GET /dream-weaver/status/{job_id}` — Poll Job Status

Poll every **2 seconds** until `ready == true`.

**Response while processing:**
```json
{ "status": "processing", "ready": false, "style": "art_deco" }
```

**Response when done:**
```json
{
  "status": "done",
  "ready": true,
  "result_url": "/dream-weaver/result/a1b2c3d4-...",
  "style": "art_deco"
}
```

**Swift polling loop:**
```swift
func pollForResult(jobId: String) async throws -> URL {
    let statusURL = URL(string: "\(AppConfig.baseURL)/dream-weaver/status/\(jobId)")!
    
    for _ in 0..<150 {  // max 5 min (150 × 2s)
        try await Task.sleep(nanoseconds: 2_000_000_000)  // 2 seconds
        let (data, _) = try await URLSession.shared.data(from: statusURL)
        let status = try JSONDecoder().decode(JobStatus.self, from: data)
        
        if status.ready {
            return URL(string: "\(AppConfig.baseURL)/dream-weaver/result/\(jobId)")!
        }
        if status.status == "error" {
            throw DreamWeaverError.generationFailed(status.error ?? "Unknown")
        }
    }
    throw DreamWeaverError.timeout
}
```

---

### 3.4 `GET /dream-weaver/result/{job_id}` — Download Result Image

Returns a PNG image stream directly. Download and display to user.

```swift
func downloadResult(resultURL: URL) async throws -> UIImage {
    let (data, _) = try await URLSession.shared.data(from: resultURL)
    guard let image = UIImage(data: data) else {
        throw DreamWeaverError.invalidImageData
    }
    return image
}
```

---

### 3.5 `POST /dream-weaver/sync` — One-Shot Blocking Call

For **testing only** or fast network connections. Waits up to 120 seconds and returns the image directly.

```swift
// Same multipart form as /dream-weaver but returns PNG bytes directly
// Not recommended for production — use async flow above
```

---

## 4. Keyword → Prompt Expansion (Backend Change Required)

> [!IMPORTANT]
> The current [dw_gateway.py](file:///C:/Windows/Temp/dw_gateway.py) does NOT yet accept `keywords` from the app.  
> Sagnik needs to add keyword expansion to the gateway. Here is the exact code change:

**In [dw_gateway.py](file:///C:/Windows/Temp/dw_gateway.py), update [build_workflow()](file:///C:/Windows/Temp/dw_gateway.py#33-65):**

```python
# Prompt library — maps style ID to (positive_base, negative, cfg, denoise, steps)
STYLE_LIBRARY = {
    "scandinavian": {
        "pos": "scandinavian minimalist interior design, light oak wood flooring, neutral beige textiles, abundant natural light, clean white walls, simple functional furniture, cozy hygge atmosphere, architectural photography, 8k resolution, photorealistic",
        "neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, extra windows, unrealistic lighting, structural changes",
        "cfg": 7.0, "denoise": 0.70, "steps": 30,
    },
    "art_deco": {
        "pos": "art deco luxury interior design, geometric chevron patterns, gold brass accents, rich velvet upholstery emerald and sapphire, sunburst mirrors, polished marble flooring, crystal chandeliers, 1920s glamour, 8k resolution, photorealistic",
        "neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, rustic, minimalism, cheap materials",
        "cfg": 7.5, "denoise": 0.72, "steps": 30,
    },
    "biophilic": {
        "pos": "biophilic organic interior design, living green walls with ferns and moss, natural stone accent walls, rattan and bamboo furniture, abundant houseplants, earth tone sage green and terracotta, 8k resolution, photorealistic, dappled sunlight",
        "neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, synthetic materials, plastic plants",
        "cfg": 7.0, "denoise": 0.68, "steps": 30,
    },
    "cyberpunk": {
        "pos": "cyberpunk neon interior design, high contrast LED strip lighting electric blue and hot pink, reflective chrome surfaces, dark matte walls, futuristic furniture, glowing circuit patterns, tech-noir blade runner aesthetic, 8k resolution, photorealistic, volumetric fog",
        "neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, natural daylight, rustic elements",
        "cfg": 8.0, "denoise": 0.75, "steps": 30,
    },
    "japandi": {
        "pos": "japandi fusion interior design, wabi-sabi textures, low-profile furniture, muted earth tones warm grays soft browns, handmade ceramic accents, light ash wood, shoji screen elements, minimal decoration, zen garden elements, 8k resolution, photorealistic, serene",
        "neg": "(worst quality, low quality, illustration, 3d render, painting, cartoon, sketch), blurry, distorted, structural changes, bright colors, ornate decoration, cluttered",
        "cfg": 6.5, "denoise": 0.70, "steps": 30,
    },
}

def build_workflow(img_name: str, style: str = "scandinavian",
                   keywords: str = "", denoise_override: float = None) -> dict:
    s = STYLE_LIBRARY.get(style, STYLE_LIBRARY["scandinavian"])
    
    # Merge user keywords into base positive prompt
    pos = s["pos"]
    if keywords.strip():
        pos = pos + ", " + keywords.strip()
    
    cfg     = s["cfg"]
    denoise = denoise_override if denoise_override else s["denoise"]
    steps   = s["steps"]
    neg     = s["neg"]
    
    return {
        "1": {"class_type": "CheckpointLoaderSimple",
              "inputs": {"ckpt_name": "realvisxlV50_v50LightningBakedvae.safetensors"}},
        "2": {"class_type": "LoadImage",
              "inputs": {"image": img_name, "upload": "image"}},
        "3": {"class_type": "CLIPTextEncode",
              "inputs": {"text": pos, "clip": ["1", 1]}},    # ← POSITIVE PROMPT
        "4": {"class_type": "CLIPTextEncode",
              "inputs": {"text": neg, "clip": ["1", 1]}},    # ← NEGATIVE PROMPT
        "5": {"class_type": "VAEEncode",
              "inputs": {"pixels": ["2", 0], "vae": ["1", 2]}},
        "6": {"class_type": "KSampler",
              "inputs": {"model": ["1", 0], "positive": ["3", 0], "negative": ["4", 0],
                         "latent_image": ["5", 0],
                         "seed": int(time.time()) % 999983,
                         "steps": steps, "cfg": cfg,
                         "sampler_name": "dpmpp_2m", "scheduler": "karras",
                         "denoise": denoise}},
        "7": {"class_type": "VAEDecode",
              "inputs": {"samples": ["6", 0], "vae": ["1", 2]}},
        "8": {"class_type": "SaveImage",
              "inputs": {"images": ["7", 0], "filename_prefix": f"dreamweaver_{style}"}}
    }
```

**Also update the endpoint signatures** to accept `keywords`:
```python
@app.post("/dream-weaver")
async def dream_weaver(
    image: UploadFile = File(...),
    style: str = Form(default="scandinavian"),
    keywords: str = Form(default=""),        # ← ADD THIS
    denoise: float = Form(default=0.0),      # 0.0 = use style default
):
    ...
    wf = build_workflow(comfy_name, style=style, keywords=keywords,
                        denoise_override=denoise if denoise > 0 else None)
```

---

## 5. Swift Data Models

```swift
// AppConfig.swift
struct AppConfig {
    static let baseURL = "http://54.172.172.2:8080"
    // Change this to HTTPS domain once SSL is set up
}

// Models
struct GenerationJob: Codable {
    let jobId: String
    let status: String
    let pollUrl: String
    let resultUrl: String
    
    enum CodingKeys: String, CodingKey {
        case jobId = "job_id"
        case status, pollUrl = "poll_url", resultUrl = "result_url"
    }
}

struct JobStatus: Codable {
    let status: String
    let ready: Bool
    let resultUrl: String?
    let error: String?
    
    enum CodingKeys: String, CodingKey {
        case status, ready, resultUrl = "result_url", error
    }
}

struct HealthResponse: Codable {
    let status: String
    let comfyui: Bool
}

enum DreamWeaverError: Error {
    case generationFailed(String)
    case timeout
    case invalidImageData
}

// Style model for the style picker
struct InteriorStyle: Identifiable {
    let id: String       // used as the `style` form field value
    let displayName: String
    let keywords: [String]   // shown as tappable chips in UI
    let previewImage: String // local asset name
}

let availableStyles: [InteriorStyle] = [
    InteriorStyle(id: "scandinavian",  displayName: "Scandinavian",  keywords: ["oak","linen","white","hygge","cozy","birch"], previewImage: "style_scandi"),
    InteriorStyle(id: "art_deco",      displayName: "Art Deco Luxe",  keywords: ["gold","marble","velvet","geometric","brass","crystal"], previewImage: "style_artdeco"),
    InteriorStyle(id: "biophilic",     displayName: "Biophilic",      keywords: ["green wall","stone","rattan","terracotta","botanical"], previewImage: "style_biophilic"),
    InteriorStyle(id: "cyberpunk",     displayName: "Cyberpunk Neon", keywords: ["neon","chrome","LED","futuristic","dark","holographic"], previewImage: "style_cyberpunk"),
    InteriorStyle(id: "japandi",       displayName: "Japandi Fusion",  keywords: ["wabi-sabi","ceramic","ash wood","zen","minimal"], previewImage: "style_japandi"),
]
```

---

## 6. Complete Generation Flow (UI → Server → Result)

```
1. User opens camera / library → picks a room photo
2. User selects a style card  → style ID captured
3. User optionally taps keyword chips → keywords[] array built
4. User taps "Generate" →

   POST /dream-weaver (multipart)
     image: <jpeg data>
     style: "art_deco"
     keywords: "gold, marble, luxury hotel"
     denoise: 0.72

5. Server returns { job_id: "abc123", status: "processing" }
6. App shows loading/progress UI
7. App polls GET /dream-weaver/status/abc123  every 2 seconds
8. When ready == true  →
   GET /dream-weaver/result/abc123  →  returns PNG bytes
9. App displays result full-screen with save/share options
```

**Expected latency on 4× L4 GPU server:** `~15–20 seconds` end-to-end.

---

## 7. WebSocket Progress (Optional Advanced Feature)

If Sayan wants a real-time progress bar (e.g. "Step 12/30"), connect directly to ComfyUI's WebSocket **only if port 8188 is opened**. Otherwise, polling `/status` is sufficient.

```swift
// WebSocket — only if 8188 is exposed externally
class ComfyProgressWebSocket: NSObject, URLSessionWebSocketDelegate {
    var onProgress: ((Int, Int) -> Void)?
    var task: URLSessionWebSocketTask?
    
    func connect(clientId: String) {
        let url = URL(string: "ws://54.172.172.2:8188/ws?clientId=\(clientId)")!
        task = URLSession.shared.webSocketTask(with: url)
        task?.resume()
        listen()
    }
    
    private func listen() {
        task?.receive { [weak self] result in
            if case .success(let message) = result,
               case .string(let text) = message,
               let data = text.data(using: .utf8),
               let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
               let type = json["type"] as? String, type == "progress",
               let inner = json["data"] as? [String: Int] {
                self?.onProgress?(inner["value"] ?? 0, inner["max"] ?? 30)
            }
            self?.listen()  // recurse
        }
    }
}
```

> [!NOTE]
> Port 8188 is currently **not open externally** in the security group. Only port 8080 is. To use WebSocket progress, Sagnik needs to add an inbound rule for 8188. Until then, using `/status` polling every 2s gives good enough UX.

---

## 8. Error Handling

| HTTP Status | Meaning | UI Action |
|---|---|---|
| `200` | Success | Show result or job_id |
| `404` on `/status` | Job expired (> 30 min) | "Session expired. Please retry." |
| `500` | Generation failed (OOM, model error) | "Generation failed. Try a simpler image." |
| Connection error | Server down or no internet | "Checking server…" + retry logic |

The job [status](file:///C:/Windows/Temp/dw_gateway.py#154-164) field can also be `"error"` with an `error` field explaining what failed.

---

## 9. Quick Checklist for Sayan

- [ ] Update `AppConfig.swift` with `baseURL = "http://54.172.172.2:8080"`
- [ ] Implement `POST /dream-weaver` multipart with `image + style + keywords`
- [ ] Implement polling loop on `GET /dream-weaver/status/{job_id}`
- [ ] Implement image download from `GET /dream-weaver/result/{job_id}`
- [ ] Add `GET /health` check on app launch
- [ ] Build keyword chips UI with the 5 style palettes from Section 2.3
- [ ] Test with the 20 sample images in `comfy_engine/test_inputs/`

## 10. Quick Checklist for Sagnik (backend)

- [ ] Update [dw_gateway.py](file:///C:/Windows/Temp/dw_gateway.py) with the full `STYLE_LIBRARY` dict (Section 4)
- [ ] Add `keywords: str = Form(default="")` to both POST endpoints
- [ ] Pass keywords into [build_workflow()](file:///C:/Windows/Temp/dw_gateway.py#33-65) for prompt expansion
- [ ] Redeploy gateway on port 8080 (`nohup python3 dw_gateway.py &`)
- [ ] (Optional) Open port 8188 in security group for WebSocket progress