ChatImage

The interactive answer image Open source Technical report

Interactive Visual Images for
Long-Form LLM Answers

Zhejiang University Yuquan Mao statue scene
9
Strict exported demos
0
Mask gaps published
0
API keys to view
100%
Reusable JSON state
What it does

From wall-of-text answer to explorable image

Instead of reading a long text block, users can click the parts of an image they care about and ask follow-up questions in context.

01

Clickable hotspots

Transparent, accurate regions are placed over the generated image so each section can be explored independently.

02

Per-region threads

Each hotspot keeps its own title, summary, and follow-up conversation instead of flattening everything into one chat.

03

Vision-aligned positioning

Real-provider mode can use LocateAnything, MiMo-vision, local OCR, and SAM3 refinement to align regions with the rendered image.

04

Provider agnostic

Run fully in mock mode, or proxy text, image, and vision providers through the backend so API keys stay off the browser.

05

Local persistence

Generated ChatImages, hotspots, calibration data, and follow-up threads are stored locally so work survives restarts.

06

Calibration tools

Inspect and correct hotspot bounds when a provider draws content differently from the planned layout.

Interaction loop

How one answer becomes a visual workspace

The hosted showcase is exported from real local runs: the image, hotspot bounds, layout state, and region detail JSON are preserved together.

Ask a question

The user submits a prompt in the ChatImage interface.

Generate an answer

A long-form answer is produced by the selected provider or by mock mode.

Normalize the answer

The answer becomes structured modules, auxiliary modules, and region metadata.

Plan a visual layout

A layout spec assigns each module a region and normalized bounds.

Render the image

The image provider draws the visual from the structured prompt.

Align hotspots

Vision grounding can localize where each module actually appears in the image.

Explore and follow up

Clicking a hotspot opens its detail panel and a focused follow-up thread.

See it in action

Real generated interactive demos

These curated cases use actual ChatImage generated PNGs and exported hotspot state. Open a demo, click its regions, and reuse the preserved JSON state when needed.

West Lake hand-drawn tour map demo

West Lake hand-drawn tour map

Hand-drawn West Lake tour image with preserved SAM-backed hotspots for the lake, causeways, islands, pagoda, hills, and garden regions.

hand-drawn map9 hotspots
Li Bai and Shakespeare comparison real generated demo

Li Bai and Shakespeare comparison

Strict literary comparison. A split-scene image with clickable poets, landscape, wine-scroll motif, theatre manuscript, and shared legacy region.

strict literary comparison6 hotspots
Zhejiang University Yuquan Mao statue scene real generated demo

Zhejiang University Yuquan Mao statue scene

Reference-aligned Yuquan campus view with clickable Mao statue, stone pedestal, front lawn, sunset hillside, and framing tree canopies.

reference-aligned campus scene6 hotspots
Transformer development timeline real generated demo

Transformer development timeline

Strict academic timeline. Six clickable milestones trace Transformer-based language models from architecture to interactive use.

strict academic timeline6 hotspots
Healthy breakfast options real generated demo

Healthy breakfast options

Fresh daily scene. A breakfast comparison with clickable oatmeal bowl, yogurt cup, sandwich, eggs, fruit, and coffee.

verified fresh daily scene6 hotspots
Boutique coffee shop scene real generated demo

Boutique coffee shop scene

Verified visual scene. A cafe layout with clickable barista, espresso machine, pastry case, window seats, pickup shelf, and queue.

verified visual scene6 hotspots
Sunny reading nook real generated demo

Sunny reading nook

Fresh daily scene. A quiet apartment corner with clickable armchair, bookshelf, floor lamp, window, and tea table.

verified fresh daily scene5 hotspots
Independent record store corner real generated demo

Independent record store corner

Fresh daily scene. A record-store corner with clickable listening station, vinyl bins, counter, new arrivals wall, and posters.

verified fresh daily scene5 hotspots
Indoor plant care corner real generated demo

Indoor plant care corner

Fresh daily scene. A plant-care setup with clickable monstera, watering can, grow light, potting bench, and humidity tray.

verified fresh daily scene5 hotspots
Get started

Run it locally in 3 commands

No API keys are required for mock mode, so the demo path works immediately after install.

$ git clone https://github.com/wencanjiang/ChatImage.git $ cd ChatImage $ npm install $ npm start # then open in your browser: # http://127.0.0.1:5178?provider=mock # to use real providers, add keys: $ cp .env.example .env.local # edit .env.local $ npm start
  1. Install Node.js 22.5+ and npm.
  2. Clone the repository and run npm install.
  3. npm start launches the local server on port 5178.
  4. Open ?provider=mock for instant key-free demos.
  5. For real providers, copy .env.example to .env.local, fill in keys, and restart.
  6. Build the frontend with npm run build when you need static assets in dist/.

Full config docs

Ready to explore answers visually?

Clone the project, start mock mode, and inspect an interactive ChatImage locally.