Back to paper page

CCBR Intervention Demo

Walk through the full pipeline, from raw image content to steerable recommendations, on the H&M dataset.

Explore the rest of the pipeline

See how individual items are described, how user profiles are built, and what the model ranks before any intervention.

STAGE 0

Item Content Verbalization

Each item image is captioned by a vision-language model.
  Item Image
→
Qwen3-VL-32B-Instruct
→
  Description
SYSTEM_PROMPT 
You are a fashion product description writer. Write a concise 1-2 sentence summary of the product.

RULES:
- Output EXACTLY 1-2 sentences. Nothing else.
- No bullet points, no numbered lists, no analysis steps, no drafts.
- Be specific about: product type, color, material, silhouette, and notable design details.
- Do NOT use marketing language or subjective opinions.
- Do NOT explain your reasoning or thought process.
- Start directly with the product description.
GENERATE_TEMPLATE 
Write a 1-2 sentence product description based on the image. Output ONLY the summary sentences, nothing else.
Item image
Generated description
Qwen3-VL-32B-Instruct
Loading...
Item 1 / 10
STAGE 1

Content-Based Profile Construction

Item descriptions are aggregated into one textual user profile.
  Item Set
→
Qwen3-30B-A3B-Instruct-2507
→
  User Summary
The LLM consumes each item the user interacted with (as descriptions from Stage 0) and writes a single preference profile.
User 1 / 50  ·  29 items

Interacted item set — 29 items

Generated user summary

— pending —
STAGE 2

Summary → Final Predictions

The textual profile is embedded and decoded into item scores.
  User Summary
→
BGE-M3
→
  CF Decoder
→
  Top-K Items

Top-20 recommended items

DEMO

Steering via Text Interventions

Edit the summary → the model follows the edit.
Four intervention setups are explored. Add injects an unseen attribute. Remove removes a present attribute. Each case shows the original summary, the edited summary, and the resulting prediction shift.
Intervention [name]  ·  Case 1 / 1
Intervention: [name] added
Click Next case to step through.
Original
Attributes present
Original summary
— pending —
Original predictions
After intervention
Expected attributes
Edited summary
— pending —
Predictions after intervention
STAGE 0

Item Content Verbalization

Each trailer is captioned by a video-language model.
  Trailer
→
VideoLLaMA 3
→
  Description
SYSTEM_PROMPT 
You are a movie trailer description writer. Write a concise summary of the trailer.

RULES:
- Describe the apparent genre, mood, setting, and tone.
- Mention notable visual or stylistic cues.
- Do NOT use marketing language or speculate about box office performance.
- Output a single short paragraph, nothing else.
GENERATE_TEMPLATE 
Write a one-paragraph description of this movie trailer. Output ONLY the description, nothing else.
— pending —
item 0
Generated description
VideoLLaMA-3
— pending —
Item 1 / 10
STAGE 1

From Items to a User Profile

The LLM aggregates a user's watched titles into a single preference profile.
User 1 / 50  ·  — movies

Watched movies — — titles

Generated user summary

— pending —
STAGE 2

Summary → Final Predictions

The textual profile is embedded and decoded into item scores.
  User Summary
→
BGE-M3
→
  CF Decoder
→
  Top-K Items

Top-20 recommended movies

DEMO

Steering via Text Interventions

Edit the summary → the model follows the edit.
Two intervention setups are explored on ML-20M. Add injects an unseen genre. Remove removes a present genre. Each case shows the original summary, the edited summary, and the resulting prediction shift.
Intervention [name]  ·  Case 1 / 1
Intervention: [name] added
Click Next case to step through.
Original
Genres present
Original summary
— pending —
Original predictions
After intervention
Expected genres
Edited summary
— pending —
Predictions after intervention
STAGE 0

Item Content Verbalization

Each track is captioned by an audio-language model.
  Audio
→
Music-Flamingo-7B
→
  Description
SYSTEM_PROMPT 
You are a music description writer. Write a concise summary of the track.

RULES:
- Describe the apparent genre, mood, instrumentation, and tempo.
- Mention notable sonic or stylistic cues.
- Do NOT use marketing language or speculate about chart performance.
- Output a single short paragraph, nothing else.
GENERATE_TEMPLATE 
Write a one-paragraph description of this track. Output ONLY the description, nothing else.
— pending —
item 0
Generated description
Music-Flamingo-7B
— pending —
Item 1 / 10
STAGE 1

From Tracks to a User Profile

The LLM aggregates a user's listened tracks into a single preference profile.
User 1 / 50  ·  — tracks

Listened tracks — — tracks

Generated user summary

— pending —
STAGE 2

Summary → Final Predictions

The textual profile is embedded and decoded into item scores.
  User Summary
→
BGE-M3
→
  CF Decoder
→
  Top-K Items

Top-20 recommended tracks

DEMO

Steering via Text Interventions

Edit the summary → the model follows the edit.
Two intervention setups are explored on MSD. Add injects an unseen tag. Remove removes a present tag. Each case shows the original summary, the edited summary, and the resulting prediction shift.
Intervention [name]  ·  Case 1 / 1
Intervention: [name] added
Click Next case to step through.
Original
Tags present
Original summary
— pending —
Original predictions
After intervention
Expected tags
Edited summary
— pending —
Predictions after intervention