Instance-Enriched Semantic Maps
for Visual Language Navigation

Q: There is one {class_name} in the scene. Describe its color,
   material, and one special feature. Don't repeat any word or
   phrase. Limit your response to 2 sentences maximum. Finish as
   soon as main traits are mentioned. Avoid repeating words,
   phrases, or patterns.

The rug is a dark grey, woven, and has a subtle pattern of
rounded shapes. It has a soft, plush pile.

Identify and describe instances. Input and output must be in JSON format.
The input field 'captions' contains a list of image captions aiming to identify the instance.
Output 'color' as a visual color of the identified instance.
Output 'material' making up the identified instance.
Output 'caption' as a concise description of the identified instance.

{
  "category": "sofa",
  "room": "living room",
  "captions": [
    "The sofa is a brown leather couch placed in front of a window,
     well-maintained and comfortable.",
    "The sofa is an antique-style couch made of brown leather with
     a vintage appearance.",
    "The sofa is a brown leather couch; a doll is sitting on it."
  ]
}

{
  "category": "sofa",
  "room": "living room",
  "color": "brown",
  "material": "leather",
  "captions": "The well-maintained brown leather couch, with a
               vintage charm, sits in front of a window."
}

You are a query signal analyzer for hierarchical object-goal navigation.
Task:
1) Analyze which signals are strong in the query: object, room, caption, abstract.
2) Return integer scores in [0, 4] for each signal.
3) Extract optional hints:
   - object_category : canonical object phrase if explicitly/strongly implied, else null
   - room_hint       : room/location phrase if present, else null
   - caption_hint    : visual-attribute phrase if present, else null
   - abstract_hint   : affordance/function phrase if present, else null

Scoring rubric:
  object   — explicit object noun/category mention strength
  room     — room/location context strength
  caption  — concrete visual details (material/color/shape/parts) strength
  abstract — functional/affordance intent without explicit object strength

Output JSON only. No markdown. No extra text.
Output schema:
{
  "scores": {"object": int, "room": int, "caption": int, "abstract": int},
  "object_category": string|null,
  "room_hint"      : string|null,
  "caption_hint"   : string|null,
  "abstract_hint"  : string|null
}

Query:
{query_text}

Return strict JSON only.

{
  "scores": {"object": 0, "room": 2, "caption": 3, "abstract": 4},
  "object_category": null,
  "room_hint"      : "office",
  "caption_hint"   : "red",
  "abstract_hint"  : "seating"
}

You are a ranking expert in a hierarchical navigation system.
Rules:
- Use ONLY the provided query and candidate fields.
- Return exactly top-{n} unique instance IDs, sorted best to worst.
- IDs must be chosen from the candidate list only.
- If confidence is low, still return best possible top-{n}.

Output JSON only. No extra text.
{ "ranked_instance_ids": [id1, id2, ...] }

--- Type-specific focus directive (appended per expert) ---

Object Expert:
  Focus on object identity/category matching.
  Primary: category/object semantics. Secondary: attribute hints.
  Do not over-weight room context unless it disambiguates ties.

Room Expert:
  Focus on room compatibility.
  Primary: room_hint / query room semantics vs candidate room_cat.
  Secondary: object plausibility in that room.

Caption Expert:
  Focus on fine-grained visual attributes.
  Primary: caption_hint / query visual details vs candidate caption.
  Prefer candidates with discriminative detail match.

Abstract Expert:
  Focus on affordance / functional intent.
  Infer likely object from function-level query semantics.
  Do not require exact object noun overlap.

Query: {query_text}

Candidates (JSON):
[
  {"id": 1, "category": "chair", "room_cat": "office",
   "caption": "A vivid red leather chair with a metal frame."},
  {"id": 2, "category": "sofa", "room_cat": "living room",
   "caption": "Light blue velvet sofa with buttoned design."},
  ...
]

Return top-{n} ranked instance IDs as JSON only.

{ "ranked_instance_ids": [1, 3, ...] }

You are a careful vision-language assistant.
Follow the format strictly. Use ONLY one choice from each provided list.
Keep EXACT object/room tokens when required.
Prefer concrete, visually verifiable attributes.

Shared lexical lists:
  VERB_VARIANTS      = [Find, Look for, Search for, Seek]
  ATTR_CLAUSE_VARIANTS = [with, featuring, showcasing,
                          characterized by, sporting]
  ROOM_PHRASES(rc)   = [in the {rc}, inside the {rc}, within the {rc},
                        at the {rc}, located in the {rc}, found in the {rc}]

Common sentence rules (all types):
  - Write EXACTLY ONE complete sentence ending with a period.
  - No bullet points, markdown, explanations, or prefixes.
  - Do NOT use quotes or brackets.
  - Avoid words like "object", "instance", or "image".

You are given an image and a target object category.

Target object category (use EXACTLY as given): "{instance_category}"

Task:
- Generate a short, natural search query that helps find this target.
- Start with EXACTLY ONE verb from VERB_VARIANTS.
- Optionally add ONE attribute clause from ATTR_CLAUSE_VARIANTS.
- Use the object token EXACTLY — no synonyms, no pluralization.
- Do NOT mention any room or location phrase.
[+ keyframe image]

Pattern:
  Find a {instance_category} featuring metal legs.
  Look for a {instance_category} with a glass top.

Find a chair featuring a dark wooden frame and a cushioned seat.

You are given an image and two target categories.

Object: "{instance_category}"  (use EXACTLY)
Room:   "{room_category}"      (use EXACTLY — no synonyms, no adjectives)

Task:
- Include BOTH tokens.
- Start with EXACTLY ONE verb from VERB_VARIANTS.
- Include the room using EXACTLY ONE phrase from ROOM_PHRASES.
[+ keyframe image]

Pattern:
  Find a {instance_category} inside the {room_category}.
  Look for a {instance_category} located in the {room_category}.

Look for a chair located in the office.

You are given an image and a target object token.

Object: "{instance_category}"  (use EXACTLY)
Optional references (from instance captions):
{reference_text}

Task:
- Include AT LEAST TWO concrete attributes if visible:
  (i)  ONE color or material word
  (ii) ONE distinctive feature that disambiguates among same-category items
       (e.g., ice maker, sliding doors, brass doorknob, rounded edge,
        metal legs, glass top, stainless-steel finish, woodgrain texture)
- Use EXACTLY ONE connector from ATTR_CLAUSE_VARIANTS.
- Do NOT mention any room or location phrase.
[+ keyframe image]

Pattern:
  Find a {instance_category} featuring a stainless steel finish
  and a built-in water dispenser.

Find a chair characterized by a vivid red leather surface and a metal frame.

You are given an image and two tokens.

Object: "{instance_category}"  (use EXACTLY)
Room:   "{room_category}"      (use EXACTLY)
Optional references (from instance captions):
{reference_text}

Task:
- Include AT LEAST TWO concrete attributes (same rules as (o,d)).
- Use EXACTLY ONE connector from ATTR_CLAUSE_VARIANTS.
- Include the room using EXACTLY ONE phrase from ROOM_PHRASES.
[+ keyframe image]

Pattern:
  Find a {instance_category} featuring a glossy white finish and
  sliding doors located in the {room_category}.

Find a chair characterized by a vivid red leather surface and
a metal frame found in the office.

You are given an image and a hidden object token.

Hidden object token: "{instance_category}"  (DO NOT output this token)
Room:               "{room_category}"       (use EXACTLY)
Optional references (from instance captions):
{reference_text}

Task:
- Produce ONE affordance-based query — NO explicit category token in output.
- Start with EXACTLY ONE verb from VERB_VARIANTS.
- Include AT LEAST TWO concrete attributes if visible (color/material +
  distinctive feature).
- Use EXACTLY ONE connector from ATTR_CLAUSE_VARIANTS.
- Include the room using EXACTLY ONE phrase from ROOM_PHRASES.
[+ keyframe image]

Pattern:
  Find something to keep food cold featuring a stainless steel finish
  and an ice maker located in the {room_category}.

Find something to sit on characterized by a vivid red leather surface
and a metal frame found in the office.