Supermarket Actions

Assistant Actions

Greet: A stand-alone greeting, welcome, offer to help, thanks, apology, farewell, or another social courtesy.
Ask Preference: Ask about a preference the customer has not yet expressed.
Clarify Need: Clarify a need the customer has expressed ambiguously, inconsistently, or with an unclear reference.
Navigate: Guide the customer to move, turn, look around, or go to a location.
Recommend: Explicitly recommend a specific product.
Explain: Proactively explain product information, features, uses, or recommendation rationale.
Answer Question: Directly answer an explicit customer question.
Compare: Compare two or more specific products.
Handle Rejection: Respond to a customer rejection and adjust based on the reason.
Confirm Purchase: After the customer expresses acceptance, confirm the product or guide purchase completion.

User Actions

Move Forward: Walk forward along the current facing direction, usually to approach the area in front.

{
  "type": "final",
  "tool_calls": [
    {
      "name": "move_forward",
      "arguments": {
        "duration": 1.0
      }
    }
  ],
  "content": "..."
}

Duration: a positive number (seconds).

Move Backward: Step backward without changing the final facing direction; useful to back away, avoid occlusion, or recover the view.
```
{
  "type": "final",
  "tool_calls": [
    {
      "name": "move_backward",
      "arguments": {
        "duration": 1.0
      }
    }
  ],
  "content": "..."
}
```
Duration: a positive number (seconds).
Turn And Move: Turn left or right by an angle, then walk forward briefly; useful for exploring the left-front or right-front area.
```
{
  "type": "final",
  "tool_calls": [
    {
      "name": "turn_and_move",
      "arguments": {
        "direction": "right",
        "degree": 30,
        "duration": 1.0
      }
    }
  ],
  "content": "..."
}
```
Direction: right / left. Degree: a positive number (degrees). Duration: a positive number (seconds).
Move And Turn: Walk forward along the current facing direction, then turn left or right by an angle; useful for passing through the space ahead before observing the area after the turn.
```
{
  "type": "final",
  "tool_calls": [
    {
      "name": "move_and_turn",
      "arguments": {
        "direction": "right",
        "degree": 30,
        "duration": 1.0
      }
    }
  ],
  "content": "..."
}
```
Direction: right / left. Degree: a positive number (degrees). Duration: a positive number (seconds).
Side Step: Step left or right without changing facing direction; useful to avoid occlusion or adjust the viewing angle.
```
{
  "type": "final",
  "tool_calls": [
    {
      "name": "side_step",
      "arguments": {
        "direction": "left",
        "duration": 1.0
      }
    }
  ],
  "content": "..."
}
```
Direction: left / right. Duration: a positive number (seconds).

Turn: Turn in place to the left or right; changes facing direction only.

{
  "type": "final",
  "tool_calls": [
    {
      "name": "turn",
      "arguments": {
        "direction": "right",
        "degree": 30
      }
    }
  ],
  "content": "..."
}

Direction: right / left. Degree: a positive number (degrees).

Look Up Down: Adjust pitch up or down only.

{
  "type": "final",
  "tool_calls": [
    {
      "name": "look_up_down",
      "arguments": {
        "direction": "up",
        "degree": 30
      }
    }
  ],
  "content": "..."
}

Direction: up / down. Degree: a positive number (degrees).

Move To Fixture: Move near a currently visible fixture. nearest goes to the closest reachable point and does not additionally adjust facing after movement. front and back move to the fixed point for that fixture side and align the customer to that side for the next observation.
```
{
  "type": "final",
  "tool_calls": [
    {
      "name": "move_to_fixture",
      "arguments": {
        "fixture_id": "<fixture_id>",
        "approach": "nearest"
      }
    }
  ],
  "content": "..."
}
```
Replace fixture_id with a key from current_obs_for_customer.visible_fixtures. Approach: nearest / front / back.
Pick Up Item: Pick up a currently visible and pickable product, entering the held-item confirmation state.
```
{
  "type": "final",
  "tool_calls": [
    {
      "name": "pick_up_item",
      "arguments": {
        "item_id": "<item_id>"
      }
    }
  ],
  "content": "..."
}
```
Replace item_id with a key from current_obs_for_customer.visible_items.

Add To Cart: Add the currently held and confirmed product to the shopping cart.

{
  "type": "final",
  "tool_calls": [
    {
      "name": "add_to_cart",
      "arguments": {
        "item_id": "<item_id>"
      }
    }
  ],
  "content": "..."
}

Replace item_id with the ID of the held product.

Drop Item: Put down the held product and exit the held-item confirmation state.

{
  "type": "final",
  "tool_calls": [
    {
      "name": "drop_item",
      "arguments": {}
    }
  ],
  "content": "..."
}

Idle: When not holding an item, do not move, pick up, or add anything; only talk naturally or wait.

{
  "type": "final",
  "tool_calls": [
    {
      "name": "idle",
      "arguments": {}
    }
  ],
  "content": "..."
}