SituRecBench

Assisting Users from Language Instructions to Physical Actions

Demo Coming Soon

Free Move

Move forward, backward, or sideways while keeping the final facing direction.

Related Benchmarks

Benchmark	Spatial Environment	Egocentric View	Dialogue	Physical Action	Recommendation	Interactive Evaluation	Scenario
SIMMC-VR (Wu et al., 2023)	Included	Included	Included	Not included	Included	Not included	Virtual Reality Shopping
SCREEN (Lin et al., 2024)	Included	Included	Included	Not included	Included	Not included	Multimodal Recommendation
MUSE (Wang et al., 2025)	Not included	Not included	Included	Not included	Included	Not included	Multimodal Recommendation
MarketGen (Hu et al., 2025)	Included	Included	Not included	Included	Not included	Not included	Embodied Retail
DeliveryBench (Mao et al., 2026)	Included	Included	Not included	Included	Not included	Not included	Food Delivery
S³IT (Sun et al., 2025)	Included	Included	Included	Not included	Not included	Included	Situated Social Test
Alexa Arena (Gao et al., 2023)	Included	Included	Included	Included	Not included	Included	Gamified Robotics
SimWorld (Ye et al., 2025)	Included	Included	Included	Included	Not included	Not included	Urban Robotics
Virtual Community (Zhou et al., 2026)	Included	Included	Included	Included	Not included	Not included	Embodied Community
SituRecBench (Ours)	Included	Included	Included	Included	Included	Included	Situated Shopping

Benchmark Construction

SituRecBench framework: situated 3D environment construction, user simulation, and interactive evaluation with role-specific observations and actions.

Leaderboard

Gemini

GPT

Claude

Qwen

Gemma

All Environments Supermarket Home Furnishings Store

Rank	Assistant	SR	Overall	AFR	LNSR	R@5	NDCG@10
1	Gemini 3.1 Pro Preview	73.0_±6.2	3.53_±0.14	80.9_±1.1	64.4_±3.9	57.1_±3.6	49.9_±4.2
2	GPT-5.6 Sol	67.0_±12.9	3.41_±0.27	85.6_±1.4	70.8_±3.0	52.2_±10.2	45.6_±8.5
3	Qwen3.6-Plus	59.5_±6.6	2.82_±0.18	68.8_±3.0	48.4_±3.3	44.5_±5.5	39.0_±5.4
4	Claude Opus 5	58.0_±3.7	2.90_±0.11	75.0_±3.1	54.2_±4.2	43.4_±3.0	38.6_±2.3
5	GPT-4o	40.0_±8.3	2.40_±0.10	59.6_±1.9	34.3_±3.5	27.3_±4.6	23.4_±3.7
6	Qwen3-VL-8B-Instruct	31.5_±6.4	1.95_±0.09	40.8_±4.7	17.7_±5.7	17.8_±3.5	15.7_±3.7
7	Gemma 4 E4B (IT)	29.5_±6.8	2.15_±0.23	44.6_±2.3	18.7_±2.8	17.3_±4.1	15.7_±4.6

Rank	Assistant	SR	Overall	AFR	LNSR	R@5	NDCG@10
1	Gemini 3.1 Pro Preview	80.0_±5.7	3.61_±0.17	80.1_±1.8	48.4_±6.4	64.4_±2.1	56.0_±1.1
2	GPT-5.6 Sol	69.0_±20.8	3.36_±0.45	89.0_±1.2	53.5_±3.9	56.3_±16.0	48.2_±13.2
3	Claude Opus 5	66.0_±10.1	3.00_±0.35	78.6_±5.5	41.0_±5.0	49.8_±10.7	43.5_±9.4
4	Qwen3.6-Plus	65.0_±11.9	2.78_±0.29	65.9_±4.3	28.5_±5.6	49.6_±10.2	42.4_±9.1
5	GPT-4o	45.0_±6.8	2.33_±0.05	62.3_±2.1	15.1_±3.2	31.6_±2.3	26.6_±1.5
6	Qwen3-VL-8B-Instruct	34.0_±9.5	1.76_±0.19	34.5_±3.6	8.8_±3.0	20.3_±4.6	17.7_±5.0
7	Gemma 4 E4B (IT)	28.0_±5.7	1.94_±0.16	49.2_±2.0	10.0_±3.7	17.3_±4.5	15.0_±4.4

Rank	Assistant	SR	Overall	AFR	LNSR	R@5	NDCG@10
1	Gemini 3.1 Pro Preview	66.0_±7.7	3.45_±0.20	81.8_±1.9	83.2_±4.1	49.8_±7.1	43.7_±7.5
2	GPT-5.6 Sol	65.0_±8.2	3.45_±0.18	82.2_±3.5	88.8_±2.4	48.1_±6.5	43.0_±6.2
3	Qwen3.6-Plus	54.0_±9.5	2.86_±0.14	71.6_±4.8	70.9_±3.9	39.4_±5.3	35.5_±4.7
4	Claude Opus 5	50.0_±9.5	2.81_±0.25	71.4_±2.2	68.1_±3.9	37.0_±6.0	33.8_±6.3
5	GPT-4o	35.0_±10.0	2.46_±0.24	56.9_±4.1	53.0_±5.8	23.0_±8.7	20.2_±6.9
6	Gemma 4 E4B (IT)	31.0_±8.2	2.36_±0.33	40.0_±5.8	29.0_±2.9	17.3_±4.9	16.4_±5.5
7	Qwen3-VL-8B-Instruct	29.0_±8.9	2.14_±0.21	47.0_±6.1	26.3_±8.6	15.3_±5.4	13.7_±5.3

Citation

@inproceedings{siturecbench2026,
  author       = {Yang, Jingtong and Lin, Dongding and Wang, Jian and Li, Wenjie},
  title        = {SituRecBench: Evaluating Frontier Models as Situated Recommendation Assistants in 3D Environments},
  booktitle    = {NeurIPS 2026 Evaluations and Datasets Track},
  year         = {2026}
}

SituRecBench: Evaluating Frontier Models as Situated Recommendation Assistants in 3D Environments

Assisting Users from Language Instructions to Physical Actions

Free Move

Related Benchmarks

Benchmark Construction

Leaderboard

Citation

BibTeX