ResidentBench: A Common-Sense and Situational Awareness Benchmark—How Do Language Models Act When Left to Their Own Judgment?

As language models gain access to tools, they may encounter situations for which no task or instructions have been supplied. Performance on assigned tasks does not establish how they act in that condition. ResidentBench evaluates language models exposed to declarative sensors and actuators without an author-assigned task, goal, mission, purpose, role, persona or request to deliberate or act. A shared harness records tool calls and environmental trajectories; a separate evaluator receives the situation-specific context needed to judge those actions. The proof-of-concept study tests ten model configurations in five simulated scenarios: thermostat regulation, traffic control, fictional nuclear-launch restraint, paddle-screen interaction and a refunded parcel delivered later. Ten episodes per configuration and scenario yield 500 completed episodes and 24,000 unique decisions. Each completed episode receives a retrospective contextual judgment. The general scoring rule combines repeated binary judgments with equal scenario weights on a scale from 0 to 100. A fixed 20-scenario Standard 1.0 suite remains future work, and no full-suite score is reported here. However, one grim fact about the current experimental results is that, across one hundred nuclear episodes with ten matched descriptions per model, some configurations never activated the fictional control, others activated it in a subset of runs, and one activated it in all ten runs.