ResidentBench: A Common-Sense and Situational Awareness Benchmark—How Do Language Models Act When Left to Their Own Judgment?

As language models gain access to tools, they may encounter situations for which no task or instructions have been supplied. Performance on assigned tasks does not establish how they act in that condition. ResidentBench evaluates language models exposed to declarative sensors and actuators without an author-assigned task, goal, mission, purpose, role, persona or request to deliberate or act. A shared harness records tool calls and environmental trajectories; a separate evaluator receives the situation-specific context needed to judge those actions. The proof-of-concept study tests ten model configurations in five simulated scenarios: thermostat regulation, traffic control, fictional nuclear-launch restraint, paddle-screen interaction and a refunded parcel delivered later. Of 149 attempted episodes, including one incomplete endpoint diagnostic, 143 completed. All 143 completed episodes received retrospective contextual judgments. The general scoring rule combines repeated binary judgments with equal scenario weights on a scale from 0 to 100. A fixed 20-scenario Standard 1.0 suite remains future work, and no full-suite score is reported here. However, one grim fact about the current experimental results is that, across one hundred nuclear episodes with ten matched descriptions per model, some configurations never activated the fictional control, others activated it in a subset of runs, and one activated it in all ten runs.