Tool and Agent Selection for Large Language Model Agents in Production: A Survey

Large Language Model (LLM) agents have demonstrated remarkable abilities to interact with external tools, functions, Model Context Protocol (MCP) servers, agents, and to take action on behalf of the user. Due to the fast-paced nature of the industry, existing literature does not accurately represent the current state of tool and agent selection. Furthermore, tool and agent selection in production has nuanced components not covered in experimental research. This work provides the first detailed examination of tool selection from a production perspective, distinguishing between the frontend layer where users interact with agents through buttons, slash commands, or natural language and the backend layer where retrieval, execution, orchestration, context engineering, and memory enable scalable reasoning. The paper contributes a unified taxonomy of modern tool and agent selection approaches spanning manual, UI-driven, retrieval-based, and autonomous methods. The backend covers dynamic tool retrieval, chunking, advanced RAG methods, context engineering, reinforcement learning, tool execution, human-in-the-loop processes, authentication, authorization, multi-turn tool calling, short- and long-term memory for tools, and evaluation. Finally, the paper identifies challenges in production components of both the backend and frontend and outlines promising avenues for research and development.