Best Administrative/IT Practices When Rolling Out Custom Agents

1. Introduction

At East Carolina University (ECU), Nurshatayeva et al. found that implementing a chatbot was especially beneficial toward first-generation and low-income college students who may not be as familiar with the complex financial and administrative systems of higher education (Nurshatayeva et al., 2021). They suggested that the chatbot was able to help with “summer melt” or the phenomenon of high school graduates accepted to college failing to enroll in the fall, likely because the chatbot was shown to help first-generation and low-income students successfully take out loans where they may not have otherwise. Generally speaking, agentic artificial intelligence typically shares components like perception, planning, execution, memory, reflection, orchestration, and interaction (Bandi et al., 2025). It stands to follow that it should be possible to establish a general set of guidelines and best practices when attempting to deploy custom agents on an administrative/IT level in educational spaces.

According to Kostopoulos et al., there are several challenges with implementing agentic AI in the education space, such as transparency, bias, data privacy, cost, and more, which manifest in everything from data ownership with 3rd parties to vendor lock-in Risks (Kostopoulos et al., 2025). A university or organization considering the deployment of a custom AI agent would have to navigate through numerous of these challenges on a much larger and complex scale. Lei Wang et al. expresses concern over the agentic hallucination such as security risks and ethical issues, but suggested incorporating human correction feedback directly into the iterative process (Wang et al., 2024). Institutions should also consider the risk of prompt injection as Gulyamov et al. showed that they could manipulate AI responses 90% of the time using Retrieval-Augmented Generation (RAG) poisoning (Gulyamov et al., 2026), which might be achieved through some form of social engineering to gain backend access to RAG databases.

1.1. Definitions

An artificial intelligence (AI) agent is a system that utilizes AI to “...adaptively perform complex tasks, use tools, and interact with its environment – for example, by creating files, taking actions on the Web, or delegating tasks to other agents – to pursue goals with little to no human oversight” (Bengio et al., 2026).

The user or end user is classified as the individual who directly operates the agent through inputs such as prompting and subsequently receives the final output after the agent completes any necessary actions. In the context of higher-education and this review, users typically refer to students at the institution, but may also include faculty, staff, prospective students, and unaffiliated individuals.

The interface refers to the front-end, user-facing layer, such as a website or application window, where a user can interact with an agent through inputs and subsequently receive agentic outputs. In the context of this review, the interface allows users to send any text input of their choosing that is not necessarily preprogrammed.

Knowledge Graphs - “organise data from multiple sources, capture information about entities of interest in a given domain or task (like people, places or events), and forge connections between them.” (The Alan Turing Institute, n.d.)

Retrieval-Augmented Generation (RAG) - “An architectural approach in which agents dynamically retrieve relevant external knowledge (e.g., documents or facts) from a datastore and condition LLM generation on this retrieved context to improve factuality and grounding.” (Raza et al., 2026)

Prompt injection is a group of security attacks that occur when an AI’s developer instructions are overridden by a malicious or misleading input, resulting in leaking of data or the execution of forbidden tasks.

Jailbreaking is a technique that is used to bypass an AI’s built-in guardrails and safety mechanisms to generate restricted outputs.

Hallucinations are responses generated by AI that present false, misleading, or entirely fabricated information as a fact.

Model inversion attacks is a privacy attack in which AI models are reverse-engineered in order to extract private or sensitive data. Through iterative prompts and intentionally designed queries, attackers can model AI outputs to reconstruct data.

Membership inference - “A membership inference attack repeatedly queries a model to determine if a given input was part of the model’s training. More specifically, given a trained model and a data sample, an attacker appropriately samples the input space, observing outputs to deduce whether that sample was part of the model’s training.” (IBM, 2025)

2. Security, Identity and Access Management

IAM or Identity and Access Management is especially relevant for enforcing digital identity in educational environments. Maintaining accurate identity checking is crucial for ensuring that only authorized actors have access to user and model data. Institutions can balance security and risk reduction through a combination of educating employees/users and maintaining a dependable IAM program to ensure proper access to data/agents (Mohammed, 2017).

2.1. Agentic Risks

There are several refined prompt injection techniques like GCG (Zou et al., 2023), AutoDAN (Liu et al., 2024), PAIR (Chao et al., 2025), and TAP (Mehrotra et al., 2024). Hofer et al. suggests how such methods, when optimized for a specific model, do well within the model itself, but struggles between models like transferring to GPT-5, suggesting that having black-box access to a model may serve some benefit (Hofer et al., 2026). Hofer et al. still reported achieving a 45.2% attack success rate for Universal TAP on Qwen showing that system/user injection prompts/jailbreaking attempts can be highly effective and dangerous. Agent access to third-party systems can further compound these issues through “confused deputy” attacks where a legitimate agent is maliciously manipulated into misusing such systems like emails, payment services, etc (Enkrypt AI, 2025).

Huang et al. bluntly explains how LLM’s often hallucinate, producing false, yet plausible responses when prompted (Huang et al., 2025). As LLM’s are the cornerstones of custom agents and are often used for information retrieval, the risk of confidently incorrect responses being returned by one’s custom agent should be strongly taken into account. Alansari and Luqman explored the risks of hallucinations and compiled evidence suggesting that even top-performing LLM’s often hallucinate specifically when their knowledge/training data is sparse, encouraging the LLM to make guesses to bridge gaps in its knowledge base (Alansari & Luqman, 2026).

In March 2016, Google released a chatbot named “Tay” onto the social media platform Twitter (rebranded as X in 2023) where it conversed with other users and picked up their language and mannerisms. Within 24 hours, Google shut down Tay due to its widespread racist, sexist, and anti-semitic comments (Victor, 2016). Wolf et al. argue that Google failed to do its due diligence when conducting this experiment and should have expected the bot’s offensive behavior when interacting directly with the public based on prior knowledge of computer ethics (Wolf et al., 2017). Governmental entities such as the National Institute of Standards and Technology have taken note of the unique risks AI poses for organizations, pointing out that the changing datasets that models are trained on and the influence of social dynamics by humans may affect its trustworthiness, and that without sufficient moderation, it can lead to detrimental consequences for organizations (Tabassi, 2023).

Bhosale et al. found that by retaining sensitive data for long-term periods, agentic AI runs the risk of preserving and exposing private or secret information across tasks, users, and sessions. By committing private information to memory, the agent becomes susceptible to privacy risks, such as model inversion attacks and membership inference. Because agents can operate with little human oversight, it creates new risks of data leakage because of the agents’ dynamic external interactions and long-term data retention features (Bhosale et al., 2026; Madkour et al., 2026). Baek et al.’s research revealed that data leakage poses risks to privacy even in non-adversarial settings, when the agentic AI is receiving benign requests (Baek et al., 2026).

Zhao et al. explores the risk of RAG poisoning through targeting Knowledge Graph Retrieval Augmented Generation methods (KG-RAG). Given that an attacker only has read and insertion-only write access to the knowledge graph (KG), they showed that the attacker could stealthily degrade accuracy and mislead LLM response generation. (Zhao et al., 2025). This is further supported by Zou et al. demonstrating that in a knowledge graph with millions of texts, they could achieve a 90% success rate by inserting only five malicious texts for each target question (Zou et al., 2025). Together, Zhao et al. and Zou et al. emphasize the need for ensuring that only authorized individuals have access to the knowledge graphs used for an agent’s RAG usage.

Furthermore, Wang et al. warns that RAG poisoning can also amplify preexisting biases. Their Bias Retrieval Reward Attack (BRRA) framework proved effective in gradually increasing model bias by injecting quality adversarial documents into external knowledge databases. RAG poisoning defense mechanisms largely focus on the detection and suppression of adversarial or inaccurate information, but does not prevent the contamination of bias. Their results demonstrate how poisoned RAG “not only significantly amplify the bias of large models but also severely worsen the unfairness of model outputs, revealing the deep connection between RAG system security and model fairness.” (Wang et al., 2026)

Finally, when deploying agents, all of the traditional risks with software are still present such as DDOS attacks, malicious server access, phishing, cross-site scripting, and many more risks that one would have to safeguard from.

2.2. Accountability

We suggest a cross-disciplinary proposal in which a technical team works with subject matter experts to deploy, maintain, and manage a given custom agent. For example, a team of experts might want to create a custom chatbot to answer questions from outsider individuals. The technical team could work to create a custom agentic flow utilizing RAG, guardrails, LLM’s, and more, while the subject matter experts could suggest LLM prompts within the flow, maintain RAG databases, and control IAM and user/model data after launch.

2.3. Permissions and Accessibility

During the initial development phase, we suggest that the technical team has sole access to IAM as well as the agentic flow. After any initial development is completed, responsibility can transition to subject matter experts who can control data sources for an agentic flow, model/log access, and any internal LLM prompt changes. Subject-matter experts should also maintain backend access to update RAG databases as they deem necessary.

3. Compliance

Educational bodies in the US with access to user/student data as a direct result of agent usage should be especially careful when it comes to what data they collect, process, and share with third party providers as to not violate FERPA. There are also numerous AI disclosure laws present in New York (New York State Senate, 2026), California (California Assembly Bill No. 853, 2025), and other states, which the agent will have to be compliant with. We recommend that educational bodies implement a basic AI transparency link or note on the user interface. Finally, we recommend that user interactions are compliant with the Americans with Disabilities Act (ADA) by following the Web Content Accessibility Guidelines (WCAG) established by W3C (Americans with Disabilities Act of 1990, 1990). The WCAG upholds ADA compliance through its four sections Perceivable, Operable, Understandable, and Robust, which each contain a tiered recommendation system for how vital each recommendation is (World Wide Web Consortium, 2023).

3.1. Data Governance

According to the Family Educational Rights and Privacy Act or FERPA, unless a student signs a waiver agreeing to waive their FERPA rights, educational agencies or institutions must be able to provide records, files, documents, and other materials containing information relating directly to a student and that are maintained by an educational agency in order to receive educational federal funds (Family Educational Rights and Privacy Act of 1974, 1974). Despite a lack of precedence in terms of what exactly constitutes a FERPA violation in the context of higher education agentic usage, FERPA violations seem likely to arise when dealing with personally identifiable information (PII) from a student such as name, address, email address, phone number, and other information. FERPA violation risks require educational institutions to either confirm that any students using their agents have waived their FERPA rights and agreed to share any student data needed with any/all third parties involved with the flow, or the more practical option of immediately discarding information including PII of any kind during the flow. For example, a FERPA complying university might consider using guardrails to identify PII in a user provided prompt and immediately discard the prompt without processing it any further, ensuring that the prompt that has been recognized to contain PII is not exposed to any third parties such as outside LLM’s or stored with traces/logs without some form of redaction.

3.2. Risk Management

When it comes to generating logs/traces of agents, one should record user input, a sequence of plans and actions taken by the agent, and final output in a way beneficial for human auditors (Chan et al., 2024; Oueslati & Staes-Polet, 2025). It is important to recognize that comprehensive logs/traceability can introduce severe privacy risks, which could lead to data leakage (Madkour et al., 2026). It can be especially beneficial to include some human oversight in a given agentic flow that might be triggered by quantitative/qualitative trigger points like some number of API calls or user requests beyond an agent’s scope (Oueslati & Staes-Polet, 2025). On trigger or regular interval, a subject-matter expert overseer can examine the logs to see what prompts users are sending to the agent and trace through the tools the agent accesses when returning an output.

4. Deployment

4.1. Rate Limiting

Founding engineer Teja Kusireddy at GetOnStack explained how two of his four autonomous AI agents had gotten stuck in a clarification loop running for 11 days without anyone noticing, resulting in a $47,000 API Bill (Kusireddy, 2025). This incident does a spectacular job underscoring the need for establishing formal token/request/session/cost budgeting for a given agent. Token usage typically peaks in accuracy at intermediate costs with higher token limits being wasted with redundant looping/overthinking (Bai et al., 2026). We recommend setting small, arbitrary token/request/cost limits and then tracing agentic requests using subsequent logs that one generates when testing. One should adjust limits by recognizing what requests are able to be successfully responded to and which are not satisfactory.

4.2. Retrieval Augmented Generation

Knowledge-based agentic hallucinations stemming from inaccurate/insufficient internal knowledge can be mitigated by implementing the three phases of RAG, which aids models by providing supplementary, accurate, and relevant knowledge it can be grounded in (Li et al., 2025). For this reason, we recommend maintaining secure access to RAG databases to avoid RAG poisoning. Additionally, RAG involves the agent trying to understand the user query in the pre-retrieval phase, retrieve the relevant information in the retrieval phase, and combine retrieved information with internal knowledge for the user in the post-0 phase (Li et al., 2025). For this reason, one must ensure that sensitive information is not included in the database as an agent may use it within a response, especially if given a well-crafted malicious prompt. Also, Lei Wang et al. suggested incorporating human correction feedback directly into the iterative process to combat hallucinations (Wang et al., 2024). This might be reflected as a thumbs up or down button with a message option on a chatbot alongside the implementation of RAG.

Choudhary et al. found that in RAG poisoning attacks, successful retrieval of poisoned data must first create a concentration of influence towards the poisoned data. Therefore, existing poisoning attacks on external knowledge databases are not designed to be stealthy and leave detectable traces. As a result, they introduced the Normalized Passage Attention Score (NPAS), “a metric that aggregates the attention weights assigned to tokens in each passage from the model’s response” (Choudhary et al., 2025). Pairing NPAS with an Attention-Variance Filter (AVF) removes passages receiving anomalous amounts of attention, filtering out potentially adversarial documents (Choudhary et al., 2025). Given the design of these malicious passages to achieve high success rates with low corruption, we recommend detecting and filtering out passages that disproportionately affect the generated output and setting similarity threshold values to 0.8 or 0.9 to detect outliers (Webster, 2024) and checking for anomalous volumes at embedding endpoints. Raw similarity scores should not be shared with clients. (OWASP, 2026)

Anthropic found as few as 250 malicious documents is enough to create a backdoor vulnerability, regardless of the size of the training data; the absolute number of poisoned documents determine the success of the attack, not the percentage (Souly et al., 2025). Backdoors can remain undetected through safety retraining, only triggering when a specific prompt activates them. To mitigate this risk, we recommend red teaming agents to detect any backdoors during every alignment cycle (OWASP, 2026). Additionally, Zhang et al. proposes a RAG Security Bench (RSB) that tests RAG poisoning attacks and their defense mechanisms on three adversarial objectives: targeted poisoning, denial-of-service (DoS), and trigger-based DoS. Using three metrics, accuracy (ACC), attack success rate (ASR), and the F-1 Score (Zhang et al., 2025).

We recommend tracking the provenance of every embedding by keeping a record of obtained data so that any corrupted material can be cryptographically traced and invalidated (Alt et al., 2024; Moia et al., 2026; OWASP, 2026). Training data should also stem from a wide range of sources, obtained at different time intervals in order to reduce the effects of poisoned data from compromised sources (Alt et al., 2024; Moia et al., 2026). The National Institute of Standards and Technology (National Institute of Standards and Technology, 2026) have developed evaluation probes, “automated tools that are integrated directly into an agentic workflow to act as adversarial verifiers. The results of these evaluations are accumulated into a machine-readable audit trail that helps users assess agent actions and outputs.” (National Institute of Standards and Technology, 2026). When developing agentic AI for institutional use, it is imperative that agent decisions can be traced back to their supporting sources. Agentic outputs should also include citations in their outputs whenever internal documents are referenced to fulfill a request, as this can help draw attention to any RAG poisoning present. These citations, along with any contextual prompting history, should be included in traces to aid subject-matter experts when auditing model responses (Nian et al., 2026).

4.3. Guardrails

In order to ensure agents avoid processing PII, terminate prompt injection attempts, avoid embarrassing outputs, and similar, we strongly recommend using guardrails within one’s agentic flow. Chennabasappa et al., was able to show that their lightweight guardrail PromptGuard 2 lowered attack success rates to just 3.3% (Chennabasappa et al., 2025), which strongly demonstrates the notion that guardrails are a crucial line of defense for preventing injection prompts from being processed by an agent. Following these results, we recommend 1) prompt preprocessing – where agent guardrails scan the prompt for any PII or injection prompt attempts and terminates early if detected and 2) post-processing - where agent guardrails scan the agent output for any private PII or embarrassing outputs and discarding the output if detected. Prompt preprocessing is especially useful for avoiding issues associated with FERPA as one can avoid issues associated with storing, processing, and sharing sensitive student information by terminating the request as soon as PII is detected by the agent.

5. Metrics

5.1. Efficiency and latency

An AI agent needs to be able to execute commands efficiently “with minimal computational or financial cost. [Efficiency] reflects system responsiveness and resource optimization, both essential for real-world deployment” (Farooq et al., 2026). Latency is critical for interactive agents. In this context, a standard metric used is Time to First Token (TTFT), which measures the length of time it takes for a user in an interactive application to see the first piece of generated output (Suruliraj, 2026). This metric assesses the AI’s responsiveness and resource optimization. (Farooq et al., 2026) A system with a high TTFT appears to be less responsive, degrading the user experience, while a low TTFT will give the interaction a real-time, conversational feel (Suruliraj, 2026; Mohammadi et al., 2025).

End-to-end latency measures the total time the agent takes to complete a task, encompassing the entire interaction. However, while latency is an important reflection of AI performance and a contributor to a better user experience, users prefer reliable output generation over fast and inaccurate outputs (Singh et al., 2025), so it is crucial to find the best latency speed that balances speed and output quality.

Cost can also measure computational expense, as many agents rely on API calls, so monetary cost of an execution can be measured by API cost per task completion, either in cost or token usage, or the average number of model queries (Farooq et al., 2026; Kehkashan et al., 2026).

5.2. Agency/Tool use

Because AI agents have a significant amount of agency when it comes to task execution, not only must the task be performed with accuracy, but the proper handling of sensitive information is also crucial. The agent must be able to determine the appropriate information and tools to use for the given task. Metrics such as Tool Invocation Success Rate, the percentage of invocations retrieved successfully, and Tool Selection Accuracy, which assess the rate that the agent correctly selects the appropriate tool, are used to ensure the sensitive information isn’t accessed in conditions that don’t require them (Schrameyer, 2025; Farooq et al., 2026).

5.3. LLM-as-Judges

When Anthropic’s team attempted to test the feasibility of human alignment auditing, they discovered several limitations: Auditors vary in skill and experience, and scalability is difficult as gathering a sufficient number of expert auditors is unrealistic (Bricken et al., 2025).

LLM-as-Judges is a framework that replaces traditional human and statistical metrics with LLM evaluation. This system presents several advantages over preexisting methods: LLMs can adjust their evaluation criteria to fit a specific context, giving it more adaptability towards specialized systems, and it can generate more comprehensive feedback. LLMs additionally offer scalability without drastically increasing cost, as would traditionally occur with human evaluation (Li et al., 2024). However, because LLMs are sensitive to prompt design, they exhibit inconsistent decision and implicit biases, affecting evaluation consistency (Li et al., 2024).

6. Discussion and Conclusion

When deploying custom agents in educational bodies, one should consider the risks associated with doing so such as prompt injection attacks (Chao et al., 2025; Liu et al., 2024; Mehrotra et al., 2024; Zou et al., 2023), hallucinations (Alansari & Luqman 2026; Huang et al., 2025), embarrassing responses, data leaks (Bhosale et al., 2026; Madkour et al., 2026), RAG poisoning, and all other traditional software risks such as DOS/DDOS, malicious server access, phishing, cross-site scripting, and more. Beginning with the development of the agent, a clear accountability structure should be established. For example, we propose that a technical team maintains sole responsibility during the development phase and subsequently passes the agent and responsibilities to subject matter experts to maintain the accuracy of the agent and review logs/traces. There are several legal considerations that educational bodies should consider such as FERPA violations, AI Disclosure laws, and ADA accessibility. When it comes to FERPA, educational bodies should be especially careful with how they choose to handle student user PII as processing, sharing with third parties, and even storing traces with unredacted PII could result in FERPA violations. Beyond following these laws, we recommend including agentic guardrails to protect against prompt injection, embarrassing responses, data leaks, and the accidental processing of student PII that could result in a FERPA violation. Additionally, using a carefully curriculated and authenticated RAG database can protect against hallucinations and RAG poisoning. Furthermore, rate limiting tokens/requests/sessions/costs can be especially helpful for mitigating risks with typical DOS/DDOS attacks and general attacks/bugs. Finally, recording agentic metrics can allow subject-matter experts to trace and correct faulty agentic outputs and optimize an agent’s response by adjusting RAG databases, token limits, available tools, and more.