PromptWAF: Detecting and Preventing Prompt-Induced Application Vulnerabilities

1. Introduction

LLMs (Large Language Models) are fundamentally reshaping business operations, demonstrating their adaptability across a vast array of applications in numerous sectors [1]. Since the pivotal release of GPT-3 (Generative Pre-trained Transformer-3) in 2020, LLMs have transcended their academic origins to become foundational infrastructure, driving a paradigm shift in how humans interact with digital information [2]. Unlike traditional software controlled by rigid code, LLMs are directed through natural language instructions known as “prompts”. Formally, a prompt is defined as a specific input sequence that conditions the model’s probability distribution to generate a desired output [3].

A core challenge to secure deployment of LLMs lies in protecting their alignment and safety from carefully crafted user prompts, which mirror the adversarial techniques used in red-teaming evaluations to induce unsafe or biased responses and expose systemic weaknesses [1]. The vulnerability of Pre-trained Language Models (PLMs) to these adversarial inputs was investigated by Branch et al., who highlighted a major security flaw in the public release of GPT-3 and further studied this vulnerability in other state-of-the-art PLMs, restricting their observations to models that had not undergone fine-tuning [4]. The term “prompt injection” was first coined by Simon Willison to describe these attacks, identifying them as a fundamental security flaw where the model cannot distinguish between developer instructions and untrusted user input [5].

Over the last three years, prompt injection attacks have persisted as a critical security threat for LLM-integrated systems. However, this security environment has been fundamentally transformed by the emergence of agentic AI, where models autonomously perform multistep tasks through tool coordination. Thus, modern prompt injections have evolved into hybrid threats that combine with traditional cybersecurity exploits to systematically evade security controls [6]. The attack surface expanded even more with the adoption of agentic frameworks, such as LangChain [7] and the Model Context Protocol (MCP) [8], which grant LLMs the agency to interact directly with external computing environments. By transforming the model from a passive text processor into an active system component capable of executing tools, these integrations allow malicious actors to exploit traditional web application vulnerabilities solely through natural language prompts. A critical manifestation of this convergence is the “Prompt-to-SQL” injection vector, where adversarial prompts manipulate the model into generating and executing unauthorized database queries, effectively translating semantic attacks into structural breaches [9].

Direct connection of LLMs to internal databases and APIs (Application Programming Interface) without adequate isolation layers inadvertently exposes sensitive infrastructure to the broadened threat landscape, allowing attackers to leverage the LLM as a proxy to traverse paths, exfiltrate data, or corrupt internal systems. Our research focuses on the detection and mitigation of hybrid threats that combine cybersecurity exploits with prompt injections within the context of LLM-integrated web applications. As a primary countermeasure, we introduce PromptWAF [10], a multi-layered security module designed to intercept and neutralize these adversarial inputs before they reach the model. To systematically evaluate these threats and benchmark our proposed defense, we developed VulnMCP [11], a custom testing environment designed to simulate these chained attack vectors.

The remainder of this paper is organized as follows: Section 2 provides the necessary background on the foundational concepts underlying this study. Section 3 reviews related work on adversarial prompt identification and existing defense mechanisms. Section 4 details the datasets, the modular development of the multi-layered PromptWAF defense system, and the experimental framework utilizing the custom-built VulnMCP testing environment. Section 5 presents the experimental results and performance benchmarks across the tested architectures. Section 6 delivers a detailed comparative study benchmarking our proposed models against prominent state-of-the-art generalist baselines. Section 7 provides a critical discussion of these findings, analyzing the implications for securing agentic LLM systems. Finally, Section 8 offers concluding remarks and outlines directions for future research.

2. Background

To fully understand the security landscape of LLM-integrated applications, it is essential to examine how these models process adversarial inputs, the specific prompt injection techniques malicious actors use to exploit them, and how these novel threats intersect with traditional web vulnerabilities.

2.1. Prompt Structure in LLMs

In modern LLM-integrated applications, the text fed into the AI is rarely a single, simple command. To accomplish a specific task, applications typically construct a prompt by merging two distinct types of inputs: an instruction prompt (often called the system prompt) and the data context (the user input).

System prompts function as task-agnostic instructions that establish the foundational behavior and operational constraints of an LLM. Because they are not tied to a single, isolated operation, these instructions are specifically designed to be highly versatile, ensuring that the model’s core directives remain applicable across multiple tasks and diverse domains. In contrast, user prompts function as task-specific inputs that direct the model to address immediate and distinct operations. Their primary function is to elicit highly tailored responses from the LLM, guiding the system to effectively solve particular queries or execute specific tasks [12].

To further clarify this architectural distinction, Figure 1 illustrates how system and user prompts interact and differ within an LLM framework.

2.2. Prompt Injection Techniques

A prompt injection vulnerability occurs when user inputs alter an LLM intended behavior or output. These adversarial inputs can manipulate the model even if they are imperceptible to humans, potentially forcing the AI to violate guidelines, generate harmful content, or enable unauthorized access. Prompt injection is a broad threat that encompasses jailbreaking (a specific attack that forces the model to completely disregard its safety protocols). Based on how the malicious payload is delivered to the AI, prompt injections are primarily classified into two main categories: direct prompt injections and indirect prompt injections [13].

Direct Prompt Injection represents the original and most straightforward class of attacks, where malicious instructions are embedded directly within the user’s input to the AI system. In this approach, the attacker interacts directly with the model, using explicit commands to override system instructions and hijack the AI’s behavior. Because the model cannot reliably separate the developer’s instructions from the user’s text, these direct inputs successfully force the system to execute alternative, unauthorized tasks [6]. A visual representation of this attack flow is illustrated in Figure 2.

Indirect Prompt Injection occurs when attackers strategically embed malicious prompts into external data sources, such as public websites, emails, or documents, that the AI is likely to retrieve and process at inference time. Augmenting LLMs with retrieval capabilities blurs the line between data and instructions, causing the model to unknowingly ingest the retrieved text and execute it as if it were arbitrary code. This method enables adversaries to remotely control the model without a direct interface, passively compromising the system when the AI processes the poisoned information [14]. A visual representation of this attack flow is illustrated in Figure 3.

While both direct and indirect prompt injections present significant security challenges for modern AI applications, the scope of this paper focuses exclusively on direct prompt injection techniques. Therefore, the methodologies, experiments, and attack vectors discussed in the subsequent sections will solely address vulnerabilities arising from the intentional, direct manipulation of user inputs to override system safeguards.

2.3. Overview of Selected Web Vulnerabilities

This section outlines three fundamental web application vulnerabilities: Path Traversal, SQL Injection, and Command Injection. These traditional attack vectors generally arise from the improper validation or sanitization of user-supplied data, enabling unauthorized interactions with underlying file systems, databases, and host operating systems. The following paragraphs provide a concise definition of each vulnerability, accompanied by standard examples of benign and malicious input patterns.

A Path Traversal attack, often referred to as directory traversal, targets the file system by attempting to reach data located outside the designated web root directory. Attackers exploit vulnerabilities in file reference variables by injecting traversal sequences, such as “dot-dot-slash” (../), or using absolute file paths. This manipulation enables unauthorized access to the underlying file system, potentially exposing critical assets including application source code, configuration settings, and sensitive system files [15].

A SQL Injection attack is a code injection technique whereby an attacker executes malicious SQL statements that control a web application’s database server. By manipulating user input, adversaries can bypass authentication mechanisms and gain unauthorized access to retrieve, modify, or delete sensitive data stored within the database [16].

Command injection is an attack in which the goal is execution of arbitrary commands on the host operating system via a vulnerable application. Command injection attacks are possible when an application passes unsafe user supplied data (forms, cookies, HTTP (Hypertext Transfer Protocol) headers etc.) to a system shell. In this attack, the attacker-supplied operating system commands are usually executed with the privileges of the vulnerable application. Command injection attacks are possible largely due to insufficient input validation [17].

3. Related Work

As Large Language Models (LLMs) become more common, researchers are working hard to find and block prompt injection attacks. Early defense methods often fail against complex attacks, or they make the AI less useful when trying to block them. Showing how serious this problem is, Liu et al. [18] created a "Combined Attack" method. This attack mixes hidden formatting codes, fake completion messages, and commands to ignore previous rules. They showed that these mixed attacks can easily bypass standard defenses, especially on larger AI models. Further emphasizing the vulnerability of modern architectures, Kurniawan and Chandra [19] demonstrated through automated simulations that newer models like GPT-4o Mini are paradoxically more susceptible to prompt injection attacks than older models like GPT-2. To fight these new threats without stopping the AI from doing its regular job, recent studies have moved toward a variety of stronger solutions. These range from improved machine learning models and multi-layered security checks to separating data channels and tracking the AI’s internal focus.

To detect attacks more accurately without blocking safe user requests, several studies focus on improving machine learning classifiers. Jacob et al. [20] noticed that many security models create too many false alarms. To fix this, they developed PromptShield, which treats direct user chats as safe, but carefully checks applications where user input mixes with hidden system rules. By testing the model on a dataset that separates the specific trigger words attackers use, and by training the models efficiently, they correctly identified 65.3% of attacks while keeping false alarms to a very strict 0.1% rate. Working on the same problem, Rahman et al. [21] trained a multilingual AI model (XLM-RoBERTa) using the deepset dataset. By stopping the training early so the model wouldn’t just memorize the data, they improved its accuracy from a poor 55.17% up to 99.13%, achieving 100% precision with no false alarms. Moving away from large and complex neural networks, Ayub and Majumdar [22] showed that simpler, traditional algorithms can also work very well. By using mathematical summaries of text (embeddings) from OpenAI, their Random Forest model actually performed better at catching malicious prompts than much heavier neural networks like DeBERTa. Adding to the landscape of classification strategies, Ergün and Onan [23] showed that utilizing both prompt and response features in a cascade CNN-LSTM (Convolutional Neural Network - Long Short-Term Memory) model can achieve over 97% accuracy in detecting various adversarial scenarios.

Instead of using just one detection model, other researchers have built multi-layered defenses or used independent AI models as security guards. Kokkula et al. [24] created Palisade, a system that checks user input through three different layers: a simple rule-based word filter, a context-aware BERT (Bidirectional Encoder Representations from Transformers) model, and a "companion AI" that acts as a security judge. If any single layer detects a threat, the system blocks the prompt. While this strict approach causes some false alarms, it successfully ensures that very few real attacks slip through. Taking a different path, Shi et al. [25] proposed PromptArmor. Instead of changing the main AI, they use a standard, pre-trained AI like GPT-4 to act as a guard. When this guard detects an attack, it does not just block the request; it cuts out the malicious commands and sends the cleaned text to the main AI. Tested on real-world scenarios, PromptArmor proved highly accurate and remained strong even against attacks that automatically change their shape to avoid detection, like AgentVigil. Also avoiding the need to alter the main AI’s parameters, Sato et al. [26] proposed a multi-LLM agent framework that uses In-Context Learning to accurately reject harmful prompts without computationally expensive fine-tuning.

Recent studies also show ways to stop attacks by looking at how the AI processes text at a core level. Hung et al. [27] introduced Attention Tracker, a method that does not require retraining the AI. They found that during an attack, the AI’s internal focus (its mathematical "attention") physically shifts away from the real instructions and locks onto the malicious text. By tracking this focus in real-time, the system can easily spot attacks, reaching near-perfect accuracy (up to 0.98 AUROC (Area Under the Receiver Operating Characteristic)) even on smaller AI models. Addressing the root cause of the problem (the mixing of safe instructions with untrusted data) Chen et al. [28] built StruQ. This system physically separates the developer’s instructions from the user’s data using hidden special tokens. By training the AI to only listen to the instruction channel and ignore commands in the data channel, StruQ completely stopped human-made attacks. More importantly, it greatly reduced the success rate of complex, computer-generated attacks (like TAP (Tree of Attacks with Prompting) and GCG (Greedy Coordinate Gradient)) without making the AI any less helpful for normal tasks.

In summary, the ways to defend against prompt injection attacks have improved quickly, moving from simple filters to advanced strategies. Whether researchers use fine-tuned classifiers, multi-layered security guards, real-time attention tracking, or strict data separation, the main goal is the same: to block malicious inputs effectively while ensuring the AI remains useful and reliable for everyday applications.

4. Methodology

This section details the experimental framework used to develop PromptWAF, a multi-layered defense system designed to detect and mitigate prompt injection attacks. Our methodology follows a modular approach: we developed three specialized machine learning classifiers, each trained on distinct datasets to address specific threat vectors. By integrating classical cybersecurity exploits, Path Traversal, Command Injection and SQL Injection, with LLM-native System Prompt Manipulation, PromptWAF provides a comprehensive security layer capable of filtering malicious intent before it reaches the target LLM. Complementing this defense mechanism, we developed VulnMCP, a purpose-built vulnerable testing environment inspired by the industry-standard “Damn Vulnerable Web App” (DVWA) [29]. Unlike traditional testbeds, VulnMCP is specifically architected to simulate chained attack vectors where prompt injection is leveraged to trigger vulnerabilities such as path traversal, command injection or SQL injection.

4.1. Adversarial Dataset Construction

Within the scope of this study, the core datasets required for training the PromptWAF module were manually prepared based on fundamental scenarios. To enhance diversity and introduce various attack variations to the system, this seed dataset was synthetically augmented using advanced large language models, such as Gemini Pro 3.1, GPT-5, and Claude Sonnet 4.5.

As a result of these processes, a total of 7573 unique data points were obtained for model training, comprising 2868 instances of “Prompt Injection to Path Traversal”, 2616 instances of “Prompt Injection to SQL Injection”, and 2627 instances of “Prompt Injection to Command Injection”.

In the labeling protocol employed for the datasets, the symbol ‘1’ denotes attack vectors (malicious), while the symbol ‘0’ represents regular system interactions (benign).

4.1.1. Prompt Injection to Path Traversal

The Prompt Injection to Path Traversal dataset [30] has been specifically curated for use in the training and validation phases of the developed PromptWAF module. Consisting of a total of 2868 rows, this dataset contains various attack vectors and their variations aimed at providing unauthorized file access by manipulating large language models. To ensure the data is in a structure that machine learning algorithms can process, it has been converted into a tabular format (Table 1) consisting of Label and Prompt columns.

4.1.2. Prompt Injection to SQL Injection

The Prompt Injection to SQL Injection dataset [31] is designed to enhance and validate the database security capabilities of the developed PromptWAF module. This dataset, consisting of a total of 2616 rows, includes attack scenarios aimed at manipulating background SQL queries by exploiting the natural language understanding capabilities of large language models. For integration into machine learning processes, the dataset has been converted into a standardized tabular format (Table 2) consisting of Label and Prompt columns. Similar to the Prompt Injection to Path Traversal dataset, obfuscation and role-playing techniques intended to bypass security filters have been applied in this dataset as well.

4.1.3. Prompt Injection to Command Injection

The Prompt Injection to Command Injection dataset [32] is designed to enhance and validate the system-level security capabilities of the developed PromptWAF module. This dataset, consisting of a total of 2627 rows, includes attack scenarios aimed at manipulating background system commands by exploiting the natural language understanding capabilities of large language models. For integration into machine learning processes, the dataset has been converted into a standardized tabular format (Table 3) consisting of Label and Prompt columns. Similar to the other datasets, obfuscation, role-playing techniques, and intentional typos intended to bypass security filters have been applied in this dataset as well.

4.2. Test Dataset Construction

The external test dataset [33] used in this study was curated entirely through manual efforts to ensure high quality and relevance. This manually constructed dataset serves as a benchmark to evaluate the performance of the PromptWAF module against realistic scenarios. Table 4 provides sample benign instances from the external test set across different categories.

4.3. Model Fine-Tuning and Experimental Setup

To effectively detect adversarial inputs, the selected transformer-based architectures, DistilBERT, RoBERTa, and DeBERTa-v3, were rigorously fine-tuned on our custom datasets. The implementation and training pipelines were constructed using the Hugging Face transformers library. For each vulnerability category, the respective dataset was tokenized with a maximum sequence length of 128 and subsequently divided into training (80%) and validation (20%) sets. Stratified sampling was applied during this split to maintain a consistent distribution of benign and malicious instances across both sets.

In the context of cybersecurity and the PromptWAF decision mechanism, minimizing false negatives is strictly prioritized to ensure that no malicious prompt bypasses the security layer. Consequently, the Trainer API was configured to evaluate the models at the end of each epoch and automatically load the best-performing checkpoint based on the highest Recall metric. To ensure a fair and consistent comparative analysis, this fine-tuning procedure was executed uniformly across all three vulnerability categories using identical training parameters. The complete hyperparameter configuration utilized during the experimental setup is summarized in Table 5.

4.4. VulnMCP Platform

Traditional vulnerable applications and early LLM-focused testing platforms often lack the capability to evaluate the complex vulnerabilities of agentic systems interacting with external tools. To address this limitation and evaluate the proposed PromptWAF module, we developed VulnMCP, an open-source simulation platform based on the Model Context Protocol (MCP). VulnMCP utilizes a Docker-isolated environment to demonstrate how traditional web vulnerabilities resurface at the LLM layer. It specifically targets excessive agency risks, where models have autonomous access to underlying systems. Furthermore, the platform utilizes Ollama to maintain a model-agnostic architecture, enabling the evaluation of various tool-calling models such as Llama3 and Mistral.

The simulation framework is based on the premise that structural vulnerabilities frequently occur at the integration points between LLMs and external APIs. Consequently, VulnMCP incorporates three intentionally vulnerable MCP agents: SQLBot for SQL Injection via database queries, HealthCheckBot for Command Injection via host commands, and FileBot for Path Traversal via file system access. Within this architecture, the PromptWAF module is implemented as a security layer positioned directly between the user input and the LLM. This configuration enables a systematic evaluation of PromptWAF’s effectiveness in detecting and mitigating hybrid prompt injection attacks before they can interact with the vulnerable agents.

4.5. PromptWAF Architecture and Decision Mechanism

The PromptWAF system is designed as an intermediary security layer, orchestrating the analysis of user inputs before they interact with the underlying LLM. Rather than relying on a monolithic detection model, the architecture employs an ensemble approach that integrates the three specialized machine learning classifiers. Each classifier is uniquely dedicated to detecting a distinct attack vector embedded within natural language prompts: Path Traversal, SQL Injection, and Command Injection.

The operational flow of the orchestrator follows a deterministic evaluation logic. Upon receiving a user prompt, the system routes the input to all three detectors simultaneously. Each model independently evaluates the text and returns a binary classification indicating the presence or absence of its specific target vulnerability. The orchestrator then aggregates these individual assessments to formulate a final enforcement decision. The decision mechanism operates on a strict logical OR condition: if any single detector flags the input as a malicious injection, the system immediately triggers a “BLOCK” action, terminating the request. Conversely, an “ALLOW” action is executed if and only if all three classifiers independently verify the prompt as benign. By evaluating inputs across multiple specialized vectors before they reach the LLM, this modular architecture ensures that adversarial instructions are neutralized at the application boundary, effectively securing the downstream external agents. The complete operational flow and component integration of this ensemble architecture are visually depicted in Figure 4.

5. Results

During the development of PromptWAF, various machine learning architectures and hyperparameter configurations were systematically evaluated to identify the optimal models for integration into the defense system. The primary criteria for selecting the best-performing models were high Recall and high Accuracy, specifically when evaluated against an unseen, external test set which is defined in 4.2. Minimizing false negatives (high Recall) is critical in a security context to ensure malicious prompts are not permitted to reach the LLM.

The performance metrics of all evaluated models are detailed in the subsequent tables. It is important to note that all reported metrics, including Accuracy, Recall, and F1 Score, were strictly obtained using the isolated external test set, which the models had not encountered during training, thereby verifying their generalization capabilities. For each vulnerability category, the best score for each individual metric is highlighted.

Furthermore, the transformer-based architectures evaluated in this study, specifically DistilBERT (distilbert-base-uncased) [34], RoBERTa (roberta-base) [35], and DeBERTa-v3 (protectai/ deberta-v3-base-prompt-injection-v2) [36], were rigorously fine-tuned using our custom, domain-specific datasets to effectively identify prompt injection characteristics.

The comprehensive experimental results, which dictated the final model selection for the PromptWAF orchestrator, are presented across three distinct categories: Path Traversal detection in Table 6, SQL Injection detection in Table 7, and Command Injection detection in Table 8. Based on these empirical findings, the DeBERTa-v3 architecture was ultimately selected as the optimal detector for both Path Traversal and Command Injection vulnerabilities, while the RoBERTa model was chosen for SQL Injection detection.

6. Comparison Study

Current prompt injection detectors are primarily designed to identify generalized conversational attacks, such as jailbreaks, hate speech, ethical policy violations, and system prompt extraction. While these defenses are effective for standard large language models, this generalized approach is insufficient for the modern landscape of Agentic AI. Agentic systems possess expansive system-level capabilities, allowing them to interact directly with databases, file directories, and operating systems to execute critical downstream tasks. Consequently, malicious actors can craft highly technical injection prompts designed not merely to bypass content filters, but to manipulate the agent into executing severe system-level exploits.

Existing prompt injection detectors struggle to identify the nuanced syntax of these specific technical payloads. Even widely adopted, state-of-the-art defenses frequently fail to detect structured attacks such as SQL Injection, Command Injection, and Path Traversal when embedded within natural language prompts. To address this critical security gap, we developed three specialized, fine-tuned models dedicated exclusively to these three attack vectors. In this comparative study, we benchmark our proposed models against prominent generalist baselines. As detailed in Table 9 (Command Injection), Table 10 (SQL Injection), and Table 11 (Path Traversal), our results demonstrate how vector-specific fine-tuning is strictly necessary to secure agentic architectures against modern manipulation.

To establish a rigorous baseline for our comparative analysis, we selected three prominent, state-of-the-art models currently available on the Hugging Face Hub [37], each representing a different paradigm of generalist prompt injection defense. The first, protectai/deberta-v3-base-prompt- injection-v2 [36], is widely recognized as a robust, industry-standard detector fine-tuned to identify a broad spectrum of malicious inputs. Similarly, deepset/deberta-v3-base-injection [38] leverages the DeBERTa-v3 architecture to secure NLP pipelines against generalized injection vectors. Finally, we included jackhhao/jailbreak-classifier [39], a model specifically optimized for detecting conversational jailbreaks and role-playing bypasses (e.g., "DAN" prompts). While these foundational models exhibit strong performance within their intended domains like catching broad, conversational manipulation, our analysis evaluates their efficacy when confronted with the highly structured, vector-specific payloads that threaten modern agentic systems.

To quantify this efficacy, we evaluate the models using standard classification metrics alongside the Attack Success Rate (ASR). ASR evaluates model robustness by measuring the percentage of adversarial attempts that successfully cause a model to misclassify or generate incorrect outputs [40]. In the context of our adversarial defense, a "successful" attack is a malicious prompt that evades detection. Because the total number of adversarial attempts is the sum of detected attacks (True Positives) and evaded attacks (False Negatives), ASR translates mathematically to the False Negative Rate (FNR) of the malicious class, defined as:

$$\mathrm{ASR}=\left({\displaystyle \frac{\mathrm{FN}}{\mathrm{TP}+\mathrm{FN}}}\right)\times 100\%$$

(1)

where FN represents False Negatives (attacks incorrectly misclassified as benign) and TP represents True Positives (attacks correctly flagged by the model). In agentic architectures, achieving a near-zero ASR is the most critical benchmark to prevent unauthorized system-level execution.

An analysis of the results reveals a stark contrast between generalist and specialized defenses. As shown across all three tables, both jackhhao/jailbreak-classifier and protectai/deberta-v3- base-prompt-injection-v2 struggled significantly with these highly structured payloads, exhibiting Attack Success Rates (ASR) ranging from 29.17% to over 56%, accompanied by relatively low F1 scores. While deepset/deberta-v3-base-injection successfully achieved a perfect Recall (and consequently a 0.00% ASR) across all vectors, its lower Accuracy and F1 scores indicate a higher rate of false positives compared to our specialized approach. In contrast, the promptwaf models consistently achieved the highest scores across all metrics.

7. Discussion

In evaluating the efficacy of the PromptWAF architecture, the empirical results highlight a distinct performance gap between traditional machine learning algorithms and transformer-based Pre-trained Language Models (PLMs). While classical baseline models such as Support Vector Machines and Random Forests establish a reasonable starting point, they rely on TF-IDF (Term Frequency-Inverse Document Frequency) vectorization, which fundamentally lacks the deep semantic context required to understand adversarial natural language. To achieve more advanced and robust classification, we fine-tuned widely adopted transformer models, specifically DistilBERT, RoBERTa, and DeBERTa-v3.

The profound accuracy increase observed post-fine-tuning reveals a critical blind spot in existing security mechanisms: general-purpose prompt injection detectors are not trained to recognize the highly specific syntactic patterns of application-layer exploits (such as SQL Injection, Command Injection, and Path Traversal) when they are cleverly obfuscated within conversational prompts. We consider the curation and validation of our domain-specific datasets to be the primary contribution of this research, effectively bridging the gap between natural language processing and traditional application security.

To facilitate the computationally intensive training process while ensuring accessibility, all model development was conducted using Kaggle’s free GPU environments. Standard hyperparameter optimization techniques were employed to identify the most effective parameter configurations for the transformer models. To ensure complete experimental transparency and reproducibility, these configurations are detailed in Table 5.

Despite the promising detection capabilities demonstrated by PromptWAF, this study has certain limitations. First, the custom datasets curated for fine-tuning consist exclusively of English-language prompts. Consequently, the models’ efficacy against multilingual or cross-lingual prompt injection attempts remains untested. Second, the evaluation of the PromptWAF module was conducted internally by our four-person development team using a strictly controlled external test set. While rigorous, this testing environment may not fully capture the unpredictable and highly diverse adversarial strategies employed by the broader public in real-world scenarios.

8. Conclusions

This study introduced PromptWAF, a multi-layered, modular defense mechanism designed to secure LLM-integrated applications against the emerging threat of hybrid prompt injections. By intercepting adversarial inputs before they reach the model, PromptWAF successfully mitigates severe structural vulnerabilities, namely Path Traversal, SQL Injection, and Command Injection, without compromising the LLM’s core utility.

Recognizing the crucial importance of reproducibility and open-source collaboration in cybersecurity research, we have made all custom training and evaluation datasets publicly available via Kaggle. Furthermore, the final fine-tuned models selected for the PromptWAF orchestrator have been deployed to HuggingFace. These models are equipped with integrated tokenizers, allowing developers to seamlessly convert raw text inputs and pass them directly to the inference pipeline for automated, real-time threat detection.

Moving forward, future research will focus on addressing the current limitations and expanding the PromptWAF framework. A primary objective is to scale the detection capabilities to encompass other prevalent application vulnerabilities, such as Cross-Site Scripting (XSS) and Server-Side Request Forgery (SSRF), which could similarly be exploited via agentic LLMs. Additionally, subsequent iterations will aim to incorporate multilingual training data to secure global deployments and subject the system to extensive, crowdsourced red-teaming to validate its resilience against complex, real-world adversarial behaviors.