A Review of Small Language Models for Code and Domain-Specific Language Artifact Generation

1. Introduction

In recent years, language models have become a key tool for automating software development. Since the first results of Codex [1], the industry has moved toward open code generators such as Code Llama [2], DeepSeek-Coder [3], and Qwen2.5-Coder [4], which achieve high scores on standard code-generation benchmarks. GitHub Copilot, Amazon CodeWhisperer, and Cursor are built on this model ecosystem. In parallel with the development of LLMs, small language models (SLMs, ≤ 7B parameters) have been actively improving. The Phi series [5] showed that high-quality data allow compact LLMs to compete with larger models. For example, on HumanEval pass@1, Qwen2.5-Coder-1.5B [4] outperforms Code Llama-7B [2], which confirms substantial progress in SLM capabilities for this class of algorithmic tasks.

Most research focuses on general-purpose programming languages (GPLs), such as Python, Java, and C++. In practice, however, a large part of infrastructure is described using domain-specific languages (DSLs): Kubernetes manifests, Terraform HCL templates, OPA/Rego policies, and iptables rules. Many of the DSLs considered in this paper are more formalized and more suitable for automated verification than GPLs. They also operate within narrow domains, which makes SLM-based generation practically attractive. At the same time, hidden errors in such specifications may lead to large-scale IT failures and security incidents. A classic survey of configuration errors [6] and more recent cloud-security reports confirm the persistent importance of this problem: the Google Cloud Threat Horizons H2 2024 report identifies misconfiguration as 30.3% of the initial-access vectors considered significant for the first half of 2024 [7]. Meanwhile, the rapid growth of the DevOps ecosystem [8,9] increases the shortage of experts able to work effectively with the full diversity of modern DSLs.

Cloud LLMs for DSL generation, accessed through APIs, are limited by high cost and confidentiality risks. For critical information infrastructure, sending data to the cloud often violates regulatory requirements, while deploying private LLMs remains expensive and complex. Domain SLMs partially address this problem through epistemic isolation, that is, restricting the model to the target DSL. This means that the model is trained and used so that it relies primarily on knowledge of the target language, its schemas, typical patterns, and validators. This reduces the risk of irrelevant generation. The economic efficiency of local SLMs is also important: in a number of scenarios, generation cost can be substantially lower. In addition, small models are suitable for edge scenarios, where inference is performed closer to a workstation, industrial node, or internal organizational perimeter. A comparison of the main characteristics is given in Table 1.

The goal of this paper is to analyze SLMs for DSL artifact generation. The following tasks are addressed:

• Assess the potential of SLM architectures for DSL tasks.
• Propose a DSL classification that determines adaptation requirements.
• Analyze the effectiveness of SLM adaptation methods (PEFT, distillation, GCD, RAG).
• Assess limitations of current benchmarks.
• Systematize promising research directions in the field.

The contribution of the paper is an SLM review focused on DSLs, a new DSL classification (C1/C2/C3) by context-model complexity for selecting an adaptation strategy, a systematization of adaptation methods, and seven priority research directions. The paper is review-analytical in nature. Experimental validation of the proposed classification and adaptation methods is planned as future work.

2. Language Models for Code Generation: From LLMs to SLMs

2.1. Evolution of Code Language Models

The development of language models for code generation has gone through several stages. Early RNN- and LSTM-based approaches [10] enabled local autocompletion but had limited contextual understanding. A breakthrough came with transformer architectures [11] and pretrained GPT-2/GPT-3 models [12]. The capabilities of large transformers for competitive-level code generation are reviewed in [13].

In 2021, Codex [1], a GPT-3 variant fine-tuned on 54 million GitHub repositories, established the first widely recognized benchmark for code generation, solving 28.8% of HumanEval tasks. It was followed by AlphaCode [14], which used massive sampling and filtering, and CodeGen [15], which demonstrated the advantages of multi-stage training from natural language to multilingual and monolingual code processing.

Generative model capabilities expanded sharply in 2023-2024 with the release of open weights:

• StarCoder [16] and StarCoder 2 [17], trained on The Stack v1/v2, one of the largest open corpora covering more than 600 programming languages.
• Code Llama [2], a family of 7-34B models based on Llama 2 and additionally trained on code.
• DeepSeek-Coder [3], models from 1.3B to 33B parameters trained on 2 trillion code tokens using fill-in-the-middle (FIM).
• Qwen2.5-Coder [4], a model family whose 1.5B version outperforms Code Llama-7B on HumanEval.

This evolution (Figure 1) shows that the largest quality gains come not only from increasing model size, but also from improving training strategies and data selection. Applications of LLMs to code-generation tasks are reviewed by Lomshakov and Nikolenko [18].

2.2. Architectures of Modern SLMs for Code Generation

Most modern SLMs for code generation use a decoder-only transformer architecture: each token is generated conditioned on all previous tokens. Key architectures are compared in Table 2.

DeepSeek-Coder-V2-Lite uses a Mixture-of-Experts architecture; although it has 16B total parameters, only 2.4B are activated per token, making its inference compute comparable to an SLM. HumanEval scores are reported from authors’ publications or model cards and are not always directly comparable across base, instruction, and chat variants.

Three architectural properties are especially important for DSL generation.

Context window. Newer models such as Phi-3 [19] and Qwen2.5-Coder [4] support context windows up to 128K tokens, enabling extensive DSL context to be placed directly in the prompt. However, effective use of long context remains problematic for SLMs ≤ 3B: generation quality decreases as input length grows [23].

FIM. DeepSeek-Coder and StarCoder 2 support fill-in-the-middle generation, which allows a model to fill gaps in existing code. For DSLs, this enables generation of individual configuration sections while preserving surrounding context.

Encoder-decoder / decoder-only. Encoder-decoder architectures such as CodeT5+ are theoretically better suited for translating natural language (NL) into DSL, because they explicitly separate input understanding and output generation. However, decoder-only models show stronger results on major metrics due to large-scale training data. For Russian-language tasks, the work by Zmitrovich et al.  [24] is relevant; it describes the ruBERT, ruGPT-3, ruT5, and FRED-T5 model families and their adaptation to specialized tasks.

2.3. Training Data: DSL Coverage

A central challenge for DSL generation is the highly uneven distribution of languages in training corpora. The largest open corpus, The Stack v2 [17], contains 67.5 TB of data and covers 619 languages, but the distribution is strongly skewed toward general-purpose languages and mass formats (Table 3).

Although The Stack v2 [17] includes hundreds of languages and common markup formats (JSON, YAML), specific infrastructure and configuration DSLs occupy a much narrower niche than Python, JavaScript, Java, and other GPLs. As a result, even large models may show unstable out-of-the-box quality for DSL generation: such languages are less represented during pretraining, and correctness checking requires knowledge of schemas, policies, and runtime semantics. This makes targeted DSL adaptation practically necessary and indicates a potential advantage of SLMs: in a narrow domain, a deeply adapted small model can compete with a large general-purpose model.

2.4. Standard Benchmarks and Their Limitations for DSLs

Code-model quality is traditionally evaluated on several benchmarks:

• HumanEval [1], 164 Python tasks with functional tests; metric: pass@k.
• MBPP (Mostly Basic Python Problems) [25], 974 Python tasks for basic programming skills.
• MultiPL-E [26], a HumanEval extension to 18 programming languages.
• DS-1000 [27], 1000 data-science tasks focused on library use.
• CodeBLEU [28], a metric that accounts for syntactic and semantic code similarity.

All these benchmarks have a substantial limitation: they target only general-purpose languages, mainly Python, and do not include tasks for DSL artifact generation. The absence of a standardized benchmark for generating configuration files, security policies, infrastructure templates, and CI/CD pipelines remains an obstacle to comparing methods; this issue is discussed further in Section 6.

For some domains, validators can serve as the basis of a benchmark: kubeval and kubectl validate for Kubernetes, terraform validate for HCL, opa eval for Rego, actionlint for GitHub Actions, and iptables-restore --test for iptables. However, these tools check only syntactic and schema correctness, not whether the generated artifact matches the natural-language request.

3. DSL Classification from the Perspective of Language Modeling

One contribution of this paper is a classification of DSLs by context-model complexity, which determines the SLM adaptation strategy. The classification is based on formal properties of languages (grammar type, nesting depth, dependencies) and empirical observations about the adaptation methods required.

3.1. Definition and Basic DSL Classifications

A DSL is a programming or specification language with limited expressiveness, designed for a specific subject domain [29]. In Fowler’s classic definition [30], DSLs are contrasted with GPLs by breadth of applicability. Approaches to DSL development are reviewed by Koznov [31]. Terekhov, Bryksin, and Litvinov [32] describe the QReal platform for visual DSL modeling and code generation.

The following basic DSL classifications are used.

By form of definition

• External DSLs have their own syntax and parser (HCL, Rego, iptables rules).
• Internal/embedded DSLs are implemented on top of a host language (Kotlin DSL for Gradle, Ruby DSL for Chef).

By execution model

• Declarative DSLs describe a desired state (Kubernetes YAML, Terraform HCL).
• Imperative / rule-based DSLs contain logic, conditions, and decision rules (OPA/Rego, iptables).
• Configuration DSLs define system parameters (YAML configurations, INI files, .env).

By formal verifiability

• Formally verifiable DSLs have a strict grammar and/or validation schema (Kubernetes YAML -> OpenAPI schema, Terraform -> HCL parser).
• Partially verifiable DSLs allow syntax checking, but semantic correctness depends on the runtime environment (OPA/Rego).
• Free-form DSLs lack a strict formal specification (Markdown, arbitrary YAML).

3.2. Proposed C1 / C2 / C3 Classification

Based on DSL formal properties and generation complexity, a three-level classification is proposed (Figure 2). It is applied in nature and is intended to guide modeling, adaptation, and verification strategies for DSL artifacts, rather than to serve as a strict formal-language classification.

3.2.1. Class C1 - Regular DSLs

Formal characterization. DSLs whose specifications are described by regular grammars or minor extensions of them. Artifact structure is flat or has fixed nesting depth.

Properties

• Limited vocabulary of allowed keys and atomic values.
• No or minimal inter-object references.
• Deterministic validation: correctness checking reduces to matching a regular expression or a simple schema.
• Context-model size is minimal (tens of rules).

Examples..env files, INI configurations, linear iptables-save structures, simple requirements.txt files. (Note: GitHub Actions and Docker Compose, often treated as simple YAML, in practice have C2 properties because they rely on inter-object references and strict schemas.)

SLM requirements. Few-shot prompting or minimal adaptation. Grammar-constrained decoding (GCD) may be sufficient to ensure syntactic correctness. Expected generation accuracy for SLMs ≤ 3B is high (expert estimate; direct experimental data for a broad set of DSLs are absent).

3.2.2. Class C2 - Context-Free DSLs

Formal characterization. DSLs whose grammars are context-free or extensions of context-free grammars. They are characterized by hierarchical structures with arbitrary nesting depth.

Properties

• Hierarchical structure (nested objects, lists, blocks).
• Inter-object references (resource names, labels, selectors).
• Value typing (integers, strings, enums, references).
• Validation requires AST construction and schema checking.
• Context-model size is medium (hundreds of rules, dozens of object types).

Examples. Kubernetes YAML (Deployment, Service, RBAC), Terraform HCL, Docker Compose, Helm charts, GitHub Actions workflows with needs dependencies.

SLM requirements. QLoRA/LoRA fine-tuning + DSL context model + grammar-constrained decoding (GCD) + schema validation. Expected quality for SLMs ≤ 3B is medium to high if a domain corpus and validators are available.

3.2.3. Class C3 - Context-Sensitive DSLs

Formal characterization. DSLs whose expression correctness depends on context (semantic dependencies, recursion, aggregations). They may have computational expressiveness similar to Turing-complete languages, although they are often deliberately limited (Turing-incomplete) to guarantee termination, as in OPA/Rego.

Properties

• Recursive and aggregating constructs.
• Semantic dependencies resolved only in a runtime environment (references to external data, query results).
• Logical expressions with variable substitution.
• Validation requires not only parsing but also interpretation/evaluation.
• Context-model size is significant (hundreds of rules, semantics + examples + runtime dependencies).

Examples. OPA/Rego (recursion, aggregation, virtual documents), SQL subsets for configuration, Dhall (a fully typed functional DSL), CEL (Common Expression Language).

SLM requirements. Full fine-tuning or QLoRA with an extended context model + semantic verification. Expected quality for SLMs ≤ 3B is the least stable among the three classes and strongly depends on coverage of runtime semantics.

3.3. Summary of DSL Classes

Table 4. DSL class characteristics.

Characteristic	C1 (Regular)	C2 (Context-free)	C3 (Context-sensitive)
Grammar	Regular	Context-free	Context-sensitive
Nesting	Fixed (0-2)	Arbitrary	Arbitrary + recursion
Links	None	References, selectors	Semantic + runtime environment
Validation	Regex / schema	AST + schema	Parser + interpreter
Adaptation method	Few-shot prompting	QLoRA + GCD	Full fine-tuning
Expected quality (≤ 3B, expert estimate)	High	Medium / high with validation	Unstable, semantics-dependent
Typical examples	.env, INI	Kubernetes, Terraform, Docker Compose	OPA/Rego, SQL, Dhall

Table 4. DSL class characteristics.

Characteristic	C1 (Regular)	C2 (Context-free)	C3 (Context-sensitive)
Grammar	Regular	Context-free	Context-sensitive
Nesting	Fixed (0-2)	Arbitrary	Arbitrary + recursion
Links	None	References, selectors	Semantic + runtime environment
Validation	Regex / schema	AST + schema	Parser + interpreter
Adaptation method	Few-shot prompting	QLoRA + GCD	Full fine-tuning
Expected quality (≤ 3B, expert estimate)	High	Medium / high with validation	Unstable, semantics-dependent
Typical examples	.env, INI	Kubernetes, Terraform, Docker Compose	OPA/Rego, SQL, Dhall

3.4. Impact of DSL Class on Adaptation Strategy

The classification directly determines the SLM adaptation strategy.

• C1 DSLs often require only few-shot prompting or GCD over a formal grammar. Fine-tuning is often excessive.
• C2 DSLs are well suited to a combination of QLoRA + domain context model + GCD as a syntactic-control mechanism.
• C3 DSLs require full fine-tuning or QLoRA on a large corpus + semantic verification.

This classification is used in Section 3 and Section 4 to systematize adaptation and evaluation methods.

4. Methods for Adapting SLMs to Domain-Specific Tasks

This section systematizes the main adaptation methods for DSL artifact generation, including their advantages, limitations, and effectiveness for different DSL classes (C1/C2/C3).

4.1. Parameter-Efficient Fine-Tuning (PEFT)

Parameter-efficient fine-tuning (PEFT) trains only a small fraction of model parameters while keeping base-model weights frozen. For SLMs with limited compute resources, PEFT is a natural choice.

Low-Rank Adaptation (LoRA) [33] is the most widely used PEFT method. It inserts trainable low-rank matrices $\Delta W=BA$ (where $B\in {\mathbb{R}}^{d\times r}$, $A\in {\mathbb{R}}^{r\times k}$, and $r\ll min(d,k)$) into transformer attention layers. With a typical rank r = 8-64, trainable parameters account for 0.1-1% of all parameters, allowing models to be fine-tuned on a single GPU with 8-16 GB VRAM.

QLoRA [34] extends LoRA by quantizing the base model to 4-bit NF4 while keeping adapters in full precision. Models with 7B parameters can be trained on a GPU with 8 GB VRAM. The efficiency of LoRA adaptation for DistilBERT in text classification is analyzed by Abodo Elundu and Quanyu [35].

Adapter modules [36] are an alternative approach in which small trainable bottleneck blocks are inserted between transformer layers. With 2-5% additional parameters, adapters achieve quality comparable to full fine-tuning on many NLU tasks.

Prefix tuning [37] and prompt tuning [38] add trainable virtual tokens to the beginning of the input sequence, allowing the model to adapt to a domain without changing its weights. These methods are the least resource-intensive but are inferior to LoRA for code generation.

Applicability to DSLs. PEFT methods are highly effective for adapting to DSL domains with limited data (100-5,000 examples). However, the literature lacks a systematic comparison of PEFT methods on DSL tasks; existing benchmarks target GPLs. The optimal rank, target layers, and training strategy for different DSL classes remain open questions.

In practice, PEFT can be excessive for C1 DSLs, where baseline quality is often achieved by prompting and formal syntax control. For C2 and some C3 DSLs, QLoRA appears to be one of the most practical compromises between quality and resource cost.

4.2. Knowledge Distillation for Code Models

Knowledge distillation [39] transfers knowledge from a large teacher model to a compact student model. For DSLs, distillation provides a direct path to synthetic data: an LLM such as GPT-4 or Claude generates high-quality instruction-response examples, and an SLM is then fine-tuned on them.

Classical distillation trains the student on the teacher’s soft labels, that is, probability distributions over the vocabulary instead of a single correct token. In data distillation, however, the student is trained on teacher outputs in an instruction -> response format.

WizardCoder [40] used Evol-Instruct to evolve and complicate training instructions, achieving the best HumanEval results among open models at the time. This confirms that instruction quality is more important than quantity.

Magicoder [41] proposed OSS-Instruct, which generates training pairs from open-source code fragments and increases diversity and realism.

Self-training is an iterative process: the model generates its own training data, filters them through a verifier, and is fine-tuned on the filtered subset. For DSLs, distillation gains an important advantage: synthetic data from the teacher can be passed through a formal DSL validator (see Section 4.2), which substantially reduces the share of erroneous training examples.

A key limitation of many distillation methods is that they do not account for formal DSL properties. Knowledge verification during transfer from teacher to student is often limited to lexical and semantic metrics such as BLEU and CodeBLEU, whereas DSLs allow automatic checks through formal grammars, schemas, and semantic analyzers. This potential remains underused.

4.3. Grammar-Constrained Decoding (GCD) and Structured Decoding

Grammar-constrained decoding (GCD) is a class of methods that restrict generation to token sequences allowed by a formal grammar. GCD is particularly important for DSLs: a correctly specified grammar guarantees syntactic correctness at every generation step.

Outlines [42] is a structured-generation library that uses finite automata to constrain model output by a regular grammar or JSON Schema. At each decoding step, a mask of allowed tokens is formed, excluding syntactically invalid continuations.

LMQL (Language Model Query Language) [43] offers a declarative language for querying language models and specifying output-format constraints (types, ranges, enumerations) directly in the query.

GBNF grammars in llama.cpp provide a practical GCD implementation for GGUF models. GBNF (GGML BNF) can specify arbitrary context-free grammars applied during decoding, excluding invalid tokens before softmax computation. However, rigid grammatical constraints may narrow the model’s reasoning space. CRANE [44] addresses this issue by combining free intermediate reasoning with strict final-structure generation.

GCD formalization. Geng et al.  [45] provided a theoretical justification of GCD for structured NLP tasks and formalized conditions under which grammatical constraints do not degrade generation quality.

Applicability to DSL classes

• C1. GCD usually solves syntactic correctness. A C1 DSL’s regular grammar is easy to express in GBNF or Outlines.
• C2. GCD ensures syntactic but not always semantic correctness. For example, Kubernetes YAML that passes grammar checking may contain invalid references to nonexistent objects.
• C3. GCD is of limited use because context-sensitive properties, such as referential integrity and type constraints, cannot be expressed by a context-free grammar.

4.4. Retrieval-Augmented Generation (RAG) for Code

RAG [46] augments a language model with an external knowledge source by retrieving relevant documents or code fragments from an indexed corpus and including them in the generation context.

RepoCoder [47] implements iterative RAG for repository-level code completion by alternating retrieval and generation. The approach improves performance by 12-15% compared with a model without context.

For DSL generation, RAG has several domain-specific limitations.

No formal guarantees. RAG enriches the context but does not control output format or correctness. A model may retrieve relevant Kubernetes YAML examples but still generate an invalid manifest.

Dependence on retrieval quality. RAG effectiveness critically depends on embedding quality and index completeness. For small DSLs with limited example corpora, such as Rego or iptables, retrieval may be ineffective.

Redundancy for constrained domains. If a DSL domain is fully described by a compact specification (context model), a complex retrieval pipeline (vectorization -> index -> retrieval -> context expansion) may be unjustified: all necessary information fits into the prompt.

Model scale. RAG effectiveness depends not only on retrieval quality but also on the model’s ability to use added context. Long-context studies show that models may lose relevant information in the middle of the prompt [23], and this risk is especially important for small models (SLMs ≤ 3B).

Nevertheless, RAG remains practically valuable in hybrid architectures where an SLM generates an initial DSL artifact, a verifier checks it, and RAG retrieves relevant examples for correction when errors are found.

4.5. Prompting and In-Context Learning

Prompting is the least resource-intensive adaptation method because it does not modify model parameters. The model receives an instruction and a set of examples (few-shot) directly in the input sequence.

Zero-shot prompting supplies only the task description without examples. For C1 DSLs, modern SLMs >= 3B can often generate correct artifacts in zero-shot mode if the DSL is sufficiently common in training data (.env, INI). For common C2 DSLs such as GitHub Actions and Docker Compose, zero-shot generation is also possible but less stable.

Few-shot prompting includes 3-10 examples of (NL request, DSL artifact) pairs in the context window. It is effective for C1 and limitedly effective for C2. For C3 languages such as Rego and SQL, few-shot prompting is usually insufficient because a few examples do not reliably capture complex semantics.

Chain-of-thought (CoT) prompts the model to generate intermediate reasoning before the final answer. For DSL generation, CoT may include analyzing the NL request, identifying the resource type, selecting fields, and generating the artifact. However, for SLMs ≤ 3B, CoT often worsens results because the model spends its limited generation budget on reasoning rather than the artifact [48].

Gap. Prompting for DSLs is not systematized. Research is lacking on optimal few-shot prompting strategies for DSL classes (C1/C2/C3), the optimal number of examples, and example format.

4.6. Hybrid Approaches

In practice, the methods described above rarely solve the task alone; therefore, hybrid architectures are of particular interest.

QLoRA + GCD. The roles are separated: fine-tuning teaches the model domain patterns, while the grammar keeps decoding within valid structures.

SLM + RAG + formal verifier. The SLM generates a DSL artifact -> the verifier checks correctness -> if errors are detected, RAG retrieves relevant examples -> the SLM corrects the artifact using the new context. This approach combines SLM autonomy, RAG-based enrichment, and output checkability through verification.

A remaining challenge is the lack of a single scalable pipeline that integrates all critical components: domain context model, distillation, PEFT adapters, strict grammar-constrained decoding (GCD), and verifiers. Developing such a hybrid architecture (Figure 3) is a priority for future research.

4.7. Summary of Adaptation Methods

Table 5 summarizes methods for adapting SLMs to DSLs.

5. Verification and Evaluation of DSL Artifact Generation

The quality of DSL artifact generation is determined by syntactic correctness, semantic correspondence to the request, configuration security, and validation efficiency.

5.1. Metrics for Evaluating Code Generation

Existing metrics for code-generation quality can be divided into three categories.

Functional metrics evaluate code correctness through tests:

• pass@k [1], the probability that at least one of k generated variants passes all tests. This is the de facto standard for evaluating code models.
• Test-case success rate, the share of test cases passed by generated code.

Stylistic and lexical metrics compare generated output with a reference solution:

• BLEU [49], which estimates n-gram overlap between generated and reference text. BLEU is often inadequate for code because it ignores syntactic structure.
• CodeBLEU [28], an extension of BLEU that accounts for AST structure and data flow.
• CrystalBLEU [50], a modification that reduces the influence of frequently occurring tokens.

Expert and LLM-based metrics

• Manual expert evaluation, the most accurate but slow and expensive method.
• LLM-as-a-judge, using large models such as GPT-4 to evaluate the output quality of a student model.

5.2. Formal Verifiability as an Advantage of DSLs

Unlike GPLs, where full code verification is NP-hard, most DSLs have properties that allow fast and accurate automatic checking of generated artifacts. Methods of software formal verification are systematized in the Russian literature by Kulyamin [51].

Automatic output verification. Each considered DSL has mature industrial validators (Table 6).

These validators create three interrelated advantages.

Automatic filtering of training data. During synthetic generation of training pairs by an LLM teacher, a formal validator automatically filters out incorrect examples without manual annotation. This enables scaling to tens of thousands of pairs. Unlike BLEU or expert annotation, the validator returns a binary and objective answer: the artifact is correct or incorrect. The same tool also serves as the last line of control: an incorrect artifact is blocked before reaching production.

Feedback loop. Verification results can be used for self-training: the model generates -> the validator checks -> correct pairs are added to the training set -> the model is further fine-tuned. This loop is fully automated due to the formal nature of DSLs.

5.3. Existing Benchmarks and Their Limitations

A standardized benchmark for DSL artifact generation is still absent. Existing benchmarks target GPLs.

Table 7. Analysis of existing code-generation benchmarks.

Benchmark	Languages	Tasks	Metric	DSL
HumanEval [1]	Python	164	pass@k	No
MBPP [25]	Python	974	pass@k	No
MultiPL-E [26]	18 GPLs	~3000	pass@k	No
DS-1000 [27]	Python + libraries	1000	pass@k	No
APPS [52]	Python	10K	pass@k	No
ClassEval [53]	Python (OOP)	100 classes	pass@k	No
Spider [54]	SQL	10K	execution accuracy	Limited

Table 7. Analysis of existing code-generation benchmarks.

Benchmark	Languages	Tasks	Metric	DSL
HumanEval [1]	Python	164	pass@k	No
MBPP [25]	Python	974	pass@k	No
MultiPL-E [26]	18 GPLs	~3000	pass@k	No
DS-1000 [27]	Python + libraries	1000	pass@k	No
APPS [52]	Python	10K	pass@k	No
ClassEval [53]	Python (OOP)	100 classes	pass@k	No
Spider [54]	SQL	10K	execution accuracy	Limited

Spider for SQL is the closest analogue of a DSL benchmark, but SQL is a query language rather than a configuration DSL, and Spider tasks focus on NL -> query translation rather than NL -> configuration.

The absence of a DSL benchmark is one of the key open problems (Section 6), preventing objective comparison of approaches and reproducibility.

For infrastructure DSLs, correctness is not limited to passing a syntactic validator; quality evaluation must also account for security. A configuration may be formally valid while granting excessive access rights, violating isolation policies, or creating undesired network permissions. Therefore, the following section treats security as a separate requirement level for DSL artifact generation, complementing syntactic, schema, and semantic verification.

6. Security and Knowledge Management in Domain SLMs

Using SLMs to generate configuration DSL artifacts requires securing both the model and its outputs. An error in an iptables rule, an OPA/Rego policy, or Kubernetes RBAC can lead to data leakage, unauthorized access, or infrastructure failure.

6.1. Threat Model

Four main attack vectors are identified for local SLM deployments.

In prompt injection, an attacker embeds hidden instructions in the NL request, and the model generates intentionally vulnerable configurations such as open ports, wildcard policies, or privileged containers. Training-data poisoning occurs earlier, during fine-tuning. If training examples contain unsafe patterns, the model reproduces them as normal, especially when dataset verification is absent.

Extraction attacks. Carlini et al.  [55] demonstrated that training data can be extracted from language models; a later study [56] quantified memorization at scale. For domain SLMs, this risk is amplified by local storage of model weights on edge devices or workstations. Unlike closed cloud LLM APIs, local access to model files allows an attacker to run offline attacks to extract fragments of sensitive configurations (IP addresses, network structure) used during fine-tuning. Cloud-security issues in the Russian context are discussed by Bespalova [57].

Indirect injection and local-data poisoning. When an SLM is used in an IDE or CI/CD pipeline with access to neighboring project files, an attacker can insert a malicious instruction into comments of third-party code or into README.md. The SLM may treat this as part of the DSL-task context and generate a deliberately vulnerable artifact. Similarly, compromise of one internal repository used to fine-tune a corporate model may poison generation results for the entire organization.

6.2. Protection Methods

Formal verification as a defensive perimeter. Automatic checking of DSL outputs forms a structured defense perimeter. Multi-level validation (syntax -> schema -> semantics -> security policies) blocks most unsafe configurations before they are applied. Svace [58] is an example of a static analyzer that checks generated code for defects. Work [59] shows that integrating a static analyzer with Qwen2.5-Coder fine-tuning reduces the frequency of unsafe-code generation.

Adversarial training. Generating adversarial NL requests that provoke unsafe configurations and including them in the training set with correct DSL responses increases resistance to prompt injection. However, adversarial training for DSLs is poorly studied; existing work focuses on text tasks and GPLs.

Epistemic isolation as a barrier. Narrow specialization of domain SLMs, for example training primarily on Kubernetes YAML syntax, acts as a natural filter and reduces irrelevant responses. However, specialization does not eliminate attacks: the model may still mishandle malicious context if it appears in a request, documentation, or training data. Therefore, epistemic isolation should be treated as one defense layer rather than a standalone security guarantee. The threat model and layered defense architecture are shown in Figure 4.

6.3. Machine Unlearning and Epistemic Isolation

Machine unlearning [55,60] studies the removal or suppression of specific knowledge from a trained model. For DSL-oriented SLMs, it is relevant when a model has been fine-tuned on confidential organizational configurations and the risk of reproducing them must later be reduced.

Two main approaches are used:

• Gradient-based unlearning, applying gradient ascent to the subset of data that should be forgotten.
• Prompt-based unlearning, modifying system prompts and fine-tuning on instructions not to generate certain patterns.

In security contexts, epistemic isolation is not an absolute guarantee but a controlled restriction of the generation domain. The target model should know enough to solve tasks within the domain, but should not become a universal text generator.

Formal guarantees of such information isolation are still insufficient. Open questions include transparent metrics for such models, evaluation of residual out-of-domain knowledge, and methods of controlled forgetting for DSLs.

7. Priority Directions for Future Research

The analysis reveals three systemic limitations: the absence of a formal domain model, a shortage of specialized benchmarks, and the vulnerability of configurations to injection attacks. Based on these limitations, seven future research directions are proposed.

1. Domain formalization and context models. A mathematically rigorous model of DSL knowledge representation inside SLM weights is still missing. Instead of informal descriptions in prompts, DSL knowledge should be formalized as explicit domain ontologies, such as CM = <META, GRAMMAR, API, PATTERNS>. Such a model would allow algorithmic narrowing of context search and improve generation accuracy for complex C3 languages.

2. Development of an open DSL-Bench benchmark. Existing datasets such as HumanEval and MBPP target general algorithmic tasks. A standardized DSL benchmark is needed to evaluate models at four levels: syntactic correctness, schema compliance (AST), semantic coherence, and configuration-parameter security.

3. Synthesis and filtering of verified data. Data synthesis from LLM teachers introduces hallucinations into the dataset. The next step is to embed formal DSL validators (Kubernetes / Terraform) directly into the self-training loop to reduce the share of examples with syntactic and schema errors.

4. Synergy between structural constraints and PEFT adaptation. The interaction between GCD and low-rank adaptation (QLoRA) with respect to semantic flexibility remains understudied. A balance must be found where strict GCD syntax constraints do not prevent generation of complex logical dependencies learned during adaptation.

5. Cross-language transfer of DSL weights. The universality of trained adapters remains an open question. Studying weight transfer between grammatically related languages, for example from Kubernetes manifests to Helm charts, could substantially reduce the computational cost of preparing specialized models for new domains.

6. Adaptive security perimeters. Given the local nature of SLM deployment, red-teaming tools specific to infrastructure code should be developed. A separate task is practical implementation of machine unlearning methods for removing confidential organizational data from model weights.

7. Comparative evaluation of adaptation methods across DSL classes. Empirically validating the proposed C1-C3 classification requires a large-scale study of how metrics such as pass@k and accuracy deltas depend on PEFT method and model size. Such a study would provide a basis for selecting an adaptation strategy for a new DSL.

The seven directions are summarized in Table 8.

8. Conclusion

This paper reviews SLMs (≤ 7B parameters) for code and DSL artifact generation. Three key results are identified.

First, domain-specific languages, unlike general-purpose programming languages, have the distinctive property of formal verifiability of syntax and runtime semantics. This changes the operating conditions for small language models: training datasets can be filtered automatically, objective quality metrics can be introduced without human participation, and reliable security perimeters with self-training loops can be formed.

Second, the proposed DSL classification by context-model complexity (regular C1, context-free C2, and context-sensitive C3) enables a more algorithmic approach to selecting SLM fine-tuning strategies. For basic artifacts, prompting is often sufficient; for complex access policies, full fine-tuning combined with a semantic verifier may be required.

Third, the analysis of adaptation methods shows that isolated use of any single method, including RAG or LoRA, is often insufficient for practical tasks. A more practical direction is hybrid architectures that combine lightweight parametric fine-tuning (PEFT/QLoRA) with grammar-masked decoding (GCD), which provides syntactic control.

Modern SLMs with 1.5-3B parameters can be used for DSL assistants in closed edge perimeters. Seven future research directions are identified.