Large Language Models for African Languages: Systematic Review of Adaptation Strategies, Evaluation Challenges, and Future Directions

1. Introduction

1.1. Background and Motivation

Natural Language Processing (NLP) research has witnessed a paradigm shift over the past few years, fueled by the rapid rise of large language models (LLMs). This has not been an isolated event; a confluence of transformer-based self-attention, massive web pretraining pipelines, and unprecedented compute clusters all contributed to the emergence of this field. Vaswani et al. [1] provided the foundation by eliminating recurrence and demonstrating that self-attention can capture contextual representations more effectively. Devlin et al. [2] then demonstrated that masked contextual representations could provide massive downstream gains. Brown et al. [3] showed that sheer scale can alter the game, as expanding models into the billions of parameters allows models to achieve surprising few-shot generalization over tasks that previously required specialized architectures. Finally, PaLM [4], BLOOM [5], and LLaMA [6] took this momentum further by increasing parameters, making models’ weights accessible, and exploring multilingual training limits. Now, foundation models form the backbone of modern NLP, where practitioners rarely build models from scratch for specific use cases. Instead, practitioners adapt a single generalized backbone to target tasks via continued pretraining, prompting, or parameter adaptation. Bommasani et al. [7] rightfully labeled this a paradigm shift, highlighting the massive upside of such a change but also pointing towards serious systemic blind spots. While multiple recent surveys showcase the sheer velocity of LLM development [8,9], the technology’s benefits are highly skewed. The challenge stems from the fact that high-resource languages benefit massively from their web presence, quality training corpora, and well-established evaluation ecosystems. Outside of this narrow set of languages, the situation is significantly worse. Joshi et al. [10] showed that the majority of the world’s languages are on the computational periphery, with limited digital text and no institutional support. In African linguistic contexts, this is an extreme challenge, with annotated datasets being scarce, domain-specific corpora minuscule, and formal evaluation tracks few. Nekoto et al. [11] showed that progress in these settings requires participatory and community-centered efforts, while Adelani et al. [12] emphasized the need for native benchmarks with MasakhaNER.

Multilingual foundation models offered an intuitive solution, as NLLB [13] demonstrated how it is possible to scale machine translation to hundreds of languages, and XLM-R [14] identified the value of joint representations for cross-lingual tasks. However, simply throwing dozens of languages into a pretraining run will rarely deliver strong performance on low-resource targets. The field has recognized this bottleneck, and the past three years have seen a move from all-in-one large models toward focused adaptation approaches, such as continued pretraining on localized text, instruction tuning, parameter-efficient fine-tuning (PEFT), and tokenizer updates. Techniques such as LoRA [18], QLoRA [19], Modular Adapters [20], MAD-X [21], AdapterFusion [22] have all emerged as solutions that keep adaptation costs low while retaining language-specific parameters.

African language NLP is seeing this operational shift as well; early efforts have focused on a general multilingual approach. NLLB [13] increased the breadth of languages for translation, while Glot500 [28] and MADLAD-400 [29] gathered and curated text for hundreds of languages. Yet despite these valuable corpora, sheer language count does not necessarily translate to practical utility. As a result, research that previously focused on multilingual masked encoders (e.g., AfroLM [25], AfroXLM-R [26], SERENGETI [27]) has shifted toward dedicated generative LLMs. Recent projects, such as EthioLLM [30], Walia-LLM [31], AfriInstruct [32], Lugha-Llama [33], AfriqueLLM [34], SabiYarn [35] reflect this change, with the focus now on tailoring the adaptation architecture according to the sociolinguistic reality of the targeted language community.

1.2. Limitations of Existing Reviews

The fast growth of LLM studies has generated several review articles and surveys that focus on different aspects of foundation models, including their architectures, training strategies, evaluation approaches, and emerging capabilities [8,9]. However, most of the reviews focus on general LLMs without emphasizing the specific adaptation challenges that arise when using such technologies in languages with limited resources. This is not the case for multilingual and African language NLP research where previous studies highlight the resource imbalance phenomenon and demonstrate specific adaptation approaches applied in the African language domain [10,11,12]. Other works have focused on building multilingual resources, benchmarks, and individual language models such as NLLB, AfroLM, SERENGETI, and related works [24,25,26,27,28,29]. Although such studies contribute significantly to the literature, they do not fully address the current understanding of adaptation approaches for the majority of languages. A major limitation in the literature is the compartmentalization of LLM-related works into distinct themes, such as model architecture, resources, adaptation, and evaluation strategies. For example, a review article could highlight continued pretraining while another focuses on instruction tuning. Practical under-resourced LLM adaptation requires an integrated view of all the components. The recent emergence of language-specific LLM projects has made this issue more important. Studies such as EthioLLM, Walia-LLM, AfriInstruct, Lugha-Llama, and AfriqueLLM demonstrate different adaptation approaches, but it has not been easy to compare their results since most works report varying datasets, initialization models, adaptation strategies, and evaluation settings. Therefore, a review study that summarizes individual model contributions is not enough to understand the overall adaptation landscape. A comprehensive analysis is essential to comprehend the relationships among adaptation approaches, resource availability, evaluation methods, and the practical limits affecting under-resourced (African) language LLM development.

1.3. Research Gap and Contributions of This Review

It can be observed from the literature review that the study of adaptation of LLMs had rapid development recently, and existing literatures are scattered from language-specific adaptation methods, resources and evaluation methods. Therefore, it’s hard to draw conclusions and find effective adaptation methods dealing with the lack-of-resource problems across languages settings. Evaluations presents a major challenge, as even established evaluation frameworks like XTREME [38], XTREME-R [39] do not fully account for the diversity of African languages and practical needs. Existing recent benchmark resources like AfroBench [36], IrokoBench [37] and LinCE [40] overcome some drawbacks, yet several limitations exist regarding linguistic variations, computational cost, model size etc. Though efficient adaptation methods like LoRA [18] and QLoRA [19] reduce the costs, more computationally efficient adaptation approaches for low-resource scenarios must be thoroughly investigated. In light of the gaps above, this study has performed a synthesis of the study in adapting LLMs for under-resourced languages concerning the models, resources, techniques and evaluation methods for low-resource African languages: The key contributions are as follows:

• A summary of recent studies on adaptation methods in under-resourced language, with the core focuses to the efforts toward specific African languages
• Comparing adaptation approaches, including continued pretraining, instruction tuning, parameter efficient fine-tuning, tokenizer adaptation, and cross-lingual transfer learning
• Reviewing evaluation approaches by analyzing multilingual and African language NLP benchmarks
• Identifying the research gaps and future directions of adaptation, including resources, evaluation, and sustainable language AI development.

1.4. Research Questions

This review investigates five main research questions related to adaptation of LLMs for under-resourced and African languages:

• RQ1: How has research on LLMs for under-resourced African languages evolved from broad multilingual representation learning toward targeted language-specific adaptation?
• RQ2: What methodologies including continued pretraining, instruction tuning, parameter-efficient fine-tuning, tokenizer adaptation, and cross-lingual transfer are employed to adapt foundation models to resource-scarce language settings?
• RQ3: What training corpora, tokenization strategies, evaluation benchmarks, and performance metrics characterize the current under-resourced LLM literature?
• RQ4: What empirical factors, linguistic characteristics, and computational constraints govern the success or failure of LLM adaptation strategies across under-represented languages?
• RQ5: What technical and governance frameworks must be in place to develop sustainable, community-anchored language AI ecosystems for under-resourced languages?

1.5. Organization of the Paper

The rest of the paper is organized as follows: Section 2 describes the methodology of this review study. In Section 3, we present the results and quantity profile. We evaluate adapter mechanisms including continued pretraining, instruction tuning, parameter-efficient methods, tokenizer adaptation and hybrid pipeline in Section 4. We compare models and discuss their trade-offs and relevant evaluation suite in Section 5. Section 6 discusses challenges, research gaps, and the sustainable ecosystem framework; and finally, we conclude the paper in Section 7 with future perspectives.

2. Methodology

2.1. Review Design

This review was designed and conducted according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) statement [44] where PRISMA-S (Systematic Reviews, with or without Meta-analysis) [45] was adapted to ensure high reproducibility throughout diverse LLM literature search workflows. Due to the large heterogeneity in model architectures, parameter scales, tokenization methods, training objectives, target languages, and benchmark metrics across the included literature, quantitative statistical meta-analysis was not feasible. Consequently, evidence synthesis was conducted following the Synthesis Without Meta-analysis (SWiM) guidelines [46], utilizing structured qualitative aggregation, comparative taxonomy matrices, and empirical distribution mapping. The protocol for this review defines five dimensions when we evaluate the adaptation work on under-resourced language LLMs.

• Paradigms of the LLMs: Tracking shift from encoder-only multilingual masked language models [14,25] to generative, autoregressive, foundation LLMs [5,6,30,31,32,33,34,35] and custom language-specific models.
• Training Corpora: Amount, mixture of various corpora used for tuning, selection standards for web crawl and use of community sourced datasets [11,28,29,34] and synthesized translations.
• Adaptation Paradigms: The types of adaptation including continued pretraining [30,33,34], LoRA [18], QLoRA [19], bottleneck architecture [20,21,22], instruction-tuning [32,42], and post-training parameter merging [60].
• Tokenization schemes: evaluation of tokenization transfer and adaptation of vocabulary size [15,16,17,50,64].
• Evaluation Frameworks: Diverse tasks, task-oriented cross-lingual benchmarks like XTREME [38,47], African languages benchmark like IrokoBench [36,37] and technical/social challenges [40,49].

2.2. Information Sources and Search Strategy

The search strategy covered six leading bibliographic databases (ACL Anthology, arXiv, Scopus, Web of Science) from January 2020 to March 2026 and the latter phase for references was carried out from 15 May 2026 for backward and forward search tracing on highly influential papers such as MasakhaNER [12], NLLB [13], AfroLM [25], EthioLLM [30], IrokoBench [37] etc. The keyword search was built using combinations of the three conceptual groups (LLM Paradigm), (Resource Level), (Adaptation Technique/Evaluation) as shown in Table 1. The exact, reproducible search syntax utilized for each database is provided in Appendix A.

2.3. Eligibility Criteria

To ensure methodological rigor, consistency, and alignment with the research questions, explicit inclusion and exclusion criteria were established prior to screening as presented in Table 2.

2.4. Study Selection and PRISMA Flow Pipeline

The selection pipeline was conducted across four standardized sequential phases as depicted in Figure 1.

2.5. Data Extraction and Outcome Measures

Information was extracted from each selected study using a structured framework. The extracted information included publication details, target languages, model architecture, training strategy, datasets used, adaptation methods, evaluation benchmarks, and reported limitations.

Table 3. Systematic Data Extraction Framework.

Extraction Category	Extracted Data Fields and Variables
Bibliographic Metadata	Author(s), publication year, venue/source (ACL, EMNLP, NAACL, NeurIPS, ICLR, arXiv), DOI.
Linguistic Profile	Target language(s), ISO 639-3 codes, language families (e.g., Niger-Congo, Afroasiatic, Indo-Aryan, Uralic), script types (Latin, Ge’ez, Arabic, Devanagari).
Model Characteristics	Base foundation architecture (e.g., LLaMA-2/3, BLOOM, XLM-R, Gemma, Qwen, mT5), parameter scale (e.g., 125M, 7B, 14B, 176B), architecture type (encoder-only, decoder-only, seq2seq).
Training Resources	Corpus volume (tokens, words, GB), data composition (monolingual, parallel, synthetic translation, web crawl, instruction pairs), source domains.
Adaptation Pipeline	Adaptation method (continued pretraining, LoRA, QLoRA, adapters, instruction tuning, delta merging, in-context prompting), compute hardware utilized.
Tokenization Mechanics	Subword tokenizer type (BPE, SentencePiece Unigram, WordPiece, Byte-level BPE), vocabulary size (V), vocabulary expansion or transfer methods.
Evaluation Framework	Benchmark suites utilized (e.g., MasakhaNER, AfroBench, IrokoBench, Belebele, LinCE), evaluation tasks, primary metrics (BLEU, ChrF, F1, exact match, accuracy, perplexity), baseline models.

Table 3. Systematic Data Extraction Framework.

Extraction Category	Extracted Data Fields and Variables
Bibliographic Metadata	Author(s), publication year, venue/source (ACL, EMNLP, NAACL, NeurIPS, ICLR, arXiv), DOI.
Linguistic Profile	Target language(s), ISO 639-3 codes, language families (e.g., Niger-Congo, Afroasiatic, Indo-Aryan, Uralic), script types (Latin, Ge’ez, Arabic, Devanagari).
Model Characteristics	Base foundation architecture (e.g., LLaMA-2/3, BLOOM, XLM-R, Gemma, Qwen, mT5), parameter scale (e.g., 125M, 7B, 14B, 176B), architecture type (encoder-only, decoder-only, seq2seq).
Training Resources	Corpus volume (tokens, words, GB), data composition (monolingual, parallel, synthetic translation, web crawl, instruction pairs), source domains.
Adaptation Pipeline	Adaptation method (continued pretraining, LoRA, QLoRA, adapters, instruction tuning, delta merging, in-context prompting), compute hardware utilized.
Tokenization Mechanics	Subword tokenizer type (BPE, SentencePiece Unigram, WordPiece, Byte-level BPE), vocabulary size (V), vocabulary expansion or transfer methods.
Evaluation Framework	Benchmark suites utilized (e.g., MasakhaNER, AfroBench, IrokoBench, Belebele, LinCE), evaluation tasks, primary metrics (BLEU, ChrF, F1, exact match, accuracy, perplexity), baseline models.

Primary and Secondary Outcome Measures

To assess the effectiveness of LLM adaptation strategies, the extracted evidence was evaluated against defined outcome indicators:

• Task-specific empirical metrics, including BLEU and ChrF for machine translation, Macro/Micro F1-scores for sequence labeling (NER, POS) and text classification, Accuracy and Exact Match (EM) for question answering and reasoning benchmarks (MMLU, GSM8K variants), and perplexity/cross-entropy loss for language modeling capability.
• Computational parameter efficiency (percentage of updated parameters relative to full fine-tuning), GPU memory footprint and training throughput, subword fertility ratios across complex scripts, out-of-distribution cross-lingual transfer stability, and representational safety/bias scores.

2.6. Methodological Quality Assessment

The quality and reproducibility of all 42 included studies were evaluated using a standardized 7-point quality assessment scoring rubric. Each study was appraised independently by two reviewers across seven criteria, with items scored as 1.0 (Fully Met), 0.5 (Partially Met), or 0.0 (Not Met / Unclear):

Table 4. Methodological Quality Assessment Criteria.

Assessment Criterion	Description and Verification Requirement	Max Score
C1: Training Data Transparency	Precise description of training data sources, corpus sizes, cleaning/filtering steps, and token counts.	1.0
C2: Model Configuration Detail	Explicit reporting of base model parameters, layer dimensions, context lengths, and hyperparameter settings.	1.0
C3: Adaptation Pipeline Clarity	Replicable documentation of the adaptation workflow (e.g., LoRA rank r, alpha α, target modules, learning rate schedules).	1.0
C4: Benchmark & Task Appropriateness	Utilization of standardized, culturally grounded, or verified evaluation benchmarks with clear task definitions.	1.0
C5: Baseline Comparison Rigor	Inclusion of meaningful baselines (e.g., zero-shot base models, full fine-tuning, or standard multilingual baselines like XLM-R/NLLB).	1.0
C6: Reproducibility & Openness	Public availability of source code, model weights, training prompts, or curated evaluation datasets.	1.0
C7: Limitations & Failure Modes	Explicit and critical discussion of study constraints, data biases, negative results, or compute boundaries.	1.0

Table 4. Methodological Quality Assessment Criteria.

Assessment Criterion	Description and Verification Requirement	Max Score
C1: Training Data Transparency	Precise description of training data sources, corpus sizes, cleaning/filtering steps, and token counts.	1.0
C2: Model Configuration Detail	Explicit reporting of base model parameters, layer dimensions, context lengths, and hyperparameter settings.	1.0
C3: Adaptation Pipeline Clarity	Replicable documentation of the adaptation workflow (e.g., LoRA rank r, alpha α, target modules, learning rate schedules).	1.0
C4: Benchmark & Task Appropriateness	Utilization of standardized, culturally grounded, or verified evaluation benchmarks with clear task definitions.	1.0
C5: Baseline Comparison Rigor	Inclusion of meaningful baselines (e.g., zero-shot base models, full fine-tuning, or standard multilingual baselines like XLM-R/NLLB).	1.0
C6: Reproducibility & Openness	Public availability of source code, model weights, training prompts, or curated evaluation datasets.	1.0
C7: Limitations & Failure Modes	Explicit and critical discussion of study constraints, data biases, negative results, or compute boundaries.	1.0

Aggregate quality scores ($Q={\displaystyle {\sum }_{i=1}^7}{C}_i$) were classified into three quality tiers: High Quality (6.0 ≤ Q ≤ 7.0), Moderate Quality (4.0 ≤ Q ≤ 5.5), and Low Quality (Q < 4.0). Studies in the moderate and high tiers were synthesized in the findings, while methodological shortcomings identified in lower-scoring studies informed our analysis of reproducibility and evaluation gaps.

3. Findings of the Systematic Review

3.1. Overview and Quantitative Profile of Included Studies

The systematic screening process yielded 42 core empirical studies published between 2020 and 2026 addressing the adaptation, evaluation, and deployment of language models for under-resourced and African languages. An analysis of publication timelines reveals an accelerating research trajectory (Figure 2):

• 2019–2021 (Foundational Multilingual Baselines): Focused predominantly on encoder-based masked language modeling, initial cross-lingual transfer benchmarks, and subword tokenization dynamics ($n=7$, 16.7%).
• 2022–2023 (Massive Multilingual Scaling): Marked by the release of massively multilingual generative foundation models (BLOOM [5]), audited translation corpora (NLLB [13]), and broad coverage suites ($n=10$, 23.8%).
• 2024–2026 (Language-Specific Generative Adaptation): Characterized by instruction-tuning suites, post-training merging, and targeted language-specific LLMs ($n=25$, 59.5%).

Figure 2. Distribution of Included Studies by Publication Year.

Figure 2. Distribution of Included Studies by Publication Year.

3.2. Evolutionary Stages of Under-Resourced Language LLMs

As presented in Table 5, the evolution of under-resourced language LLM development can be broadly divided into four stages.

3.3. Distribution of Adaptation Approaches

The papers use various adaptation techniques either to deal with the cost of updating parameters or the linguistic mismatch between source and target. Figure 3 presents the analysis of adaptation techniques applied in actual adaptation works ($n=28$); the distribution follows as below:

• Language-Adaptive Continued Pretraining (LAPT / CPT): used in 16 studies (57.1%), as a foundational stage to adapt decoders such as LLaMA, Qwen and Gemma to target language vocabularies and domains.
• Supervised Fine-tuning / Instruction tuning: used in 13 studies (46.4%), for prompt template modeling, multi-task training, and conversation-aligned training.
• PEFT: LoRA, QLoRA and other module-based adapters were used in 10 studies (35.7%) to adapt LLMs using single GPU and efficient computation.
• Modular Language Adapter (MAD-X / bottleneck layers): used in 5 studies (17.9%) to achieve adaptive modular tasks and avoid catastrophic forgetting during further fine-tuning.
• Tokenizer Transfer and Expansion: used in 4 studies (14.3%), which solves sequence fragmentation in models based on LLMs.
• Post-Training Parameter Delta Merging: Used in 3 studies (10.7%), a way to fine-tune based on parameter-delta without further backpropagation which has shown good outcome in aligning instruction and language vectors when merged.
• In-context prompting and RAG: used in 3 studies (10.7%).
• Hybrid Multi-Stage Pipelines: Applied in 15 studies (53.6%), confirming that multi-stage workflows (CPT $\to$PEFT $\to$SFT) yield the highest empirical gains.

Figure 3. Adoption frequency of technical adaptation mechanisms across empirical LLM studies.

Figure 3. Adoption frequency of technical adaptation mechanisms across empirical LLM studies.

3.4. Key Empirical Findings Across Analytical Dimensions

3.4.1. Data Mixture and Pretraining Strategies

The literature survey highlights the fact that the success of continued pretraining depends more on corpus composition than on the number of tokens. For instance, in AfriqueLLM [34], it was found that the continuation of pretraining with 26B tokens on LLaMA-3.1 and Gemma-3 helped mitigate catastrophic forgetting of base English reasoning ability by mixing synthetic translated reasoning data with native African text. Lugha-Llama [33] found out that, in continuation pretraining with 10B tokens, including good quality educational data in English (FineWeb-Edu) along with African language text helped improve the downstream performance in reasoning benchmarks like AfriMMLU and AfriMGSM.

3.4.2. Parameter-Efficient Fine-Tuning vs. Full Fine-Tuning Trade-Offs

While LoRA [18] and QLoRA [19] help in reducing GPU memory consumption by 65–80% during fine-tuning, the synthesized literature indicates a specific trade-off between the two methods, as PEFT achieves 92–98% of full fine-tuning performance on classification and extractive question-answering tasks when there are abundant training examples. However, in target languages that are morphologically complex or are heavily inflected or agglutinative, low-rank updates may fail to capture fine-grained inflectional variations without scaling the rank hyperparameters or being coupled with target vocabulary expansion.

3.4.3. Subword Tokenization Constraints

Tokenization is another important factor that acts as a bottleneck in under-resourced non-Latin script languages. BPE and SentencePiece unigram tokenizers trained mostly on Latin-script corpora have subword fertility ratios between 3.5–6.0 tokens per word for Ge’ez, Arabic, and Indic scripts, whereas for English it ranges between 1.2–1.5. Studies on Trans-tokenization [50] and Tokenizer Transfer [64] show that replacing or expanding tokenizer vocabulary helps reduce the average sequence length by 35–60%, which lowers inference latency and improves context window utilization.

3.4.4. Benchmark Adoption and Evaluation Disparities

Evaluation practices across the 42 studies reveal a structural reliance on synthetic and machine-translated evaluation sets. While translation suites (BLEU, ChrF on Flores/NLLB) remain the most widely used metrics ($n=21$), the adoption of culturally grounded, native-speaker curated evaluation suites has risen sharply. AfroBench [36] and IrokoBench [37] provided multi-task evaluation suites across 16+ African languages covering reasoning, translation, and sentiment analysis [68]. Belebele [47] and BELA [54] established parallel multilingual reading comprehension and entity-linking baselines across 122 language variants. Linguistic Code-Switching (LinCE [40], GLUECoS [49]) quantified severe performance drops (15–30% relative F1 degradation) when LLMs process natural code-switched African and South Asian dialects.

3.5. Synthesis of Representative Under-Resourced Language LLMs

Table 6 summarizes the architectural configurations, training token budgets, adaptation strategies, and benchmark performances of landmark systems synthesized in this review.

4. Adaptation Strategies and Technical Mechanics

4.1. Language-Adaptive Continued Pretraining

Continued pretraining (or domain/language-adaptive pretraining) exposes a fully converged, pretrained foundation model to additional auto-regressive or masked language modeling objectives on target-language corpora. Rather than re-initializing weights from scratch, LAPT updates existing self-attention and feed-forward parameters using smaller learning rates (typically 1 × 10⁻⁵ to 5 × 10⁻⁵) to align internal representations with the target distribution.

Foundation models trained on internet-scale high-resource corpora contain minimal target-language tokens in their original training mixtures. LAPT directly addresses this initial under-exposure by updating token transition probabilities, allowing the model to acquire language-specific grammar, syntactic structures, and cultural idioms. In AfriqueLLM [34], continuing pretraining across 26 billion tokens on LLaMA-3.1-8B, Gemma-3, and Qwen-3 substantially improved downstream translation and reading comprehension across 20 African languages. Similarly, Lugha-Llama [33] executed continued pretraining over 10 billion tokens from the WURA corpus, demonstrating that injecting 4B tokens of high-quality English educational data (FineWeb-Edu) alongside African language data mitigated catastrophic forgetting and retained reasoning performance on AfriMMLU and AfriMGSM. In the Horn of Africa context, EthioLLM [30] adaptation on Amharic, Ge’ez, Tigrinya, Oromo in Horn of Africa demonstrated improvement in the extractive QA and NER from the source multilingual model, aligning with broader continual pretraining strategies [41,57,59]. The LAPT approach can demand tremendous computational resources, for example, 8 to 64 A100/H100 GPUs to tune the model across multiple nodes (e.g., from a local research institute with limited hardware accessibility). With small and noisy target corpora (containing OCR noise, HTML tags or automatically translated data) the quality will be significantly degraded in terms of meaning, overfitting on the target language as well as reducing cross-lingual performance. As the result, we conclude that LAPT is a successful first-stage of adaptation only when massive clean target corpora (at least over 1 billion words) are available. For extremely low-resource languages (under 10 million words), the most influential factors are likely to be compute constraints as well as data quality, and LAPT is unlikely to be the right path.

4.2. Supervised Instruction Tuning and Alignment

Instruction tuning is a form of fine-tuning where models are trained to optimize sequence prediction and in doing so align with the users’ explicit instructions and multi-turn conversations by being prompted with question, instruction and expected answer pairs. It helps a foundation model to unlock and apply latent cross-lingual understanding into particular non-English contexts when instruction data is unavailable or not sufficient, following foundational alignment paradigms such as FLAN [42] and InstructGPT [43]. For example, by training on a subset of English instruction data followed by instructions relevant to low-resource context. For instance, the study AfriInstruct [32] tuned the LLaMA-2-7B model on instruction prompts written in 16 African languages spanning over 17 million pairs and 870 million tokens and showed a performance gain in NER, question answering and translation on the respective African languages. In similar work, Walia-LLM [31] demonstrated efficient instruction-following transfer into a non-Latin script on Amharic with instruction pairs being 122 thousand while Basque Instruct [56] employed template-based dialogues which perform at scale compared to current task-specific solutions.

Trade-offs and Limits: Manually curating high-quality instruction datasets for under-resourced languages is time and compute intensive, with cost prohibitive at scale. Machine translated instruction datasets from high-resource languages such as Alpaca or FLAN commonly suffer from slight translation errors, unnatural syntax, and cultural biases. As observed in Walia-LLM [31], instruction-following in MP translation degrades downstream task performance (specifically with high precision tasks like abstractive summarization), while improving task fluency. Instruction tuning is crucial for developing user-facing utility, but for many tasks it only functions to format existing linguistic abilities in response rather than imbue the model with actual semantic understanding, especially if lacking pre-trained linguistic foundation.

4.3. Parameter-Efficient Fine-Tuning

Parameter-Efficient Fine-Tuning adapts foundation models to target languages or downstream tasks by freezing the vast majority of pretrained parameters ${\mathsf{\Theta }}_{\mathrm{base}}$ and updating only a small set of injected parameters $\mathsf{\Delta }\mathsf{\Theta }$ (< 2% of total weights):

• LoRA [18]: Decomposes weight updates into low-rank matrices, $W=W_0+B\cdot A$, where $B\in R^{d\times r}$, $A\in {\mathrm{R}}^{r\times k}$, and rank $r<\min \left(d,k\right)$.
• QLoRA [19]: Enhances LoRA by quantizing the frozen base model to 4-bit NormalFloat (NF4) and utilizing double quantization and paged optimizers to dramatically cut GPU memory.
• Modular Adapters (MAD-X [21] / AdapterFusion [22]): Injects lightweight bottleneck feed-forward layers after transformer attention blocks, training separate language adapters ($LA$) and task adapters ($TA$).

High-performance GPUs represent a critical barrier in the Global South. PEFT enables academic institutions, local communities, and researchers with consumer-grade hardware (e.g., single 24GB RTX 3090/4090 or 40GB A100 GPUs) to adapt 7B to 14B parameter models effectively. In Walia-LLM [31], applying LoRA on LLaMA-2-7B with rank r = 16 allowed fine-tuning within standard memory envelopes while achieving viable generation in Amharic. MAD-X [21] demonstrated that decoupling language-specific adapters from task-specific adapters enables zero-shot cross-lingual transfer across diverse languages without catastrophic forgetting of previous knowledge. DIPLomA [60] and related model composition works [53] demonstrated that parameter delta merging (combining task vectors and language vectors post-hoc) enabled efficient instruct adaptation for Basque without full-model gradient updates.

The efficiency of PEFT comes with a clear structural cost (Figure 4); in classification and extraction tasks, PEFT achieves 92-98% of full fine-tuning performance. In morphologically complex, highly inflected, or typologically distant languages, low-rank updates ($r\le 16$) often struggle to adapt the model’s underlying representations, requiring higher ranks ($r\ge 64$), full-parameter unfreezing, or adapter stacking to match full fine-tuning [58]. PEFT is the primary technological enabler for democratizing LLM adaptation in compute-scarce environments, but its parameter constraints make it less robust for typologically distant and highly agglutinative target languages unless paired with vocabulary adaptation.

4.4. Tokenizer Adaptation and Vocabulary Expansion

Tokenizer adaptation modifies or expands the subword tokenizer vocabulary V (via SentencePiece Unigram [15], Byte-Pair Encoding [17], or BPE regularization [16]) to represent target-language scripts and morphemes more compactly. Techniques such as Trans-tokenization [50] and Tokenizer Transfer [64] re-initialize or map token embedding matrices to support new vocabularies without training the entire network from scratch. Standard tokenizers in foundational models (e.g., LLaMA, Mistral) are heavily optimized on English and Latin-script corpora. When processing non-Latin scripts (e.g., Ge’ez, Arabic, Indic) or highly agglutinative languages (e.g., Bantu family), standard tokenizers exhibit extreme subword fertility-fragmenting single words into multiple arbitrary character tokens or byte sequences. This inflates inference sequence lengths, causes out-of-memory errors, and diminishes the effective context window [51].

Trans-tokenization [50] and Tokenizer Transfer [64] demonstrated that expanding the vocabulary with target-language subwords reduced sequence fertility by 35% to 60% across under-represented languages, directly accelerating training throughput and inference speeds. In Ethiopian NLP, EthioLLM [30] highlighted that adapting the subword vocabulary to properly segment Ge’ez script morphemes directly improved cross-lingual NLI and NER scores. Adding new token embeddings creates a parameter alignment problem: newly initialized embedding vectors lack contextualized alignment with the frozen upper layers of the transformer, requiring substantial warmup steps and additional text corpora to stabilize training. Furthermore, aggressive vocabulary expansion inflates the model’s memory footprint at the input/output projection layers ($\left|\mathit{V}\right|\times {\mathit{d}}_{\mathrm{model}}$).

Tokenizer adaptation is mandatory for non-Latin and morphologically rich under-resourced languages to overcome subword fertility bottlenecks, provided sufficient target-language text is available to tune the newly added embedding parameters.

4.5. Cross-Lingual Knowledge Transfer

Cross-lingual transfer leverages shared representation spaces learned during multilingual pretraining (or intermediate fine-tuning) to project capabilities from a high-resource source language (${\mathit{L}}_{\mathit{s}}$) to an under-resourced target language (${\mathit{L}}_{\mathit{t}}$). Because under-resourced languages lack extensive annotated datasets for downstream tasks (e.g., NER, reading comprehension, sentiment analysis), cross-lingual transfer allows models to be fine-tuned on abundant English or French task data and evaluated zero-shot on the target language. XLM-R [14] and NLLB-200 [13] established the viability of cross-lingual parameter sharing across 100 to 200+ languages. AfroXLM-R [26] and SERENGETI [27] proved that fine-tuning multilingual models on African text enabled effective zero-shot transfer on benchmarks such as MasakhaNER [12], AfriSenti [48], and MasakhaPOS [66]. In extreme low-resource contexts, in-context cross-lingual learning [55] and graph-based small model transfers [62] confirmed that shared contextual representations support rapid few-shot task adaptation.

Cross-lingual transfer effectiveness is highly asymmetric: transfer is strongest when source and target languages share typological traits, syntactic structures, or vocabularies. When transferring from English to structurally distant languages (e.g., tonal Niger-Congo, agglutinative Bantu, or Semitic root-and-pattern languages), zero-shot performance drops significantly. Additionally, cross-lingual transfer frequently propagates cultural assumptions and social biases from dominant source languages into low-resource target outputs [52]. Cross-lingual transfer provides a crucial cold-start mechanism for zero-shot downstream capability, but it cannot replace target-language contextual grounding and native task evaluation.

4.6. Hybrid Adaptation Pipelines

A hybrid adaptation pipeline combines different techniques into a seamless multi-step process: Base Foundation Model ⟶ Tokenizer Adaptation ⟶ Language-Adaptive CPT ⟶ PEFT Instruction Tuning ⟶ Delta Merging / Alignment. Adaptation of under-resourced languages faces a number of bottlenecks together, namely inefficient tokenization, lack of linguistic expertise, limited computational resources, and the lack of task data. None of the techniques solves all the aspects separately. EthioLLM [30] has combined vocabulary adaptation and multilingual continued pretraining with multi-task tuning for Ethiopian languages. AfriInstruct [32] has combined continued pretraining with extensive instruction tuning in 16 African languages. Lugha-Llama [33] has used co-mixing of English educational data, continued pretraining on the WURA dataset, and instruction alignment. Basque Instruct [56] and DIPLomA [60] systematically combine continued pretraining with post-training parameter delta merging based on LoRA [61].

Hybrid pipelines increase architectural complexity, requiring fine-tuned hyperparameters in every stage transition. Faults that happen in earlier stages (e.g., inappropriate token initialization or noisiness of pretraining mixtures) propagate downstream and can trigger instability during instruction tuning. The literature confirms that successful adaptation of under-resourced LLMs is a hybrid engineering process by nature. The question for practitioners is no longer about finding a single optimal technique; it is about building multi-stage pipelines matching specific data quantity, script types, and computational capacity of the target language.

5. Comparative Analysis and Discussion

5.1. Cross-Model Comparative Synthesis

Three architectural paradigms were identified as core to the landscape of under-resourced NLP adaptation. These models are characterized as either small models trained from scratch for a specific task [25,35], massively multilingual models [5,13,27], or adapted open Foundation models ([30,31,32,33,34]). These smaller from-scratch models trained to complete a specific task, like AfroLM [25], demonstrated impressive potential despite very limited compute usage and language-specific corpora when tasked with classification, NER and sentiment tasks [25,35]. However, these models are not capable of zero-shot generalization or abstract reasoning, due to low model size and specific task formulation. Multilingual foundation models like BLOOM [5], NLLB-200 [13], and SERENGETI [27] maximize multilingualism, offering strong multilingual representation benchmarks but varying widely in capability for under-resourced languages and consistently suffering from high hallucination rates and significant language degrading on non-Latin scripts. Adapted Foundation models like those created and explored in EthioLLM [30], Lugha-Llama [33], Walia-LLM [31] presented the most promising medium between the two. Adaptation using strong open weight foundation LLM backbones such as LLaMA-3, Gemma, and Qwen combined with task specific or multilingual LAPT and instruction tuning consistently achieved state-of-the-art on under-resourced NLP benchmarks. However, their downstream success is strongly dictated by data composition: co-mixing high-quality English educational text during adaptation is essential to prevent the catastrophic forgetting of complex reasoning abilities [63].

5.2. Comparative Trade-Offs Across Adaptation Frameworks

As can be seen in Table 7, the synthesized empirical evidence reveals fundamental trade-offs between parameter efficiency, computational overhead, and linguistic expressiveness across adaptation techniques.

5.3. Comparative Analysis of Tokenization and Subword Dynamics

Subword segmentation directly governs how effectively LLM process non-Latin scripts and morphologically rich languages (Table 8).

5.4. Audit of Multilingual and African Evaluation Suites

A systematic audit of evaluation suites highlights an ongoing transition from generic cross-lingual transfer benchmarks to culturally and linguistically grounded evaluation suites.

Table 9. Systematic Audit of Benchmarks Utilized in Under-Resourced LLM Studies.

Benchmark Suite	Key Sources	Evaluated Tasks	Linguistic Coverage	Strengths & Innovations	Gaps & Evaluation Bottlenecks
XTREME / XTREME-R	[38,39]	NLI, QA, POS, NER, Retrieval	40–50 languages (global)	Standardized, cross-lingual multi-task baseline.	Minimal representation of African and indigenous languages; relies heavily on translated test sets.
MasakhaNER 2.0	[12,65]	Named Entity Recognition (NER)	20+ African languages	Gold-standard, community-curated human annotations.	Task-specific; does not evaluate generative reasoning or instruction- following.
IrokoBench	[9,37]	Multi-task reasoning (AfriMMLU, AfriMGSM, AfriQA, Translation)	16+ African languages	Culturally grounded evaluation suite designed explicitly for generative LLMs.	Still emerging; requires broader expansion into central and southern African language varieties.
AfroBench	[36]	Generative QA, Sentiment, Translation, Summarization	Large African language portfolio	Comprehensive evaluation of open and closed foundation models.	Uneven test set granularity across extreme low-resource dialects.
Belebele / BELA	[47,54]	Reading Comprehension, Entity Linking	122 language variants	Parallel, multi-choice reading comprehension across diverse typologies.	Multiple-choice format may not reflect open-ended generation quality or safety alignment.
LinCE / GLUECoS	[40,49]	Code-switched Sentiment, NER, POS, NLI	Multilingual & mixed dialects	Evaluates realistic, dialectal, and mixed- language NLP settings.	Fragmented across specific language pairs; difficult to integrate into standard LLM leaderboards.

Table 9. Systematic Audit of Benchmarks Utilized in Under-Resourced LLM Studies.

Benchmark Suite	Key Sources	Evaluated Tasks	Linguistic Coverage	Strengths & Innovations	Gaps & Evaluation Bottlenecks
XTREME / XTREME-R	[38,39]	NLI, QA, POS, NER, Retrieval	40–50 languages (global)	Standardized, cross-lingual multi-task baseline.	Minimal representation of African and indigenous languages; relies heavily on translated test sets.
MasakhaNER 2.0	[12,65]	Named Entity Recognition (NER)	20+ African languages	Gold-standard, community-curated human annotations.	Task-specific; does not evaluate generative reasoning or instruction- following.
IrokoBench	[9,37]	Multi-task reasoning (AfriMMLU, AfriMGSM, AfriQA, Translation)	16+ African languages	Culturally grounded evaluation suite designed explicitly for generative LLMs.	Still emerging; requires broader expansion into central and southern African language varieties.
AfroBench	[36]	Generative QA, Sentiment, Translation, Summarization	Large African language portfolio	Comprehensive evaluation of open and closed foundation models.	Uneven test set granularity across extreme low-resource dialects.
Belebele / BELA	[47,54]	Reading Comprehension, Entity Linking	122 language variants	Parallel, multi-choice reading comprehension across diverse typologies.	Multiple-choice format may not reflect open-ended generation quality or safety alignment.
LinCE / GLUECoS	[40,49]	Code-switched Sentiment, NER, POS, NLI	Multilingual & mixed dialects	Evaluates realistic, dialectal, and mixed- language NLP settings.	Fragmented across specific language pairs; difficult to integrate into standard LLM leaderboards.

5.5. Critical Synthesis

The evidence from the 42 core studies overwhelmingly demonstrates definitive answers to the primary research questions:

Answer to RQ1: The development of under-resourced NLP technologies has transitioned from the purely manual processes of collecting and utilizing human-written data [11,12], to incorporating pretraining multilingual encoders [14,25,26], then scaling in a multilingual fashion [5,13,27], and the eventual, optimal goal, discrete language adaptation through targeted generation tuning ([30,31,32,33,34,35]). The primary goal now is to understand if and how a language can be represented linguistically and functionally as opposed to simply recognized.

Answer to RQ2: Single technique adaptation processes alone were not enough to address under-resourced NLP models effectively. Initial LAPT pretraining provided important lexical and structural representation building blocks [30,34], and PEFT methods lowered computational expense during adapter application and training [18,19,31], but empirical evidence concluded that most effective adaptation pipelines utilize a combination of multiple individual techniques. Hybrid processes incorporate tokenizer adaptation, multilingual LM adaptation with instruction co-mingling, parameter efficient instruction tuning, and ultimately post-training parameter deletion and/or alignment [32,33,60].

Answer to RQ3: Use of traditional sub-word tokenizers leads to significant sequencing inflation due to the vast number of linguistic features required to express under-resourced language nuances. This necessitated the integration of tokenizer adaptation strategies [15,50,64], similarly to the shift from synthetic translated instruction benchmarks like XTREME ([38]) to locally and culturally pertinent evaluation suites like MasakhaNER [12], AfroBench [36], and IrokoBench [37].

Answer to RQ4: Model parameter count proved insignificant compared to data composition, local-topic accuracy and computational resources when analyzing model performance. Co-mingling reasoning and descriptive corpora is crucial to avoid catastrophic forgetting of complex tasks in the downstream direction of adaptation [33,34] and reliance purely on synthetic translation led consistently to language degradation and the inclusion of translation errors in task-oriented summaries, among others, which highlights the need for culturally relevant instruction sets.

6. Research Gaps, Sustainable Ecosystem Framework, and Future Directions

6.1. Move Beyond Models to an Ecosystem Paradigm

Recent developments in LLMs have designed new prospects for languages that have received limited attention in NLP research. Multilingual models, language-specific adaptation approaches, and efficient fine-tuning methods have extended the technical promises for under-resourced languages [13,25,26,27,28,29]. However, research must continue to account for issues outside model architecture itself, such as limited linguistic resources and insufficient benchmark comparisons. The issues identified consistently affect and hinder the success rate and accuracy of adapted models; limited corpora and poor evaluation practices in particular continue to restrict how far LLM adaptation can advance for low-resourced languages. Therefore, future development should be approached as an ecosystem-building challenge, as presented by Figure 5, rather than only a model development problem.

6.2. Language Resource Constraints

Language resource availability remains one of the key bottlenecks in the development of LLMs for under-resourced languages. While multilingual initiatives such as NLLB, SERENGETI, Glot500, and MADLAD-400 have broadened language coverage [13,27,28,29], many languages are still understudied. Previous studies highlighted the uneven nature of the distribution of linguistic resources in NLP, revealing that many languages are disadvantaged due to the lack of digital data and technological support [10,11,12]. However, recent advances in LLMs have not resolved this issue; instead, the necessity of high-quality and representative language resources is emphasized. Future research should therefore focus on a data-centered approach, including open corpus development, linguistic documentation, improved annotation strategies, and enhanced community participation in resource creation.

6.3. Adaptation Strategy Selection and Evaluation Challenges

Recent studies have explored various strategies for developing under-resourced language models, including continued pretraining, instruction tuning, parameter-efficient fine-tuning, tokenizer adaptation, and cross-lingual transfer [13,14,15,16,17,18,19,20,21,22,25,26,27,28,29,30,31,32,33,34,35,41,42,43]. Current studies are limited in their understanding of how adaptation strategies should be chosen for different language and resource contexts. The relevance of each approach depends on factors such as the amount of data available, computational capacity, and the characteristics of the target language. For languages with adequate text resources, continued pretraining can improve language representation, while parameter-efficient methods may be more practical if computational resources are restricted. Further studies should seek to identify effective combinations of adaptation strategies for specific language contexts rather than viewing each method as an isolated solution.

Evaluation remains a significant limitation in the field of under-resourced language LLMs. While existing benchmarks, including XTREME, XTREME-R, AfroBench, IrokoBench, BELA, and LinCE, provide valuable evaluation resources, many languages and practical tasks are still underrepresented [36,37,38,39,40,47,54]. Moreover, long-term sustainability requires enhanced local capacity. Sustainable language AI development requires accessible infrastructure, open datasets, trained researchers, and enduring language community participation [11,12].

6.4. Multi-Level Future Research Roadmap

In order to address these challenges, Table 10 maps the identified research bottlenecks to actionable future technical and governance directions, directly addressing the core research questions.

6.5. Sustainability and Local Capacity

Sustainability of language AI development cannot be achieved through technical advancements alone. Numerous existing projects make use of open foundation models, computing power from the outside, and international cooperation. While these are contributing to the success, sustainability will require increased local capacity. The sustainable environment should provide available computing power, local researchers, open datasets, organizations to conserve the language, and active participation of the language community [11,12]. Language communities should not only be seen as providers of the data but also active participants of research agendas and evaluation of model results.

7. Conclusion and Future Perspectives

LLMs have broadened the scope of language technology development for communities and languages that have previously been underrepresented in NLP research. However, the benefits of these advances are not distributed evenly, and many under-resourced languages face significant hurdles in terms of limited training data, inadequate evaluation resources, and limited computational capacity. This review shows that African language LLM research has gradually shifted from a focus on wide multilingual representation to a focus on language adaptation. Recent contributions are no longer limited to the expansion of language coverage, but now consider language-specific resources, adaptation strategies, and evaluation requirements. The reviewed studies show that effective LLM development for African languages requires the interplay of trustworthy language resources, appropriate adaptation strategies, and evaluation methods that reflect actual language characteristics.

The findings also highlight that improving LLM capabilities for under-resourced languages is not necessarily a matter of increasing model size. The availability and quality of training data, adaptation strategy selection, and consideration of linguistic characteristics greatly contribute to model capability. For many African languages, improving these foundations is equally important to model architectural enhancements. Overall, the development of language AI for under-resourced languages is reliant on various aspects beyond technical advancements. The availability of language resources, effective adaptation strategies, reliable evaluation methods, research capacity, and access to compute resources are all critical factors in determining whether LLMs can effectively support the needs of diverse linguistic communities.