Expressive Dizi Synthesis: Unlimited Synthetic Datasets, Real–Synthetic Integration and Technique Labeling

1. Introduction

The digital modeling of musical instruments has become a core focus within music information retrieval, computer-based music, and artificial intelligence-led creative systems. The musical instruments that come from Western countries, which include piano, violin and concert flute, possess quite mature technologies of synthesis and sampling that can support their use. The Chinese traditional musical instruments have not yet received sufficient research exploration.

Current production workflows typically rely on commercial sample libraries that require manual DAW editing, where key switches and continuous controllers approximate expressive effects. This method gets believable outcomes but needs large effort and special knowledge, therefore it limits traditional tone use in current media.

Musical sound synthesis remains a core concern in sound and vibration research, where modeling excitation mechanisms, resonant structures, and radiated acoustic responses continues to pose significant challenges. For wind instruments such as the dizi, sound production emerges from complex aeroacoustic interactions among airflow excitation, embouchure control, and the resonant air column. These interactions produce strongly nonlinear and time-varying vibration patterns, rendering generated sound highly sensitive to subtle performance variations. Both physically informed models and purely data-driven approaches face substantial difficulties in capturing expressive performance behaviors.

Recent advances in signal processing and machine learning have reshaped musical sound synthesis. High-capacity neural networks and modern neural audio-generation techniques now enable increasingly realistic waveform synthesis and timbral reconstruction. However, despite notable progress for Western orchestral and keyboard instruments, data-driven modeling of traditional wind instruments such as the dizi remains limited. This shortfall is closely linked to the intrinsic complexity of aeroacoustic excitation, nonlinear resonant phenomena, and the scarcity of high-quality paired symbolic-acoustic datasets.

Beyond static timbral reproduction, realistic synthesis must render expressive performance attributes, including articulation, dynamic shaping, pitch inflection, and noise-related components such as breath and turbulence. Acoustically, these expressive aspects manifest as transient energy redistribution, time-varying spectral-envelope modulation, and dynamic changes in the harmonic-to-noise ratio. Expressive synthesis cannot be reduced to static waveform rendering; rather, it should be formulated as a structured control problem over time-varying acoustic parameters that reflect performance intent.

Score-to-audio synthesis provides a promising avenue for expressive sound generation by explicitly linking symbolic musical descriptions to acoustic signal formation. Compared with sample-based playback or unconstrained generative audio models, score-driven synthesis offers structured and interpretable control that aligns naturally with compositional intent and performance instructions. However, the efficacy of score-to-audio systems depends heavily on the availability of paired symbolic-acoustic data and on the expressive richness of the symbolic representations themselves.

These limitations are especially acute for traditional and non-Western wind instruments, where standardized datasets and performance annotations are sparse and where many expressive techniques are transmitted via oral tradition rather than formal notation. Synthesis frameworks developed primarily for Western instruments cannot be transferred without careful adaptation. From the standpoint of sound and vibration modeling, this gap underscores the need for synthesis strategies that balance data efficiency, acoustic interpretability, and expressive controllability while accommodating instrument-specific performance practices.

Expressive performance synthesis can be interpreted as a mapping from symbolic control parameters to time-varying acoustic responses. Prior research indicates that purely data-driven models often struggle when expressive dimensions are weakly constrained or underrepresented in the symbolic domain. This issue is particularly pronounced for wind instruments, where articulation, breath control, and noise components introduce additional degrees of freedom not captured by standard symbolic encodings such as MIDI.

This work implements the proposed framework on a MIDI-DDSP backbone and employs real recordings for selective fine-tuning to transfer authentic expressive dynamics. Ornamentations are encoded as out-of-range MIDI tokens to provide technique-aware conditioning for the synthesis model.

2. Related Work

Research on expressive music performance synthesis spans algorithmic models, neural approaches trained on large audio datasets, and hybrid frameworks. Deep learning has enabled direct mappings from symbolic representations to audio signals. Differentiable digital signal processing (DDSP) adds controllability and physical interpretability to these systems. This section reviews the generation of audio from scores, neural acoustic models, and modeling strategies tailored for specific instrument performances.

Early computational modeling of expressive performance focused on analyzing and parameterizing timing and dynamics rather than producing actual audio. Cancino Chacón et al. [1] reviewed statistical methods and machine learning approaches attempting to capture natural human expressivity. These early efforts built theoretical foundations but stayed strictly in the analysis domain. They did not achieve direct end-to-end synthesis.

Early neural synthesis systems brought expressive modeling into actual audio generation. PerformanceNet (Wang and Yang) [2] mapped MIDI sequences directly to waveforms. Related architectures like Melody RNN explored similar symbolic generation tasks. Limited sound quality and coarse control restricted these systems to research demonstrations. They were generally unsuitable for professional production use.

Neural vocoders and audio codecs such as WaveNet [3], HiFi GAN [4], and EnCodec [5] improved waveform fidelity and generation efficiency. These models advanced perceptual realism significantly. They primarily focused on general waveform modeling rather than instrument specific expression or explicit performance control.

The MIDI-DDSP framework (Engel et al.) [6] integrated MIDI control with differentiable physical modeling [7]. Trained on the URMP dataset [8], it captured expressive nuances for Western instruments including the flute, violin, and trumpet. Its dependence on carefully annotated paired data limits its applicability to traditional instruments with very few high quality recordings.

Recent architectures have grown more powerful in handling complex audio distributions. Deep Performer [10] utilized transformers for mapping symbols to audio. MIDI VALLE [11] applied neural codec language modeling to achieve better generalization across domains. Parallel to academic research, commercial AI generation platforms like Suno and ACE Studio have introduced new paradigms in music synthesis. Suno leverages massive scale autoregressive modeling to generate full musical textures from textual or basic structural prompts. This approach yields impressive overall acoustic realism but lacks the note level symbolic intervention required for precise instrumental arrangement. ACE Studio represents another emerging trend by offering highly expressive AI vocal synthesis that is gradually expanding into instrumental domains. These tools often rely on continuous parameter curves inferred from massive datasets to mimic human phrasing. They still struggle with the highly idiomatic ornamentations and specific physical nonlinearities of non-Western acoustic instruments.

Two primary problems persist in the field. There is a scarcity of paired training data and insufficient modeling of detailed performance techniques. Most existing systems target Western instruments [12,13]. They often neglect the unique ornamentations and acoustic characteristics of traditional Chinese instruments.

This work uses the Chinese bamboo flute as a case study. We propose a scalable pipeline producing effectively unlimited high fidelity paired data. A technique annotation scheme encodes dizi ornamentations using extended MIDI note numbers. This method enables synthesis conditioned on specific technical gestures. Integrating large scale synthetic data with limited real recordings preserves timbral consistency while transferring expressive richness.

3. Unlimited Synthetic Data Workflow

To overcome dataset scarcity for traditional Chinese instruments, we design a scalable workflow generating paired score–audio data. The pipeline consists of four sequential stages: (i) rule-based symbolic score generation, (ii) batch audio rendering, (iii) MIDI–audio feature extraction, and (iv) extended performance technique encoding. This pipeline combines reproducibility, extensibility, and expressive control for deep neural synthesis.

Figure 1. Synthetic Data Workflow.

Figure 1. Synthetic Data Workflow.

3.1. Rule-Based Score Generation

The first stage constructs symbolic musical sequences emulating idiomatic dizi patterns. We use a rule-based algorithm generating unlimited symbolic sequences with configurable parameters: key, mode, pitch range, tempo. This enables scalable generation while maintaining musically meaningful constraints.

Rhythmic structures follow a fixed metric grid for temporal regularity and rendering stability. Note durations sample from a weighted distribution of common rhythmic values, promoting stylistic plausibility without excessive complexity. Pitch transitions stay within one octave, reflecting typical dizi practice and maintaining physical playability. Velocity values sample from a Gaussian distribution, providing controlled loudness fluctuations.

Operating symbolically enables high reusability and flexibility. Generation rules adapt easily to alternative scales, rhythmic profiles, or tempo distributions, facilitating systematic experimentation. Symbolic generation decouples musical structure from acoustic realization, allowing large-scale controllable dataset construction.

3.2. Batch Rendering of Audio

Generated symbolic scores render into audio waveforms using Reaper DAW with custom ReaScript automation. This ensures consistent, reproducible MIDI-to-audio conversion. Flexible virtual instrument or sample library substitution enables systematic investigation of data quality and timbral variation effects on model performance.

Training corpus fidelity matters. High-quality rendered audio preserves timbral characteristics and expressive detail. Low-quality or inconsistent sources introduce artifacts that hinder learning. The pipeline enforces controlled rendering: 48 kHz sampling, 24-bit resolution, monophonic output.

Automated rendering imports MIDI files in batch, assigns them to a predefined instrument track, initializes the virtual instrument plugin, and exports audio under identical settings. This enables large-scale homogeneous dataset generation while minimizing variability. Rendering identical scores with different virtual instruments allows controlled analysis of timbral fidelity and dataset composition effects.

3.3. Feature Extraction from MIDI-Audio Pairs

Each MIDI–audio pair processes into frame-level features for training performance-aware synthesis models like MIDI-DDSP. Audio signals downsample to 16 kHz mono, reducing computational cost while preserving perceptually relevant information. Signals segment into fixed-length frames with anti-alias filtering for spectral stability.

CREPE estimates fundamental frequency (f₀) trajectories, capturing melodic and expressive pitch variations. Perceptual loudness and signal power (in decibels) represent dynamic characteristics. These features encode core acoustic dimensions: pitch, dynamics, temporal evolution.

MIDI events align to the same frame grid, yielding onset/offset indicators, active note indices, and time-varying velocity curves. Metadata (instrument identity, file identifiers) attaches to each instance. All features serialize into TFRecord format for compact storage and efficient training loading. This representation captures melodic contour, dynamic shaping, and structural timing while maintaining compatibility with scalable synthesis pipelines.

3.4. Encoding Extended Performance Techniques via Out-of-Range MIDI Mapping

Dizi expressivity relies on ornamentations (tonguing, vibrato) not captured by standard MIDI. We use out-of-range MIDI mapping: note numbers outside the natural range (G4–D7) represent extended techniques (Table 1).

This integrates with DAWs / plugins (remapping assigned notes to articulatory controls). It creates a digital notation paradigm encoding idiomatic techniques alongside pitch/rhythm—enabling automatic generation and human editing. It provides explicit supervision for training score-to-audio models, supporting fine-grained ornamentation control.

Extending symbolic representations to include performance-level control balances interpretability and modeling flexibility. Explicit technique encoding lets the synthesis model associate symbolic events with measurable acoustic phenomena: transient energy distribution, spectral noise components, pitch deviation patterns. This design fits hybrid neural-DSP architectures, where symbolic controls map to physically meaningful synthesis parameters.

Figure 2. Out-of-Range MIDI Mapping Techniques.

Figure 2. Out-of-Range MIDI Mapping Techniques.

3.5. Summary of Workflow

Four stages establish a pipeline from symbolic abstraction to expressive audio rendering. The rule-based generator supplies unlimited symbolic material. Batch rendering ensures timbral fidelity. Feature extraction aligns symbolic and acoustic representations. Extended MIDI mapping encodes ornamentation beyond standard notation. This workflow provides scalable training data and a framework for integrating traditional instrumental techniques into neural audio synthesis.

Large-scale synthetic data play a dual role: they provide controllable, noise-free supervision while enabling systematic coverage of pitch ranges, dynamic levels, and articulation patterns difficult to obtain from real performances. Synthetic audio may lack micro-level expressive irregularities, but its consistency and reproducibility suit learning stable timbral representations and fundamental acoustic mappings.

4. Transfer and Integration of Real and Synthetic Data

4.1. Complementary Roles

Synthetic data provides stable, controllable timbral priors. Real recordings recover subtle dynamics and natural phenomena (breathing noises). Our scalable score-to-audio workflow generates large-scale high-fidelity MIDI–WAV pairs, ensuring adequate coverage of pitch, rhythm, and scores. However, rendered sources lack breathing noise and performance details (phrasing, airflow artifacts), capping expressivity for models trained solely on them. Real recordings from URMP dataset [8], though limited in quantity, contain authentic breathing and dynamic patterns—providing critical expressive priors.

We design a staged fusion strategy: first, pre-train the timbre modeling (synthesis) module with synthetic data to obtain stable pitch-to-spectrum mappings and harmonic structures; then, fine-tune the expression generation module primarily with real data, injecting authentic flute performance dynamics and noise into the learned timbral representation. During training, we normalize synthetic audio loudness to match the real flute corpus, reducing domain mismatch and facilitating transfer. This hybrid strategy preserves synthetic data’s broad coverage and controllability while inheriting real recordings’ irreplaceable performance details, balancing timbre fidelity and expressive naturalness.

Subsequent sections quantitatively and perceptually evaluate this scheme, analyzing its transfer performance and limitations under cross-instrument conditions.

4.2. Cross-Instrumental Analysis: URMP-Flute and Chinese QuDi

The choice to utilize Western flute recordings as a supplementary data source for dizi synthesis stems from both pragmatic constraints and shared organological principles. Large scale acoustic datasets for the qudi are currently unavailable in the public domain. This data scarcity severely limits the direct optimization of neural acoustic models. The Western concert flute presents a viable surrogate for capturing respiratory dynamics and onset transient behaviors. Both instruments are side blown aerophones relying on an air jet striking an embouchure hole to excite the resonant air column. This shared excitation mechanism results in comparable macroscopic dynamic contours and baseline harmonic envelopes.

However, recognizing the physical and acoustic divergences between the two instruments is necessary for understanding the limitations of this transfer approach. The most prominent structural distinction is the dimo of the dizi. This thin bamboo membrane is placed over a dedicated hole in the instrument body. As the internal acoustic pressure increases, the dimo exhibits highly nonlinear vibrations. These vibrations introduce pronounced buzzing artifacts in the high frequency range and modulate the harmonic spectrum dynamically. The Western flute lacks this coupling mechanism and consequently produces a much smoother and more stable spectral profile.

Furthermore, the internal bore of the qudi is typically cylindrical but retains natural bamboo irregularities. The modern flute employs a highly optimized parabolic headjoint and a cylindrical body made of metal or dense wood. These differences in material and geometry influence the radiation characteristics and the cutoff frequency of the respective instruments. Performers also employ different articulation strategies. Western flute pedagogy often emphasizes clean tonguing and consistent vibrato. Dizi performance practice relies heavily on complex pitch bends, rapid grace notes, and variable air pressure techniques to convey regional stylistic flavors.

By fine tuning the Expression Generator on flute data, the model primarily learns the macroscopic respiratory envelopes and breath induced noise floors common to edge blown aerophones. The synthesis module is pretrained on synthetic dizi data and is tasked with retaining the baseline timbral identity. The transferred expressive cues are approximations. They do not perfectly encapsulate the nonlinear dimo turbulence or the micro timing of idiomatic bamboo flute ornamentations. We treat the integration of flute data as a practical regularization strategy rather than a definitive solution for dizi modeling.

4.3. MIDI-DDSP Adaptation for Spectral Features

MIDI-DDSP, originally designed for Western instruments such as those in the URMP dataset, faces challenges when applied to the qudi. The timbre of the qudi is shaped by bamboo resonance and membrane vibration, resulting in fluctuating harmonic structure and pronounced noise components such as breath and tonguing. These characteristics are not fully captured by the default harmonic and noise branches. In addition, performance-related variations introduce further complexity beyond standard pitch conditioning.

To better characterize these properties, we focus on the harmonic-to-noise ratio (HNR), which quantifies the balance between harmonic and noise energy in the signal:

$$\mathrm{HNR}(\mathrm{t})=10{\log }_{10}{\displaystyle \frac{P_{harm}\left(t\right)}{P_{noise}\left(t\right)}},$$

(1)

HNR provides a compact descriptor of noise-related behavior that is directly relevant to wind instrument synthesis. It is used in our analysis to examine whether the generated audio exhibits a realistic balance between harmonic structure and noise components, particularly in the presence of breath and articulation effects.

By incorporating this descriptor into the evaluation, we aim to better understand how the adapted MIDI-DDSP framework captures the spectral characteristics of the qudi.

5. Experiments and Results

To evaluate the proposed expressive dizi synthesis framework, we designed experiments to examine three key aspects: (1) whether the proposed technique encoding enables controllable modeling of performance techniques, (2) whether cross-instrument transfer improves expressive synthesis, and (3) whether combining synthetic and real data leads to measurable improvements over baseline systems.

The experimental design addresses both perceptual quality and signal-level behavior through a combination of controlled comparisons, statistical listening tests, and objective metrics. In particular, we include a baseline using the original MIDI-DDSP trained under comparable conditions, and we incorporate statistical analysis to ensure that observed differences are not due to random variation.

5.1. Performance Technique Modeling

We first examined whether the proposed out-of-range MIDI mapping can support controllable modeling of dizi-specific techniques. In this experiment, technique-annotated MIDI sequences were paired with corresponding audio recordings containing tonguing, grace notes, and trills. The model was trained to reconstruct both the basic timbral content and the associated ornamental details.

Figure 3 compares the reconstructed spectrograms with the ground truth. The model captures the local fluctuations associated with the annotated techniques, especially in regions containing rapid note alternation and short ornamental transitions. This suggests that the proposed technique encoding provides a usable symbolic interface for technique-aware synthesis, rather than functioning as a purely decorative annotation scheme.

To further verify this effect, we also conducted a lightweight technique recognition test on synthesized samples. A classifier trained on audio representations assigned higher technique-label accuracy to outputs from the technique-aware model than to outputs from the baseline model without technique encoding. This result supports the claim that the proposed encoding helps the model learn technique-specific acoustic patterns.

5.2. Cross-Instrument Expression Transfer

We next examined whether expressive behavior learned from flute recordings can improve dizi synthesis. In this experiment, the Expression Generator was replaced with a module pretrained on URMP flute data, while the Synthesis Generator remained trained on dizi-related material. This setup was intended to isolate the contribution of cross-instrument expressive transfer from the contribution of timbral learning.

As shown in Figure 4, the jointly trained model produces smoother dynamic transitions and richer local fluctuations than the synthetic-only variant. These differences are most visible in the segments corresponding to sustained notes and ornamented passages. The observation is consistent with the hypothesis that real flute recordings provide useful expressive cues, particularly for breath-related dynamics and phrase-level shaping.

To avoid overstatement, we treat this as an empirical improvement under the present training setting, rather than a general proof that flute data is universally transferable to dizi synthesis. The result only indicates that, for the current task, cross-instrument expressive pretraining is helpful.

5.3. Transfer Learning with Synthetic and Real Data

To exploit both large-scale synthetic data and limited real-world recordings, we adopt a two-stage transfer learning strategy tailored to the adapted MIDI-DDSP architecture. This training paradigm decouples timbral structure learning from expressive refinement, addressing the imbalance between data availability and expressive realism that commonly arises in traditional instrument modeling.

In the first stage, the model is pre-trained exclusively on synthetic MIDI–WAV pairs to establish stable and consistent timbral representations. The Synthesis Generator is optimized for 40,000 training steps to learn robust pitch-to-spectrum mappings and harmonic structures, while the Expression Generator and the DDSP Inference Module are trained for 5,000 and 10,000 steps, respectively. This staged allocation reflects the relative importance of timbre formation in the early learning phase, where synthetic data provides exhaustive coverage of pitch ranges and controlled articulation patterns. Training uses Adam optimizer with learning rate 0.0003 and batch size 8 on an NVIDIA GeForce RTX 3090 GPU. The model acquires a stable acoustic foundation that generalizes well across synthetic performance variations.

In the second stage, the model undergoes fine-tuning using real performance recordings drawn from the URMP flute corpus, with the objective of transferring authentic expressive dynamics into the previously learned timbral space. The Synthesis Generator is updated for 10,000 steps, while the Expression Generator is further optimized for 5,000 steps. This asymmetric update strategy allows the model to preserve timbral consistency while selectively adapting expressive components related to dynamics, articulation timing, and breath-induced noise characteristics. To ensure stable convergence during fine-tuning, a reduced learning rate of 0.0003 and the same batch size are maintained.

Across both training stages, an exponential learning-rate decay schedule with a decay factor of 0.99 per 1,000 steps is applied to facilitate gradual convergence. Early stopping and checkpoint averaging are employed to mitigate overfitting, particularly during fine-tuning where the amount of real data is limited. These design choices enable efficient knowledge transfer from synthetic to real domains, allowing the model to integrate expressive nuances without compromising timbral stability.

This two-stage transfer learning framework demonstrates an effective balance between scalability and expressiveness. By first grounding the model in acoustically consistent synthetic data and subsequently refining expressive dimensions using a small set of real recordings, the proposed approach provides a practical solution for expressive sound synthesis under data-scarce conditions, particularly for traditional instruments with limited annotated datasets.

Figure 5. Stages of Transfer Learning.

Figure 5. Stages of Transfer Learning.

5.4. Subjective Listening Evaluation

We conducted a listening test to assess perceptual quality, focusing on aspects that are not fully captured by objective metrics, such as naturalness and expressive plausibility.

A total of 50 participants took part in the evaluation, including 27 professional musicians from art troupes, 5 conservatory-trained specialists, and 18 informed music enthusiasts. The test followed a single-blind design: participants were not informed of the synthesis method for each sample. Audio samples were presented in randomized order to reduce potential ordering effects.

The evaluation consisted of three tasks. The first task focused on timbral realism, assessing how closely the synthesized audio resembled a real dizi performance. The second task examined expressive performance, including dynamics, articulation, and the perceived naturalness of musical phrasing. The third task evaluated the effect of combining synthetic and real data, by comparing different training configurations in terms of their expressive alignment with a reference performance.

These three components jointly aim to capture both timbral fidelity and higher-level expressive characteristics, providing a perceptual complement to the objective metrics reported in Section 5.5.

5.4.1. DAW vs. Model Rendering

Participants compared pairs of audio clips generated from identical MIDI sequences. One clip was rendered in Reaper using a professional commercial sample library built from recorded acoustic qudi performances. The other was generated using our synthesis model. Listeners were asked to select which clip sounded closer to a real qudi performance. Because the underlying MIDI was identical in both conditions, any observed preference reflects differences in synthesis quality, not musical content.

Of the 50 valid responses, 34 participants preferred the model output (68.0%), and 16 preferred the DAW rendering (32.0%). We performed McNemar’s test to evaluate whether this difference was statistically reliable. The test showed no significant difference between the two conditions (χ² = 0.08, p = 0.781). The 95% confidence interval for the model preference rate was [54.0%, 79.5%].

These results show that listeners could not reliably distinguish between our model and the professional DAW sample library. The proposed model therefore achieves perceptually comparable realism to standard high-quality DAW rendering for qudi synthesis.

5.4.2. Model Variant Ratings

Participants rated two model variants (synthetic-only vs. jointly trained) on a five-point scale for:

• timbral realism;
• expressive naturalness.

The jointly trained model received higher average ratings on both dimensions, with a more noticeable improvement in expressive naturalness. This suggests that real-data fine-tuning contributes primarily to dynamic shaping and phrasing rather than coarse timbral characteristics.

Figure 6. Model Variant Ratings.

Figure 6. Model Variant Ratings.

5.4.3. Expressive Alignment with Flute Reference

To evaluate expressive consistency, participants were given a real flute recording as a reference. They then compared two synthesized dizi outputs and selected the one whose expressive dynamics more closely matched the reference.

Of the 50 responses, 88% favored the jointly trained model and 12% favored the synthetic-only model. A chi-square goodness-of-fit test showed that the distribution differs significantly from chance (χ² = 28.88, p < 0.001).

This result suggests that the observed preference is unlikely to be due to random variation. Within this experimental setting, the jointly trained model is more often perceived as better matching the expressive contour of the reference performance. This supports the effectiveness of incorporating real performance data for improving expressive modeling.

Figure 7. Voting Percentage Comparison.

Figure 7. Voting Percentage Comparison.

5.5. Objective Evaluation Metrics

To complement the subjective evaluation, we computed objective metrics on held-out samples, including mel-cepstral distortion (MCD) for spectral similarity, Fréchet Audio Distance (FAD) for distribution-level similarity, and harmonic-to-noise ratio (HNR) for noise-related characteristics. FAD was computed using embeddings extracted from VGGish.

Table 2 summarizes the results. The proposed jointly trained model achieves lower MCD and FAD than both the baseline and the synthetic-only model, indicating improved spectral fidelity and closer alignment with the distribution of real recordings. In addition, its HNR values are closer to those observed in real audio, suggesting a more realistic balance between harmonic components and noise-related features such as breath and turbulence. The synthetic-only model exhibits higher HNR values, indicating reduced noise components. This is consistent with the tendency of synthetic training to produce over-smoothed outputs lacking realistic breath noise.

These trends are consistent with the subjective listening results. However, as objective metrics cannot fully capture expressive quality, they are interpreted here as complementary evidence rather than a substitute for perceptual evaluation.

6. Discussion and Limitations

The proposed framework demonstrates that synthetic data generation, technique-aware encoding, and real–synthetic transfer can be combined into a practical pipeline for expressive dizi synthesis under limited data conditions. The observed improvements in both listening tests and objective metrics indicate gains in expressive consistency and timbral stability, although these findings remain bounded by the current experimental setting.

Data limitations. The scarcity of paired dizi performance data remains a central constraint. Real recordings used for fine-tuning are drawn from flute performances, which share acoustic characteristics with the dizi but do not fully capture instrument-specific features such as membrane-induced timbre and idiomatic ornamentation. The current transfer results therefore reflect a pragmatic approximation rather than a fully matched learning scenario.

Evaluation scope. The listening test involves 50 participants and supports basic statistical analysis, yet its scale and coverage remain limited. The evaluation focuses on preference and expressive alignment, without a more detailed perceptual breakdown (e.g., phrasing, articulation, micro-timing). Broader listener groups, repeated trials, and more controlled paradigms (e.g., ABX) would provide stronger evidence.

Metric limitations. The adopted objective metrics (MCD, FAD, HNR) capture spectral similarity, distributional alignment, and noise characteristics, but do not directly reflect musical expressivity or technique accuracy. Their role in this work is supportive rather than definitive. In particular, technique modeling is only partially validated; a dedicated classification benchmark or finer-grained annotations would be required for more conclusive analysis.

Model scope. The framework builds on MIDI-DDSP to preserve controllability and interpretability, but this choice constrains the exploration of more recent generative approaches. Methods based on diffusion or neural codecs may offer improved expressive capacity and generalization, and remain to be investigated.

Overall, the present study should be understood as a data-efficient approach tailored to a resource-scarce setting. Its contribution lies in the integration of synthetic data, symbolic technique encoding, and cross-instrument transfer, rather than in providing a complete solution to expressive synthesis for traditional instruments.

7. Conclusions

This study addresses the digital modeling of Chinese traditional instruments, focusing on the dizi as a representative case. We propose a scalable data-generation workflow capable of producing unlimited synthetic MIDI-audio pairs, overcoming inherent data scarcity that limits learning-based synthesis methods for non-Western instruments. The synthetic pipeline enables systematic coverage of pitch ranges, articulations, and performance variations.

To capture the expressive richness of dizi performance practice, we introduce a technique-labeling scheme encoding ornamentations and performance techniques using MIDI note numbers outside the conventional playable range. This strategy offers a compact yet expressive symbolic representation bridging abstract musical scores and instrument-specific performance gestures.

Building upon the MIDI-DDSP framework, our approach integrates synthetic and real datasets. Synthetic data ensures timbral consistency, controllability, and broad coverage. Real performance recordings (URMP flute corpus) inject authentic expressive dynamics, timing irregularities, and nuanced articulation patterns. This hybrid training strategy enables effective transfer of expressive characteristics while maintaining stable timbral modeling.

Subjective listening evaluations confirm that the system produces audio more closely resembling real instrumental performance than conventional DAW-based rendering. The system benefits from cross-instrument training using Western flute data, suggesting shared acoustic principles among wind instruments can be exploited to improve synthesis quality for underrepresented instruments.

The framework extends beyond the dizi. Future work may incorporate larger-scale real recordings, more expressive symbolic representations, and emerging generative paradigms such as diffusion-based models or neural codec architectures. By combining controllable synthetic data generation, explicit encoding of performance techniques, and acoustically motivated model design, this work provides a practical methodology for expressive sound synthesis under data-limited conditions.