Submitted:
09 June 2026
Posted:
10 June 2026
You are already at the latest version
Abstract
Background With the advancement of sequencing technologies and increasing quality of assembly and annotation tools, the amount of biological sequence data produced by research grows rapidly. To keep this data accessible for potential re-analysis, it is typically stored in public online databases, such as the European Nucleotide Archive (ENA). However, the increase in data entails an increase in time needed for metadata curation and upload. We propose a tool to simplify and automatize the upload of experimental and metadata to public repositories. Results We developed ENflorA, a set of scripts by which sample, sequence read and (annotated) assembly data can be uploaded to ENA in a simple and standardized manner. Our software allows easy metadata entry via spreadsheets, is suitable for batch upload of data from different sources, and can be fully integrated in bioinformatic workflows. It is platform independent and can be run natively on High Performance Computers, integrates an lftp upload option for big files and a test upload option. Conclusions By easing the ENA submission process, ENflorA contributes to making high-throughput sequencing data findable, accessible, interoperable and reusable.
Keywords:
ENA
; FAIR data
; file transfer protocol
; INSDC
; high-throughput sequencing
; plastome
; submission
Background
High-throughput sequencing technology has unlocked access to realms of new data from both human and non-human genomes, creating entirely new fields of research, particularly in biodiversity genomics [1,2]. While per-base sequencing prices continue to fall [3], the overall cost and effort involved in producing this data, as well as the wealth of information contained in it, thoroughly justify its long-term storage for potential reuse. Beside the need for dedicated infrastructure to store petabyte-sized amounts of digital DNA sequences, data stewardship itself increasingly becomes a bottleneck: to make data findable, accessible, interoperable and reusable (FAIR principles [4]), it needs to be well documented with associated metadata, uploaded to a freely accessible, searchable online database, and formatted in accordance with existing data standards. As also the sequencing projects continue to grow [1,2,5], these time-consuming, intertwined tasks can no longer be managed “by hand”, but need automation, both to minimize errors and to be achievable within a set time frame.
Public online databases, allowing researchers to upload/download (DNA) sequences and associated metadata, stored in standardized formats which facilitate searches and reuse, are a cornerstone of modern genetic research. In 1982, the International Nucleotide Sequence Database Collaboration (INSDC) was founded as a joint global effort to store this type of data and to make it available to researchers worldwide [6]. This major effort to coordinate the global exchange and long-term stewardship of nucleotide sequence data across partner archives, including the European Nucleotide Archive(ENA [7]), GenBank (NCBI [8]), and the DNA Data Bank of Japan (DDBJ [9]), has facilitated thousands of research projects and continues to ensure their repeatability, and thus scientific soundness. By regular mirroring of data between geographically independent infrastructure, the INSDC to date remains the safest standard for global research, though it is not the only DNA sequence “vault” (e.g., China National GeneBank CNGB [10]).
In Europe, DNA sequence data is mainly brokered and maintained through ENA, a European Molecular Biology Lab (EMBL) infrastructure founded in 1980 and currently managed by the European Bioinformatics Institute (EBI). ENA submissions typically require consistent handling of multiple linked object types according to ENA’s own metadata model (Figure 1), including “Biosamples”, raw read libraries (“Experiments”/”Runs”), and genome assemblies/annotations (“Analyses”), each with strict metadata requirements and validation constraints. The submission tools provided by ENA itself are dichotomous: the online Webin Portal offers an intuitive graphical interface which is, however, functionally unscalable for bulk operations, leading to repetitive manual metadata entry and high error rates. Conversely, the Webin-CLI tool offers a robust validation environment suitable for bulk data, but requires the user to programmatically generate structured manifest files for every single data object, a task that necessitates scripting skills and thus also leaves a significant part of data curation with the user.
While large sequencing centers or projects may have bespoke, automated pipelines to push data to INSDC nodes [11], smaller or geographically distributed research groups often struggle with the technical complexity of submission interfaces. Platforms like COPO (Collaborative OPen Omics [12]) have emerged to address this by providing centralized data brokering services. While powerful, such platforms typically require users to operate within specific web ecosystems and rely on external infrastructure and support, and may not be generally accessible. This may cause a significant “submission gap”: submission, and thus publication, may be significantly delayed, or existing data – especially metadata, large raw sequence files and further analysis results – lost through being omitted to “save time” or reduce effort [13,14]. Thus, a “middleware” solution for sequence data submission is needed — a tool that runs locally on a user’s machine or High-Performance Computing (HPC) cluster, integrates with metadata handling in standard scientific collection and laboratory workflows via spreadsheets, and automates low-level interactions with ENA’s submission API even for bulk data without requiring the overhead of full platform adoption.
The new software ENflorA was developed to help close the submission gap for NGS data, especially, yet not exclusively, for medium-sized projects without access to custom-made, in-house submission pipelines. It provides a lightweight, script-based workflow that standardizes the conversion of user tables into ENA biosample, run/experiment and analysis submission artifacts. Its design is pragmatic: it keeps the user interface familiar (Excel/TSV tables), enforces schema constraints early, and automatically repeated steps such as staging, compression, manifest generation, and structured logging. The result is a small toolkit that helps research groups move from “metadata and sequence files” somewhere in their local research data storage, to a “submission-ready data package” with a reproducible, auditable process.
Implementation
ENflorA is implemented as a modular suite of Python 3 scripts, designed to be platform-independent and easily deployable in diverse computing environments, from personal laptops running under Windows, macOS or Linux, to institutional HPC clusters using the slurm task manager. The tool consists of three primary modules: biosamples.py, runs.py, and analysis.py, which mirror ENA’s hierarchical object model (Figure 1). While the typical use case would be the submission of first sample metadata (biosamples.py), then the associated read files (runs.py) and finally analysed data such as assembled plastid genomes (analysis.py) associated to a pre-existing Bioproject (created through the ENA Webin Portal), all three parts can be used independently and do not assume a bijective relationship between the handled data objects.
1. Environment Management
To ensure reproducibility and ease of use, ENflorA includes a wrapper script that automates the initialization of its runtime environment. This wrapper handles the creation of a dedicated Python [15] virtual environment and installs necessary dependencies, including pandas [16] for data manipulation, pyyaml [17] for configuration management, openpyxl [18] for Excel support, and Biopython [19] for sequence formatting. Crucially, it also manages the retrieval and path configuration of the ENA Webin-CLI Java archive (.jar), ensuring that the user always validates against a compatible version of ENA’s data standards.
2. Metadata Parsing and Validation
To date, most researchers and lab workflows store, or export, metadata in spreadsheets. ENflorA therefore uses this common and easy to manipulate format for metadata input, accepting both standard Excel (.xlsx) or tab-separated value (.tsv) files. ENflorA’s parsers expect specific column names that map to the corresponding ENA fields. For runs.py and analysis.py, these column names follow ENA’s own Webin-CLI manifest field names directly (e.g., STUDY, SAMPLE, INSTRUMENT, ASSEMBLY_TYPE), which are documented in the bundled template spreadsheets. For biosamples.py, the expected columns implement the ENA plant sample checklist ERC000037, with mandatory and recommended fields enforced at the validation step; additional optional attributes accepted by ENA can be included by simply adding extra columns to the spreadsheet, which are then forwarded to ENA as supplementary sample attributes. To submit data under a different ENA checklist (e.g., for non-plant organisms), the hardcoded checklist identifier and field lists may be replaced in biosamples.py; the full list of ENA checklists is browsable online [7].
On invocation, each script performs a “pre-flight” validation of the associated metadata spreadsheet. This consists of two steps, syntax checking and file validation: first, it checks that all required columns are present and that mandatory fields are not empty. In biosamples.py, this includes parsing collection dates into ISO 8601 format (YYYY-MM-DD) from common alternatives like DD.MM.YYYY, and verifying that the ENA plant checklist’s mandatory fields are not empty; in runs.py, it checks that file type columns (FASTQ, BAM, CRAM) are present and consistent; in analysis.py, it verifies that exactly one of FLATFILE or FASTA is specified per row and that the assembly level is valid. Second, all scripts verify that referenced data files exist on the local disk and are readable.
3. Module Breakdown
The three ENflorA scripts are run independently, so as to incorporate data identifiers created during the previous step. In consequence, the scripts may also be run individually, depending on the requirements of the project, e.g., if previously available read data already stored in ENA are reused to produce a new Analysis object (e.g., a plastome). Before the first script can be run, an ENA submission account is needed, and a Bioproject, with which all the newly uploaded data will be associated, needs to be created by hand.
The first module, biosamples.py, manages the registration of biological source material to create Biosample objects. It generates the necessary XML submission files by mapping spreadsheet columns to ENA checklist attributes (e.g., geographic location, specimen voucher, collection date; based on the ENA sample data checklist “ERC000037”). It supports batch registration, returning and logging the accession numbers (SAMEAxxxx) required for linking downstream data.
The second module, runs.py, handles raw read data and creates Experiment and Run objects in ENA. It triggers the automated validation and MD5 checksum calculation via the Webin-CLI for large FASTQ files, which is a mandatory step for ENA integrity verification. It constructs the Webin-CLI manifest files, defining the relationships between the Run, the Experiment (library strategy, source, sequencing instrument, selection), and the Sample, and returns the resulting accession numbers.
The third module, analysis.py, handles derived data, such as genome assemblies and annotations, creating ENA Analysis objects. Beyond the submission itself, it can also perform automated file conversion: ENA requires annotated assemblies to be submitted in the EMBL flatfile (.embl) format, which differs significantly from the GenBank (.gb) format put out by some sequence analysis tools (e.g., PlastidPipeline [20]). Through integration with Biopython, analysis.py can convert a Genbank file into a valid EMBL flat file, correctly mapping feature qualifiers to INSDC standards.
To make submission outputs easy to retrieve in both interactive and batch (HPC/SLURM) settings, all three modules write the accessions assigned by ENA into a plain-text file inside their respective submission folder (biosample_accessions.txt, run_accessions.txt, analysis_accessions.txt). These files are appended across runs and deduplicated, with test-server entries clearly flagged. The same accessions are also printed to standard output as they are returned by ENA, so they remain visible in SLURM .out log files. This separation between full submission logs (per-sample, verbose) and a flat accession registry (per-script, machine-readable) directly supports the user’s next workflow step, where the previous object’s accession must be entered into the next spreadsheet.
4. Robust Data Transfer with lftp
Submitting terabytes of raw sequencing data over standard hypertext transfer protocol (HTTP) or basic file transfer protocol (FTP) connections is prone to failure due to network instability. ENflorA addresses this by integrating submission via lftp [21]: this robust FTP client supports automatic resumption of interrupted transfers. ENflorA includes a standalone helper script, independent from the three main Python modules, for lftp that mirrors the local staging directory to the ENA’s FTP server, webin2.ebi.ac.uk. This ensures that, if a connection drops e.g., during the overnight upload of a 100 GB dataset, the process can resume automatically from the last successful byte, rather than restarting from zero.
5. Testing
Beside “live” submission, the three ENflorA scripts also allow testing the workflow and input data through “test” submission to the ENA Webin test server. From there, data is not made public, but automatically gets deleted after several days. To make this testing immediately accessible to new users, ENflorA includes a dedicated “--demo” mode: when any script is invoked with this flag, it uses bundled synthetic test data (biosample metadata file, paired-end FASTQ reads, and both FASTA and EMBL flat-file assemblies) and submits them to the ENA test server, producing verbose logs that trace every step of the submission. This allows new users to end-to-end verify their entire setup, consisting of the credentials, the Python environment and the Java Webin-CLI, before preparing real data. The software can thus also be tested early on in a project before the actual project data arrives, e.g., by using already published data downloaded from ENA for a ‘dry run’. The demo dataset is fully synthetic and has no biological meaning: nucleotides are generated at random with uniform base composition and FASTQ quality scores assigned at random within a fixed Phred range (Q20-Q40). It comprises one plant biosample according to ENA checklist ERC000037, 100 paired-end reads of 150 bp, and two small contig-level assemblies, one unannotated as a FASTA file (two ~500 bp contigs) and one annotated as an EMBL flat file. These files pass format validation and are therefore accepted by the ENA test server, but test submissions are automatically deleted the next day.
We tested ENflorA on a variety of data, produced in several research projects at FU Berlin. This included a phylogeographic study on the medicinal plant Arnica montana L. (PRJEB89801) [20] and a phylogenetic study on the Cunnoniaceae genus Weinmannia L. (PRJEB107803) [22], involving both new and already databased samples / read data, as well as new raw sequences and plastome assemblies (Additional file A1). Upload was tested both from local computers and FU Berlin’s HPC system Curta [23].
Discussion
The rapid expansion of genomic research necessitates tools that can bridge the gap between data generation and data archiving. ENflorA addresses the specific needs of individuals, groups and small to medium-sized projects that generate high-value biodiversity data, but lack the dedicated bioinformatics engineering support of large sequencing centers.
By decoupling the definition of metadata (in spreadsheets) from the technical mechanics of submission (XML/manifest generation), ENflorA allows researchers to maintain control over their data, using familiar tools while ensuring compliance with INSDC standards. Unlike manual portal submissions, where users often omit optional metadata to save time, ENflorA’s template-based approach encourages the completion of rich metadata fields, such as geolocation, collection date, and specimen vouchers, before the submission process begins. This ensures that the resulting data is findable and reusable for future meta-analyses, contributing towards the FAIR data goal. Furthermore, by automating the conversion of annotation files (e.g., from GenBank format to ENA-compliant EMBL flatfiles) and reusing standard NCBI Taxonomy identifiers, the tool ensures a high level of interoperability.
In comparison to existing solutions, ENflorA occupies a unique niche: while Webin-CLI provides the “gold standard” for validation, it acts primarily as a validator and uploader rather than a workflow manager. By itself, it does not natively handle the batch logic required to link hundreds of samples to their respective runs and assemblies. Conversely, centralized platforms like COPO offer excellent wizard-driven interfaces but introduce dependencies on external servers and web-based workflows that may not align with local HPC data storage policies. ENflorA serves as a lightweight middleware that combines the batch-processing power of a command-line tool with the usability of a spreadsheet-driven workflow, running entirely within the user’s local infrastructure.
Its modular, yet compact design with one script per type of submission is an asset for projects working with different data sources: previously submitted samples and/or raw sequences can easily be referenced in the submission of “downstream” data types, and submitted alongside their counterparts resulting from new field or lab work. Equally, the scripts can easily be used by geographically distributed research projects, either for decentralized submission to the same project, or through gathering sequence files and format-compatible metadata spreadsheets from multiple groups for central submission by a data manager. It will thus significantly reduce the time spent on data deposition, freeing up resources e.g., for more thorough validation or to link additional metadata. The scripts can also be fully integrated into bioinformatic data analysis workflows, resulting in their better documentation and traceability.
While ENflorA’s “botanical” name reflects its origins, its architecture is fundamentally agnostic towards the (taxonomic) origin of the submitted data. Currently, biosamples.py has plant-specific metadata requirements (the ENA plant checklist ERC000037), and analysis.py defaults to a single circular plastid chromosome for chromosome-level submissions; both can be customized, e.g., for non-plant samples, as documented in the README. Future versions of the software might include a way to automatically adapt the underlying ENA checklist, or to submit additional types of analysis data.
Conclusions
ENflorA provides a streamlined, user-friendly solution for the bulk submission of NGS data, effectively removing the technical barriers that often delay or discourage the deposition of genomic data into public repositories. By automating the translation of standard laboratory spreadsheets into ENA-compliant formats, the tool empowers researchers to adhere to FAIR data principles without requiring extensive bioinformatics expertise. As the volume of biodiversity data continues to grow, lightweight middleware tools like ENflorA will be essential for ensuring that this data is not only stored but remains discoverable, structured, and valuable to the global scientific community.
Availability and Requirements
Project name ENflorA
Project home page https://github.com/alfarodeprado/ENflorA
Operating system(s) Platform independent (tested on CentOS 7 Linux, Windows 11 and MacOS 12.7.6)
Programming language Python (primarily), Bash; Java required for Webin-CLI
Other requirements
Mandatory: Python 3.8+, pandas (≥1.2), PyYAML (≥5.0), openpyxl (≥3.0), Java 17+, ENA Webin-CLI (≥7.0; tested with 8.2.0)
Optional: Biopython (≥1.78; .gb to .embl file format conversion), lftp (≥4.6; FTP-based upload), HPC with SLURM (file upload from HPC)
License MIT
Any restrictions to use by non-academics None
Supplementary Materials
The following supporting information can be downloaded at the website of this paper posted on Preprints.org, ENflorA_AdditionalFileA1.pdf. Additional file A1 – Project data uploaded with ENflorA. IDs of all data objects submitted with ENflorA to the European Nucleotide Archive (https://www.ebi.ac.uk/ena).
Author Contributions
AAM prepared this work during a student assistantship under the supervision of KR and AM. AAM wrote the source code, and improved it based on test feedback from FFG and KR. AAM and AM participated in the conceptualization, which was led by KR. AM secured funding and FFG suggested the software’s name. AAM and KR wrote an initial draft of the manuscript, which was subsequently refined by AM, FFG and KR. All authors read and approved the final version of the manuscript.
Funding
This study was funded by an FU Berlin START program grant, awarded to AM. Open Access funding was provided by Freie Universität Berlin.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
The source code described in this article, including the synthetic test data, is available from https://github.com/alfarodeprado/ENflorA . Project data used to test live upload is available from ENA, https://www.ebi.ac.uk/ena/, under the project IDs PRJEB89801 and PRJEB107803, and further IDs listed in Additional file A1.
Acknowledgments
We thank Thomas F. Brown from the Berlin Center for Genomics in Biodiversity Research and Leibniz Institute for Zoo and Wildlife Research for valuable information on ENA and COPO, our colleagues at BO Berlin, especially Belen Escobari, Jozefien Van de Velde and Konstantina Koutroumpa, for constructive feedback on the functionality of the software, and HPC Service of FUB-IT, Freie Universität Berlin, for computing time.
Conflicts of Interest
The authors declare that they have no competing interests.
Abbreviations
API: Application Programming Interface
COPO: Collaborative OPen Omics
DDBJ: DNA Data Bank of Japan
EMBL: European Molecular Biology Laboratory
ENA: European Nucleotide Archive
FAIR: Findable, Accessible, Interoperable, Reusable
FTP: File Transfer Protocol
HPC: High‐Performance Computing
HTS: High‐Throughput Sequencing
HTTP: Hypertext Transfer Protocol
INSDC: International Nucleotide Sequence Database Collaboration
JAR: Java Archive
NCBI: National Center for Biotechnology Information
NGS: Next‐Generation Sequencing
TSV: Tab‐Separated Values
WGS: Whole Genome Sequencing
XML: Extensible Markup Language
References
- Blaxter M, Lewin HA, DiPalma F, Challis R, Da Silva M, Durbin R, et al. The Earth BioGenome Project Phase II: illuminating the eukaryotic tree of life. Front Sci. 2025;3:1514835. [CrossRef]
- Lavergne S, Pouchon C, Roquet C, Alberti A, Boleda M, Perrier C, et al. Towards a comprehensive barcoding and phylogenomic reference for the European arctic–alpine flora. Bot Lett. 2025;172:495–507. [CrossRef]
- Wetterstrand K. DNA Sequencing Costs. DNA Sequencing Costs: Data from the NHGRI Genome Sequencing Program (GSP). https://www.genome.gov/about-genomics/fact-sheets/DNA-Sequencing-Costs-Data. Accessed 26 Apr 2026.
- Wilkinson MD, Dumontier M, Aalbersberg IjJ, Appleton G, Axton M, Baak A, et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016;3:160018. [CrossRef]
- Formenti G, Theissinger K, Fernandes C, Bista I, Bombarely A, Bleidorn C, et al. The era of reference genomes in conservation genomics. Trends Ecol Evol. 2022;37:197–202. [CrossRef]
- Karsch-Mizrachi I, Arita M, Burdett T, Cochrane G, Nakamura Y, Pruitt KD, et al. The international nucleotide sequence database collaboration (INSDC): enhancing global participation. Nucleic Acids Res. 2024. [CrossRef]
- David Y, Alisha A, Awais A, Rajkumar D, Dipayan G, Muhammad H, et al. The European Nucleotide Archive in 2025. Nucleic Acids Res. 2026;54:D120–7. [CrossRef]
- Sayers EW, Cavanaugh M, Frisse L, Pruitt KD, Schneider VA, Underwood BA, et al. GenBank 2025 update. Nucleic Acids Res. 2025;53:D56–61. [CrossRef]
- Ara T, Kodama Y, Fujisawa T, Kosuge T, Lee K, Mashima J, et al. DDBJ update in 2025: system integration for global data-sharing including pathogen surveillance. Nucleic Acids Res. 2026;54:D106–10. [CrossRef]
- Guo X, Chen F, Gao F, Li L, Liu K, You L, et al. CNSA: a data repository for archiving omics data. Database. 2020;2020:baaa055. [CrossRef]
- The Darwin Tree of Life Project Consortium, Blaxter M, Mieszkowska N, Di Palma F, Holland P, Durbin R, et al. Sequence locally, think globally: The Darwin Tree of Life Project. Proc Natl Acad Sci. 2022;119:e2115642118. [CrossRef]
- Shaw F, Minotto A, McTaggart S, Providence A, Harrison P, Paupério J, et al. COPO - Managing sample metadata for biodiversity: considerations from the Darwin Tree of Life project. Wellcome Open Res. 2024;7:279. [CrossRef]
- Bergström A. Improving data archiving practices in ancient genomics. Sci Data. 2024;11:754. [CrossRef]
- Speir ML, Teh WK, Perry MD, Schwartz R, Nejad P, Harris T, et al. Making genomic data FAIR through effective Data Portals. Sci Data. 2025;12:1872. [CrossRef]
- Python Software Foundation. Welcome to Python.org. Python.org. 2026. https://www.python.org/. Accessed 1 May 2026.
- McKinney W. Data Structures for Statistical Computing in Python. Austin, Texas; 2010. p. 56–61. [CrossRef]
- Simonov K. PyYAML. https://pyyaml.org/. Accessed 26 Apr 2026.
- Gazoni E, Clark C. openpyxl - A Python library to read/write Excel 2010 xlsx/xlsm files — openpyxl 3.1.3 documentation. https://openpyxl.readthedocs.io/en/stable/. Accessed 26 Apr 2026.
- Cock PJA, Antao T, Chang JT, Chapman BA, Cox CJ, Dalke A, et al. Biopython: freely available Python tools for computational molecular biology and bioinformatics. Bioinformatics. 2009;25:1422–3. [CrossRef]
- Annaldasula SJ, Vera M, Casanova A, Borsch T, Reichel K. Towards whole plastome phylogeography: Resolving small genetic distances among European Arnica montana L. with the PlastidPipeline. 2025. [CrossRef]
- Lukyanov A. LFTP - sophisticated file transfer program. https://lftp.yar.ru/. Accessed 26 Apr 2026.
- Fajardo-Gutiérrez F, Calbi M, Dillenberger MS, Tello S, Fuentes A, Oleas NH, et al. Expelled by the Antarctic ice: Evolutionary history of the tribe Cunonieae (Cunoniaceae). J Syst Evol. 2025;63:1441–57. [CrossRef]
- Bennett L, Melchers B, Proppe B. Curta: A General-purpose High-Performance Computer at ZEDAT, Freie Universität Berlin. 2020. [CrossRef]
Figure 1.
Overview of ENA data objects and submission pathways. Includes metadata requirements for each step and sphere of application for the ENflorA scripts.
Figure 1.
Overview of ENA data objects and submission pathways. Includes metadata requirements for each step and sphere of application for the ENflorA scripts.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
