Artificial Intelligence in Cryo-EM Protein Particle Picking: The Hope, Hype, and Hurdles

I. Introduction

II. Emergence of AI in Particle Picking

1. Classical Particle Picking Methods

Earlier in 2004, Mallick et al. presented a pioneering machine learning approach for automated particle picking in cryo-EM micrographs, employing the Adaboost learning algorithm [14]. Their discriminative algorithm learned essential features of particle appearance from training examples, enabling generic detection unrestricted by particle shape or size. It trained a cascaded classifier using Adaboost, which was augmented by rotating test micrographs over various orientations to enhance detection versatility. However, the method encountered difficulties in assessing particle detection performance due to variability in particle labeling by different microscopists and the complexity of protein shapes in micrographs. The dataset used by the method only included ground truth labels only for particles of rectangular shape, leading to suboptimal performance of recognizing particles of circular shapes and potential biases in evaluation. Additionally, the method’s reliance on a specific feature type (rectangular features derived from templates) and a single training methodology, Adaboost, was noted as a limitation, suggesting that exploring more feature types and training approaches might offer greater accuracy.

gEMpicker [15] is a particle picking method leveraging parallel programming techniques distributed across GPUs and CPU cores in a computer cluster to efficiently gather and combine cluster calculation results, thereby enhancing overall picking throughput for generation of higher resolution 3D cryo-EM density maps from picked particles. gEMpicker’s performance was evaluated with various Fast Fourier Transform (FFT) libraries. Given that most computational costs stemmed from FFT-based normalized cross-correlation (NCC) calculations, the choice of FFT library significantly influenced the performance. This study found that FFT size and zero-padding were important for efficiency optimization.

Langlois et al.‘s method, AutoPicker [16], represented an advancement in selecting particle images in low-contrast conditions, outperforming manual selection on close-to-focus micrographs and leading to improved or comparable 3D reconstruction resolution. Leveraging unsupervised learning, AutoPicker minimized manual intervention, requiring only the approximate size of the macromolecule to be picked as input. Employing template matching and unsupervised learning, AutoPicker identified potential particles while excluding contaminants and noise windows, thereby enhancing particle selection accuracy. Its View Classifier (ViCer) further refined candidate particles, particularly in cases with contaminants, contributing to improved accuracy in noisy micrographs. The integration of real-time feedback mechanisms and adaptive algorithms was proposed to enhance the system’s performance over time for effective handling of diverse macromolecules.

APPLE Picker [17], developed in 2018, tackled cryo-EM particle picking with a template-free approach, reducing user effort and bias. By leveraging cross-correlation with a reference set derived from cryo-EM micrographs themselves, it efficiently identified particles and minimized false positives. However, this dependence on cross-correlation could be susceptible to noise mimicking particles, and the method’s effectiveness can vary substantially with respect to particle complexity.

Table 1. A list of classical AI methods for particle picking, their techniques and train/test data.

Table 1. A list of classical AI methods for particle picking, their techniques and train/test data.

SN	Method	Techniques	Train/Test Data	Year
1	Mallick et al.’s method [14]	Adaboost Learning Algorithm	Keyhole Limpet Hemocyanin (KLH) [18]	2004
2	gEMpicker [15]	Roseman’s NCC Matching Algorithm	Keyhole Limpet Hemocyanin (KLH)	2013
3	Langlois et al.’s method [16]	Principal Component Analysis (PCA) and Otsu’s Algorithm	VA-ATPase from T. Thermophilus HB8 And 70S Ribosome from E. Coli,	2014
4	APPLE picker [17]	Support Vector Machine	Β-Galactosidase, T20S Proteasome, 70S Ribosome, and Keyhole Limpet Hemocyanin (KLH)	2018
5	SuperCryoEMPicker [19]	Super-Clustering Approach	80S Ribosome and Beta-Galactosidase Datasets	2019
6	AutoCryoPicker [20]	Unsupervised Clustering	Apoferritin Dataset [21] and Keyhole Limpet Hemocyanin (KLH) Dataset	2019
7	Li et al. [22]	Segmentation-Aware Synergy Framework	EMPIAR-10028, EMPIAR-10097, and EMPIAR-10333	2022

Al-Azzawi et al. proposed two methods in 2019: AutoCryoPicker [20] and SuperCryoEMPicker [19]. AutoCryoPicker carried out micrograph cleaning and particle shape detection using a customized Circular Hough Transform algorithm. This technique efficiently identified particle shapes and centers, facilitating accurate particle selection through the creation of bounding boxes. Notably, the method incorporated a series of image preprocessing steps, and introduced an intensity distribution model for particle clustering, demonstrating superior performance over traditional clustering algorithms like K-means and Fuzzy C-Means (FCM) [20]. SuperCryoEMPicker introduced a super-clustering technique designed to handle complex shapes, irregularities, and low SNR in particle images. The approach consisted of three primary stages: pre-processing, particle clustering, and particle selection. It surpassed traditional clustering methods like k-means, fuzzy c-means (FCM), and intensity-based cluster (IBC) by incorporating super-pixel algorithms. The pre-processing stage involved various image enhancement procedures, such as global intensity adjustment, contrast enhancement, noise suppression, and edge enhancement, aimed at improving cryo-EM image quality and enabling more precise particle detection [19]. Finally, in the particle picking stage, a final set of particles was selected and picked from the clustered candidates after post-processing steps like binary mask cleaning and particle property measurement.

In 2022, Li et al. introduced a segmentation-aware synergy network, which incorporated an instance-level attention regression module during denoising to address vulnerable feature representation challenges and bridge the gap between denoising and recognition tasks [22]. By treating particle selection as a semantic segmentation problem, the method enhanced the semantic-level label generation pipeline, overcoming some limitations of traditional mask obtainment techniques, particularly for datasets with missing angle information. The proposed multitask ensemble learning framework enhanced the synergy between denoising and downstream recognition, resulting in reliable location estimations for single particle analysis. The method was trained with pixel-level denoising loss, instance-level denoising loss, and particle segmentation loss. The segmentation loss was implemented as the binary cross-entropy (BCE) loss function for supervised training.

2. Advanced Deep Learning Methods

In 2016, DeepPicker [23] introduced an iterative process to train its particle picking model, where top predicted particles from initial training steps were incorporated as new training data. It used a convolutional neural network (CNN) with a pooling layer to manage model complexity and prevent overfitting, thereby enhancing its capability to differentiate correct particles from background noise. Recall, precision, and center deviations were used to evaluate picked particles against ground-truth particles. Additionally, 2D clustering and class averaging were employed to validate the quality and accuracy of the selected particles. However, despite the efforts to mitigate overfitting through the pooling layer and iterative refinement, DeepPicker remained susceptible to overfitting the training data, limiting its generalizability to unseen datasets.

DeepEM [24] employed a deep CNN for single particle recognition, which was trained with labeled ‘good’ and ‘bad’ particles to differentiate desired features from undesired ones, facilitating template-free particle picking and reducing reference particle-dependent bias. The performance evaluation, conducted via precision-recall curves, demonstrated its good accuracy and efficiency in automated particle extraction. However, DeepEM was shown to be effective for feature-rich cryo-EM micrographs but struggled with datasets lacking distinctive particle features.

Xiao et al. presented an approach for automated particle picking in 2017 using Fast R-CNN [25], a deep learning-based object detection framework [26]. It employed a ‘Cross Molecule Training Strategy’ to train the CNN with particles of various protein complexes to capture common latent features in protein particles, facilitating efficient and generalized particle picking without human intervention. The optimization of box-step settings for regions of interest (RoI) balanced GPU memory constraints and accuracy requirements, enhancing the overall performance. Several potential enhancements to this method include exploring advanced image processing techniques for data preprocessing, RoI proposal generation, and false positive reduction to improve efficiency and accuracy. Additionally, investigating alternative neural network architectures or employing ensemble learning methods may further enhance particle picking performance. The continuous refinement of the training strategy, such as incorporating more diverse training datasets or fine-tuning network parameters, could enhance method robustness and adaptability to different protein particle types.

In 2019, five deep learning methods including Pixer, HydraPicker, CrYOLO, Wrap and Topaz were developed, leveraging cutting-edge algorithms in deep learning and computer vision to enhance the accuracy and efficiency of identifying particles.

PIXER [27] adopted an image segmentation-based approach, utilizing a deep neural network architecture. It first trained a classification network and then leveraged its parameters to expedite the segmentation network’s training. During testing, cryo-EM micrographs were inputted into the segmentation network to generate probability density maps. Preliminary particle coordinates were obtained using a grid-based local-maximum method, followed by a classification network to eliminate false positive particles. The classification network was equipped with multiple parallel Atrous convolution channels [28], enabling the processing of multiscale particles. The segmentation network’s performance was evaluated using pixel intersection-over-union criteria on a validation dataset. It used a grid-based local-maximum method to pinpoint particles from the probability density maps, enhancing both accuracy and efficiency in particle selection.

HydraPicker [29] developed a customized CNN architecture based on ResNet, with larger filters and group normalization layers. Its multi-headed design, comprising shared and dataset-specific heads, facilitated effective handling of unseen particles. Furthermore, the model was trained on different kinds of particles, combining specialized and generic models. Through a fully convolutional design, it divided the model into shared and specialized parts. Exploring limited data scenarios, it introduced a few-shot learning approach, enhancing adaptability to new datasets. While the method presented a couple of advancements in automated particle picking, it carried limitations. Firstly, its focus on specific particle structures during training may limit the model’s generalizability to a broader range of datasets and particle types, potentially constraining its applicability in diverse experimental settings. Moreover, the complexity of the proposed methodology, particularly with multiple specialized heads and a shared trunk, may have high computational cost, necessitating careful optimization and resource allocation for practical deployment.

CrYOLO [30] introduced a ‘You Only Look Once’ (YOLO) framework for automated particle picking, employing a 21-layer CNN for accurate particle detection. By reformulating the classification problem as a regression task, it predicted particle positions in a single pass of the full image, significantly speeding up the process compared to traditional methods. Despite training with a relatively small number of particles per cryo-EM micrograph dataset, CrYOLO achieved high recall and precision rates at a speed of up to five micrographs per second. The method may be further improved via transfer learning by pre-training it on diverse datasets first and then fine-tuning it to improve the generalization capability. Additionally, applying additional data augmentation techniques during training, such as rotation and scaling, might enhance robustness. Furthermore, incorporating domain-specific knowledge or constraints into the model, such as particle size distribution, may improve accuracy and efficiency in particle picking.

Warp [31] introduced a particle-picking method centered on the BoxNet machine-learning algorithm, a fully convolutional ResNet architecture comprising 72 layers. Trained on both real cryo-EM micrograph data from the EMPIAR repository and synthetic data simulated from Protein Data Bank (PDB) structures, BoxNet adeptly masked out high-contrast artifacts like ethane. The system incorporated a retraining interface, enabling users to fine-tune BoxNet’s performance by supplying positive examples of particle positions. Notably, Wrap’s BoxNet model had some advantage in dealing with heterogeneous datasets, exhibiting superior performance over some alternative models such as crYOLO or RELION [32] 3.0’s Laplacian of Gaussian approach.

Table 2. A list of advanced deep learning methods for protein particle picking, their techniques and train/test data.

Table 2. A list of advanced deep learning methods for protein particle picking, their techniques and train/test data.

SN	Approach	Techniques	Train/Test Data	Year
1	DeepPicker [23]	Deep Learning (using Cross-Molecule Training Strategy)	y-secretase, spliceosome, TRPV1, b-galactosidase, N- ethylmaleimide sensitive factor complex	2016
2	DeepEM [24]	Convolutional Neural Network (CNN)	800 manually selected particle images from the keyhole limpet Hemocyanin (KLH) dataset	2017
3	Xiao et al.’s method [26]	Region-based Convolutional Network (R-CNN)	Gammas, Spliceosome, Trpv1	2017
4	Warp [31]	Convolutional ResNet Architecture	EMPIAR-10097, EMPIAR-10045, EMPIAR-10078, EMPIAR-10061, EMPIAR-10164, EMPIAR-10153	2019
5	SPHIRE-crYOLO [30]	Deep Learning (Based on YOLO)	TcdA1 toxin subunit, Drosophila transient receptor channel NOMPC, human peroxiredoxin-3 (Prx3), simulated data of the canonical TRPC4 ion channel, and keyhole limpet hemocyanin (KLH)	2019
6	HydraPicker [29]	ResNet Architecture	Data from Warp [31]	2019
7	Pixer [27]	Deep Neural Network	beta-galactosidase, influenza hemagglutinin trimer, Plasmodium falciparum 80S ribosome, cyclic nucleotide-gated ion channel, and GroEl + TRPV1, KLH, bacteriophage MS2, and rabbit muscle aldolase	2019
8	Topaz [33]	Positive U learning CNN	EMPIAR-10025, EMPIAR-10096, EMPIAR-10028, EMPIAR-10261, EMPIAR-10234, and EMPIAR-10096	2019
9	DeepCryoPicker [34]	Unsupervised Learning	Apoferritin, KLH, 80S ribosome, β-galactosidase	2020
10	McSweeney et al.’s method [35]	Convolutional Neural Networks	EMPIAR-10204, 10218, 10028, 10335, 10184, and 10059.	2020
11	DRPnet [36]	Double Convolutional Neural Network (CNN) Cascade	TRPV1 dataset (EMPIAR-10005)	2021
12	TransPicker [37]	End-to-End Transformer-based Architecture	EMPIAR-10093, EMPIAR-10017, EMPIAR-10028, EMPIAR-10096, EMPIAR-10406, and EMPIAR-10590	2021
13	CASSPER [38]	Full Resolution Residual Network (FRRN)	TcdA1 (EMPIAR 10089), HCN1 (EMPIAR 10081), TRPV1 (EMPIAR 10005) and b-galactosidase (EMPIAR 10017)	2021
14	Urdnet [39]	U-Net based residual intensive neural network	71 human 80S ribosomal micrographs, 30 HCN1 micrographs, 24 TcdA1 micrographs, and 16 KLH micrographs	2022
15	CryoSegNet [40]	U-Net + SAM Model	CryoPPP [2]	2024
16	CryoTransformer [41]	Transformer + ResNet Architecture	CryoPPP [2]	2024

Topaz [33] is one of the widely used particle picking methods, which is based on a positive-unlabeled (PU) learning method, minimizing the need for extensive manual labeling by training the classifier with few labeled positive regions and unlabeled regions. It used a Generalized Expectation (GE) criteria to mitigate overfitting, improving prediction accuracy with minimal labeled data. Leveraging CNNs in conjunction with the PU learning, Topaz retrieved more particles at a low false-positive rate. However, manual examination of class averages of true particles or false positives picked by Topaz could be time-consuming and subjective.

In 2020, DeepCryoPicker [34] was developed to train a deep learning network using ‘good’ and ‘bad’ particle examples, similar to DeepEM [24]. However, the key difference was its use of a sliding window technique to classify sub-images as particles or background. To further improve performance, techniques like Non-Maximum Suppression (NMS) were utilized during testing to reduce false positives. Moreover, the method applied data augmentation and preprocessing steps to enhance the quality of low signal-to-noise ratio micrographs.

In the same year, McSweeney et al. proposed a self-supervised workflow for particle picking, employing an iterative strategy with 2D particle class averaging and a progressively improved CNN [35]. Automation was achieved through defining a threshold (%/Res: the proportion of particles in a 2D class average to the total particle count), minimizing user input. The iterative process optimized a fine-tuned CNN-based particle picker, enhancing particle picking quality. While offering automated particle picking and integration into single-particle analysis packages, the method required additional 2D and 3D particle classifications to filter out damaged particles. Efficiency limitations were noted for small particles, regardless of the picking program used. Though aiming to eliminate trial-and-error parameters, the dependence on data-dependent parameters such as particle size remained a limitation.

In 2021, TransPicker [37] introduced a framework leveraging the crDETR module based on a transformer-based detection method for 2D particle picking. Using an improved deformable Transformer [42], TransPicker adeptly handled particle relocation and global context without using traditional components like anchors or sliding windows [43], reaching rapid convergence with a combined loss function. The framework addressed limitations in object query numbers through a divide-and-conquer algorithm and incorporated optimizations such as denoising, enhancing (sharpening edges and contours), and adding masks to carbon region to improve accuracy. Moreover, it implemented particle filtering techniques and applied masks on carbon regions and ice contaminants to further reduce false positive ratios.

In the same year, CASSPER [38] utilized semantic segmentation, a deep learning technique enabling pixel-level classification in cryo-EM micrographs, to streamline particle identification without manual picking. It used the Contrast Limited Adaptive Histogram Equalization (CLAHE) to enhance particle detection in challenging conditions, delivering robust performance. While streamlining particle picking, iterative 2D classification and 3D refinement steps were still necessary for detailed structural analysis, incorporating feedback mechanisms for self-correction and adaptive learning may further enhance CASSPER’s robustness and accuracy over time, particularly with complex datasets.

DRPNet [36], a deep learning-based particle picking network, was tailored to tackle issues like low signal-to-noise ratios and diverse particle characteristics. It employed a fully convolutional regression network (FCRN) to generate continuous distance maps representing particle centers and a classification CNN to refine detections, minimizing false positives. The model treated particle picking as a blob detection problem, considering particles as convex blobs with distinct textures and locating particle centers based on size estimates. Micrograph preprocessing enhanced performance by improving contrast and correcting artifacts. Based on its evaluation, DRPNet improved particle picking in terms of recall, precision, and F-measure. However, it encountered difficulty in the scenarios where particles varied substantially in dimensions and shapes.

UrdNet [39] developed a U-Net based residual dense neural network that integrated point-level and pixel-level labels to streamline manual particle labeling and enhance the precision of particle picking. This approach combined multiple annotations (point-level and pixel-level annotations) and an improved U-Net architecture to select particles. The network model, comprising 49 convolution layers, four max-pooling layers, four up-sampling layers, and four feature concatenations, facilitated the extraction of global features and deep feature fusion, thereby enabling efficient particle picking. However, it had a limitation in the output definition of the residual block (ambiguity in feature representation), indicating the need for further refinement in the network architecture.

In 2024, CryoTransformer [41] introduced a self-attention-based transformer model, typically utilized in natural language processing, to cryo-EM particle picking. By combining the transformer model and residual connections within the CNN component, CryoTransformer aimed to capture long-range dependencies/correlation between particles within cryo-EM images, thereby enhancing the accuracy of particle selection. The evaluation of the method on the CryoPPP dataset [2] involved ablation studies to assess various technical components, including the impact of denoising (pre-processing) micrographs. A set of comprehensive evaluation metrics encompassing machine learning metrics, 2D particle resolution, 3D density map resolution, and particle orientation diversity were applied to assess its performance. The study highlighted the importance of denoising micrographs in particle picking tasks for improving particle picking accuracy. Potential enhancements for CryoTransformer includes leveraging transfer learning techniques, embracing ensemble learning approaches, developing interpretability and visualization methods. Developing a user interface to provide more intuitive controls and real-time feedback during particle picking could streamline the workflow and make the tool more user-friendly for researchers with varying levels of expertise.

Also in 2024, CryoSegNet [40] was developed to integrate the attention-gated U-Net architecture with a general foundational image segmentation model - the Segment Anything Model (SAM) [44] - to achieve heightened precision and recall in particle selection. Notably, the method addressed challenges in picking particles for small proteins through fine-tuning with predicted labels from a pre-trained model [45], improving the accuracy of this difficult task. Moving forward, potential enhancements for CryoSegNet includes fine-tuning the model on the cryo-EM data of diverse proteins, augmenting the training dataset with predicted labels via distillation, and optimizing post-processing steps of selecting particles.

III. A Comparative Study of the AI-Based Cryo-EM Particle Picking Methods

IV. Remaining Challenges in Particle Picking

V. Potential Future Development

Conclusions

Key Points

References

1. Frank, J. Single-particle imaging of macromolecules by cryo-electron microscopy. Annu Rev Biophys Biomol Struct 2002, 31, 303–319. [Google Scholar] [CrossRef]
2. Dhakal A, Gyawali R, Wang L, et al. A large expert-curated cryo-EM image dataset for machine learning protein particle picking. Sci Data 2023, 10.
3. Frank J, Radermacher M, Penczek P, et al. SPIDER and WEB: Processing and visualization of images in 3D electron microscopy and related fields. J Struct Biol 1996, 116, 190–199. [Google Scholar] [CrossRef] [PubMed]
4. Dhakal A, Gyawali R, Wang L, et al. CryoPPP: A Large Expert-Labelled Cryo-EM Image Dataset for Machine Learning Protein Particle Picking. bioRxiv 2023. 2023.02.21.529443.
5. Dhakal A, McKay C, Tanner JJ, et al. Artificial intelligence in the prediction of protein-ligand interactions: recent advances and future directions. Brief Bioinform 2022, 23.
6. Corso G, Stärk H, Jing B, et al. Diffdock: Diffusion steps, twists, and turns for molecular docking. arXiv preprint 2022, arXiv:2210.01776.
7. Stärk H, Ganea O, Pattanaik L, et al. Equibind: Geometric deep learning for drug binding structure prediction. International conference on machine learning 2022, 20503–20521.
8. Dhakal A, Gyawali R, Cheng J. Predicting Protein-Ligand Binding Structure Using E(n) Equivariant Graph Neural Networks. bioRxiv 2023. 2023.08.06.552202.
9. Scheres SHW. RELION: Implementation of a Bayesian approach to cryo-EM structure determination. J Struct Biol 2012, 180, 519–530. [Google Scholar] [CrossRef]
10. Tang G, Peng L, Baldwin PR, et al. EMAN2: An extensible image processing suite for electron microscopy. J Struct Biol 2007, 157, 38–46. [Google Scholar] [CrossRef]
11. Gyawali R, Dhakal A, Wang L, et al. CryoVirusDB: A Labeled Cryo-EM Image Dataset for AI-Driven Virus Particle Picking. bioRxiv 2023.
12. Iudin A, Korir PK, Somasundharam S, et al. EMPIAR: the Electron Microscopy Public Image Archive. Nucleic Acids Res 2023, 51, D1503–D1511. [Google Scholar] [CrossRef]
13. Bradski, G. The opencv library. Dr. Dobb’s Journal: Software Tools for the Professional Programmer 2000, 25, 120–123. [Google Scholar]
14. Mallick SP, Zhu Y, Kriegman D. Detecting particles in cryo-EM micrographs using learned features. J Struct Biol 2004, 145, 52–62. [Google Scholar] [CrossRef] [PubMed]
15. Hoang T V, Cavin X, Schultz P, et al. gEMpicker: a highly parallel GPU-accelerated particle picking tool for cryo-electron microscopy. BMC Struct Biol 2013, 13, 25. [Google Scholar]
16. Langlois R, Pallesen J, Ash JT, et al. Automated particle picking for low-contrast macromolecules in cryo-electron microscopy. J Struct Biol 2014, 186, 1–7. [Google Scholar] [CrossRef]
17. Heimowitz A, Andén J, Singer A. APPLE picker: Automatic particle picking, a low-effort cryo-EM framework. J Struct Biol 2018, 204, 215–227. [Google Scholar] [CrossRef]
18. Zhu Y, Carragher B, Glaeser RM, et al. Automatic particle selection: Results of a comparative study. J Struct Biol 2004, 145, 3–14. [Google Scholar] [CrossRef]
19. Azzawi A Al, Ouadou A, Tanner JJ, et al. A super-clustering approach for fully automated single particle picking in cryo-em. Genes (Basel) 2019, 10.
20. Al-Azzawi A, Ouadou A, Tanner JJ, et al. Autocryopicker: An unsupervised learning approach for fully automated single particle picking in cryo-em images. BMC Bioinformatics 2019, 20.
21. Rona G, Wynder EL, Helman P, et al. Plasma Hormone Profiles in Populations at Different Risk for Breast Cancer. Elife 2018, 36, 1883–1885. [Google Scholar]
22. Li S, Li H, Zhang C, et al. A Segmentation-aware Synergy Network for Single Particle Recognition in Cryo-EM. Proceedings - 2022 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2022, 1066–1071.
23. Wang F, Gong H, Liu G, et al. DeepPicker: A deep learning approach for fully automated particle picking in cryo-EM. J Struct Biol 2016, 195, 325–336. [Google Scholar] [CrossRef] [PubMed]
24. Zhu Y, Ouyang Q, Mao Y. A deep convolutional neural network approach to single-particle recognition in cryo-electron microscopy. BMC Bioinformatics 2017, 18.
25. Girshick, R. Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision 2015, Inter, 1440–1448. [Google Scholar]
26. Xiao Y, Yang G. A fast method for particle picking in cryo-electron micrographs based on fast R-CNN. AIP Conf Proc 2017, 1836.
27. Zhang J, Wang Z, Chen Y, et al. PIXER: An automated particle-selection method based on segmentation using a deep neural network. BMC Bioinformatics 2019, 20.
28. Chen LC, Papandreou G, Kokkinos I, et al. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Trans Pattern Anal Mach Intell 2018, 40, 834–848. [Google Scholar] [CrossRef]
29. Masoumzadeh A, Brubaker M. HydraPicker: Fully automated particle picking in cryo-em by utilizing dataset bias in single shot detection. 30th British Machine Vision Conference 2019, BMVC 2019 2020.
30. Wagner T, Merino F, Stabrin M, et al. SPHIRE-crYOLO is a fast and accurate fully automated particle picker for cryo-EM. Commun Biol 2019, 2.
31. Tegunov D, Cramer P. Real-time cryo-electron microscopy data preprocessing with Warp. Nat Methods 2019, 16, 1146–1152. [Google Scholar] [CrossRef]
32. Scheres SHW. RELION: Implementation of a Bayesian approach to cryo-EM structure determination. J Struct Biol 2012, 180, 519–530. [Google Scholar] [CrossRef]
33. Bepler T, Morin A, Rapp M, et al. Positive-unlabeled convolutional neural networks for particle picking in cryo-electron micrographs. Nat Methods 2019, 16, 1153–1160. [Google Scholar] [CrossRef]
34. Al-Azzawi A, Ouadou A, Max H, et al. DeepCryoPicker: fully automated deep neural network for single protein particle picking in cryo-EM. BMC Bioinformatics 2020, 21.
35. McSweeney DM, McSweeney SM, Liu Q. A self-supervised workflow for particle picking in cryo-EM. IUCrJ 2020, 7, 719–727. [Google Scholar] [CrossRef] [PubMed]
36. Nguyen NP, Ersoy I, Gotberg J, et al. DRPnet: automated particle picking in cryo-electron micrographs using deep regression. BMC Bioinformatics 2021, 22.
37. Zhang C, Li H, Wan X, et al. TransPicker: A Transformer-based Framework for Particle Picking in cryoEM Micrographs. Proceedings - 2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021, 1179–1184.
38. George B, Assaiya A, Roy RJ, et al. CASSPER is a semantic segmentation-based particle picking algorithm for single-particle cryo-electron microscopy. Commun Biol 2021, 4.
39. Ouyang J, Zhang Y, Fang K, et al. Urdnet: A Cryo-EM Particle Automatic Picking Method. Computers, Materials and Continua 2022, 72, 1593–1610. [Google Scholar] [CrossRef]
40. Gyawali R, Dhakal A, Wang L, et al. CryoSegNet: accurate cryo-EM protein particle picking by integrating the foundational AI image segmentation model and attention-gated U-Net. Brief Bioinform 2024, 25, bbae282. [Google Scholar]
41. Dhakal A, Gyawali R, Wang L, et al. CryoTransformer: a transformer model for picking protein particles from cryo-EM micrographs. Bioinformatics 2024, 40. [Google Scholar]
42. Zhu X, Su W, Lu L, et al. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint 2020, arXiv:2010.04159.
43. Pratyush P, Bahmani S, Pokharel S, et al. LMCrot: an enhanced protein crotonylation site predictor by leveraging an interpretable window-level embedding from a transformer-based protein language model. Bioinformatics 2024, 40, btae290. [Google Scholar]
44. Kirillov A, Mintun E, Ravi N, et al. Segment Anything. Proceedings of the IEEE International Conference on Computer Vision 2023, 3992–4003. [Google Scholar]
45. Gyawali R, Dhakal A, Wang L, et al. Accurate cryo-EM protein particle picking by integrating the foundational AI image segmentation model and specialized U-Net. bioRxiv 2024. 2023.10.02.560572.
46. Punjani A, Rubinstein JL, Fleet DJ, et al. CryoSPARC: Algorithms for rapid unsupervised cryo-EM structure determination. Nat Methods 2017, 14, 290–296. [Google Scholar] [CrossRef]
47. Pettersen EF, Goddard TD, Huang CC, et al. UCSF ChimeraX: Structure visualization for researchers, educators, and developers. Protein Science 2021, 30, 70–82. [Google Scholar] [CrossRef]
48. He F, Yang Z, Gao M, et al. Adapting Segment Anything Model (SAM) through Prompt-based Learning for Enhanced Protein Identification in Cryo-EM Micrographs. ArXiv 2023.
49. Xu C, Zhan X, Xu M. CryoMAE: Few-Shot Cryo-EM Particle Picking with Masked Autoencoders. ArXiv 2024.
50. Zamanos A, Koromilas P, Bouritsas G, et al. Towards generalizable particle picking in Cryo-EM images by leveraging Masked AutoEncoders. ICML 2024 Workshop on Efficient and Accessible Foundation Models for Biological Discovery 2024.
51. Liu Z, Lin Y, Cao Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF international conference on computer vision 2021, 10012–10022.
52. Carion N, Massa F, Synnaeve G, et al. End-to-end object detection with transformers. European conference on computer vision 2020, 213–229.