Version 1
: Received: 18 April 2023 / Approved: 20 April 2023 / Online: 20 April 2023 (10:07:48 CEST)
How to cite:
Pervan-Akman, N.; Berkol, A.; Erdem, H. Automated Word-level Lip Reading using Convolutional Neural Networks with New Turkish Dataset. Preprints2023, 2023040645. https://doi.org/10.20944/preprints202304.0645.v1
Pervan-Akman, N.; Berkol, A.; Erdem, H. Automated Word-level Lip Reading using Convolutional Neural Networks with New Turkish Dataset. Preprints 2023, 2023040645. https://doi.org/10.20944/preprints202304.0645.v1
Pervan-Akman, N.; Berkol, A.; Erdem, H. Automated Word-level Lip Reading using Convolutional Neural Networks with New Turkish Dataset. Preprints2023, 2023040645. https://doi.org/10.20944/preprints202304.0645.v1
APA Style
Pervan-Akman, N., Berkol, A., & Erdem, H. (2023). Automated Word-level Lip Reading using Convolutional Neural Networks with New Turkish Dataset. Preprints. https://doi.org/10.20944/preprints202304.0645.v1
Chicago/Turabian Style
Pervan-Akman, N., Ali Berkol and Hamit Erdem. 2023 "Automated Word-level Lip Reading using Convolutional Neural Networks with New Turkish Dataset" Preprints. https://doi.org/10.20944/preprints202304.0645.v1
Abstract
Automated lip reading is a research problem that has developed considerably in recent years. Lip reading is evaluated both visually and audibly in some cases. The lip reading model is a field of use for detecting specific words using images from security cameras, but it is not possible to use audio-visual databases in this situation. It is not possible to obtain the sound input of the pronounced word in all cases. We collected a new Turkish dataset with only the image in this study. The new dataset is produced using Youtube videos, which is an uncontrolled environment. For this reason, images have difficult parameters in terms of environmental factors such as light, angle, color, and personal characteristics of the face. Despite the different features on the human face such as mustache, beard, and make-up, the visual speech recognition problem was developed on 10 classes including single words and two-word phrases using Convolutional Neural Networks (CNN) without any intervention on the data. The proposed study using only-visual data obtained a model which is automated visual speech recognition with a deep learning approach. In addition, since this study uses only-visual data, the computational cost and resource usage is less than in multi-modal studies. It is also the first known study to address the lip reading problem with a deep learning algorithm using a new dataset belonging to the Ural-Altaic languages.
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
