Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

A Deep Learning Model of Perception in Color-Letter Synesthesia

Version 1 : Received: 17 December 2017 / Approved: 19 December 2017 / Online: 19 December 2017 (06:45:50 CET)

How to cite: Bock, J.R. A Deep Learning Model of Perception in Color-Letter Synesthesia. Preprints 2017, 2017120128. Bock, J.R. A Deep Learning Model of Perception in Color-Letter Synesthesia. Preprints 2017, 2017120128.


Synesthesia is a psychological phenomenon where sensory signals become mixed. Input to one sensory modality produces an experience in a second, unstimulated modality. In “grapheme-color synesthesia”, viewed letters and numbers evoke mental imagery of colors. The study of this condition has implications for increasing our understanding of brain architecture and function, language, memory and semantics, and the nature of consciousness. In this work, we propose a novel application deep learning to model perception in grapheme-color synesthesia. Achromatic letter images, taken from database of handwritten characters, are used to induce synesthesia. Results show the model learns to accurately create a colored version of the inducing stimulus, according to a statistical distribution from experiments on a sample population of grapheme-color synesthetes. The spontaneous, creative mental imagery characteristic of the synesthetic perceptual experience is reproduced by the model. A model of synesthesia that generates testable predictions on brain activity and behavior is needed to complement large scale data collection efforts in neuroscience, especially when articulating simple descriptions of cause (stimulus) and effect (behavior). The research and modeling approach reported here begins to address this need.


synesthesia; deep learning network; color perception; generative adversarial network; cognitive modeling; character recognition; GPU computing

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)
Views 0
Downloads 0
Comments 0
Metrics 0

Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.