Semantic Training for Cognitive Stimulation in Healthy Aging: Motives, Dimensions, and Usability of the CerUp Online Platform

1. Introduction

Semantics is the study of meaning, and linguistic semantics concerns meaning as part of the language system (as opposed to other forms of communication, e.g., facial expressions). Linguistic meaning is present in single words (word-level or lexical semantics), but it also depends on how words are sequentially arranged into sentences according to the rules of a given grammar (sentence-level or syntactic semantics) [1,2,3].

The concept of the mental lexicon is key to understanding how humans process linguistic meaning. The mental lexicon is an idiom-specific repository containing information about the meaning of single words as well as their derived and compounded forms, e.g., words modified by affixes or formed into compound expressions. It also contains information about the organization of words into hierarchies of semantic categories (e.g., dogs as animals, animals as living creatures) and other semantic relations (e.g., part-whole relationships) [4,5,6,7].

Along with this strict lexical-semantic (single word) information, the mental lexicon is also concerned with the role of each word within the grammar of a language (e.g., whether it is a verb or noun). In this sense, the mental lexicon is engaged in both lexical and syntactic semantics [8,9,10]. The mental lexicon supports both the comprehension and production of language. During comprehension, listeners (or readers) match incoming input against their mental representations to decide whether a word is real (lexical decision), determine its meaning, and define its semantic relations with other words (group into categories, define part-whole relationships or others) [11,12]. During language production, the mental lexicon provides the sound form for the semantic content to be conveyed. This process is known as lexical retrieval [13,14,15] and occurs, for instance, when naming a picture.

Linguistic semantic deficits are part of various neurological conditions, primarily degenerative, such as semantic dementia, posterior cortical atrophy [16], frontotemporal dementia [17], aphasia or Alzheimer’s disease [18]. Production problems -lexical retrieval- are pervasive in degenerative diseases like Alzheimer’s [18,19,20]. In semantic dementia, patients experience impairments in comprehension (e.g., lexical decision) alongside more easily observed production difficulties, such as lexical retrieval [21,22]. Similar issues occur in aphasia, where various subtypes affect either comprehension (e.g., global aphasia) or production (e.g., non-fluent aphasia) [23,24].

Although research and neuropsychological practice tend to focus on the most basic semantic level—the word level—sentence-level semantics is also compromised in these clinical conditions, as the former is a prerequisite for the latter. Regarding linguistic semantic processing in healthy aging, the literature presents mixed findings. Some studies show no significant differences between younger and older adults (e.g., [25,26,27]), while others highlight word-level difficulties in older participants, specifically in production accuracy and speed [28,29,30,31,32]. Difficulties in comprehension seem to be absent, and age-related decline may explain occasional low performances under certain conditions.

Given the importance of semantic processing for basic social functioning and the vulnerability of the semantic system to degeneration and/or brain lesions, the need for methods and tools to rehabilitate or preserve the semantic system is well justified. The so-called “lexical-semantic therapies” (LS) [33,34] emerged as semantic exercises aimed at remediating linguistic semantic deficits and rehabilitating patients. To maximize clarity, we will refer to these as “semantic therapies”, as that the prefix “lexical” may suggest a restriction to word-level semantics.

Semantic methods and exercises enhance not only semantic but also general language skills, such as phonological abilities [18,35,36,37]. Critically, semantic exercises may positively impact general cognition. For instance, studies with early Alzheimer’s disease patients showed that semantic therapy improved episodic memory [37], working memory and processing speed [38]. Additionally, the extensive interrelations between linguistic and non-linguistic skills (namely executive functions) suggest that training in one domain may benefit the other [35]. Specifically, semantic skills are strongly related to inhibition [39] as well as working memory [40]. Moreover, sentence-level semantics has a privileged association with memory updating, both at the levels of sentence comprehension [41] and production [42]. Considering that non-pathological aging affects general cognitive functioning, the idea of using semantic tasks for cognitive stimulation (training and prevention, as opposed to rehabilitation) is logical. Even though linguistic semantic deficits may be limited in this context, semantic exercises can have a positive impact on other cognitive domains.

Despite the potential of semantic training as a cognitive stimulation tool, systematic reviews of available programs show little investment in this area [43,44]. One exemption is the instrumental use of categorization to enhance memorization, which, however, is viewed merely as a compensatory strategy. To address this gap and promote LS stimulation in healthy aging, we adapted the BOX linguistic semantic program (original by [34], used in [37,38] and made it freely available on an online platform for guided or unguided use (Cerup platform, https://estimulacerup.wixsite.com/website). Online activities may pose challenges for both users and practitioners, due to age-related limitations of the former (hearing and/or vision impairment, poor confidence in using computer-based technology) and additional demands imposed on the latter (need for initial training and technical proficiency; see [45]. Nevertheless, research shows that computer-based technology is well accepted by patients undergoing rehabilitation [46,47], suggesting that the same may apply to healthy older adults. Critically, the BOX program, which we adapted, was tested in face-to-face versus computer-based, remote modalities, and the outcomes were similar [38].

In the current paper, our goal was to prepare the optimization of CerUp users’ future experience. This includes potential changes in task structure, level of difficulty and instructions. Additionally, it presents a map of hypothesized task-specific features that may guide tailor-made uses of the program. To this end, we provide preliminary results concerning the usability of CerUp, including participants’ adherence, engagement, performance accuracy, perceived difficulty and rating of the adequacy of the instructions provided that might guide future changes. By crossing task-specific usability data with the hypothesized map of task-specific features, we also tested the validity of the latter.

2. Materials and Methods

2.1. Participants

Twenty-seven individuals provided valid responses. Fourteen were individual users who did not report any health problems that could impact cognitive functioning. The other thirteen were multiple sclerosis patients attending an association, who provided their answers during group sessions led by an intern psychologist. The relevant sociodemographic characteristics are presented in Table 1.

2.2. Materials

Based on descriptions and examples from the literature, we made a free adaptation of the 8 tasks that composed the BOX lexical-semantic rehabilitation program [34,48] to European Portuguese. As in the original program, we created three levels of difficulty (1 -easy. 2—medium, 3—hard), each comprising 10 to 20 questions with multiple-choice response options (see Supplementary material for questions in Portuguese). Among the 8 tasks, four focused on single word meaning (W1-W4), while the other 4 required semantic integration at the sentence or text-level (S1-S4).

In Table 2, we characterize each task according to four complexity indices: unit size (1 = short, 2 = long), the minimal number of processing steps required (cardinal), additional cognitive processes (beyond linguistic semantics: 0 = no, 1 = yes), and response uncertainty associated with these (i.e., whether participants remain uncertain about their success in these additional processing levels, 0 = no, 1 = yes). By summing the assigned scores, we obtained a global complexity score, which we used to evaluate the results of the usability study.

W1-FIND THE INTRUDER: Originally titled “semantic categories”, this task presents lists of five semantically related words mixed with one word from a different category (the intruder). Participants are asked to identify the intruder word among the six options. For instance, in the list consisting of “skirt, socks, shirt, curtain, coat, trousers”, most words are pieces of clothing, so “curtain” would be the intruder. The pre-assigned difficulty level decreased with word frequency, concreteness, the degree of common knowledge (vs. jargon or specialized knowledge), and the semantic unrelatedness of the intruder. Each level contained 18 questions.

The task required participants to carry out two interdependent operations: (1) hypothesize the identity of the dominant category/object type (most items were clothes) and (2) find the option outside that category. If (2) could not be performed, (1) should be revised. Along with knowledge about the meaning of words, participants should activate the hierarchical representations inherent in categorization processes (e.g., a skirt is a piece of clothing, and so is a shirt). Participants’ guess for the dominant category—an intermediate step—remained uncertain since the category was not in the list of response options (only the intruder).

W2- WORD FAMILIES: Also related to semantic categories, this task was originally named “semantic classification” and involved presenting a list of words pertaining to a given, undefined category. Participants were given two category names as response options, one correct and one incorrect. For example, participants would see “tulip, daisy, rose, carnation, sunflower, orchid, lilium”, with “flowers” and “vegetables” as response options. The correct answer would be “flowers”. As in W1, the difficulty level decreased with word frequency, concreteness, and the degree of common knowledge (vs. jargon or specialized knowledge). In this case, the semantic unrelatedness of the incorrect option made the question easier. Levels 1 and 2 contained 14 questions each, and level 3 had 15.

To complete this task, participants could reach the solution by first identifying the category as they saw the items (e.g., looks like flowers) and then checking if it appeared as an option (yes). Alternatively, they could start with the two response options and then review the items (one would be enough) to eliminate one option and keep the other (e.g., tulip is a flower, not a vegetable). Although categorization processes were also involved here, only one basic step was involved (recognize the correct category among two). This made W2 potentially easier than W1.

W3-THE DOOR’S HANDLE: In this task, initially designated as “part-whole relationships”, participants are required to find the missing word in an “x is to X as y is to Y” construction, where x and y are parts and X, and Y are the respective wholes or vice-versa. The missing word could be any of these four, and participants had four options from which to choose the correct answer. For example, given the sentence “Screen is to laptop as wall is to …”, with the options “door, ceiling, concrete, room”, the correct answer would be “room”. Each level comprised 12 instances of the “is to/ as / is to’‘ construction. In the easy level, only the final words were missing. In the medium and difficult levels, other words could also be missing.

To successfully accomplish the task, participants had to (1) abstract the part vs. the whole relationship in the complete “is to” construction, (2) identify which role (part or whole) was missing in the incomplete construction, and (3) find the word that instantiated the missing role among the given options. Responses to (1)—an intermediate step—remained uncertain, as they were not included in the response options.

W4-GLUEING WORDS: In this task, originally named “Compound words”, participants were given a root word that could be concatenated with others, such as “well-”. Then they were shown four examples of compounds, only one of which made sense. For example, they might see “well-educated, well-cat, well-umbrella, well-eat”, with the first option being correct. Each level contained 16 root words and their respective compound options. The difficulty increased with word frequency and potential lack of familiarity (jargon, specialized knowledge).

To successfully complete the task, participants could make lexical decisions (determining if the option was real word) for each option.

S1-DISCOVER THE SENTENCE: In this sentence-level semantic task (previously designated as “syntagmatic and paradigmatic relationships”), participants were presented with a sentence containing a missing part. They were asked to choose the most appropriate word or word group from four options to complete the sentence. For instance, given “Paul ate…” and the options “a chair, a story, an ice-cream, the door”, participants should select the third option. Incorrect options could be unsuitable due to semantic incongruence (as in the previous example) or grammatical violations (e.g., “Paul ate …jumped”, where two verbs follow each other). In the latter case, semantic incongruence also naturally occurs. The task difficulty was varied by manipulating sentence complexity (simple, coordinate, subordinate sentences). The easy level included 48 sentences, while the other two levels each had 30. In each level, half of the sentences contained grammatical violations.

To accomplish the task, participants could follow a process similar to W2. They could either identify features of the missing part as they read the incomplete sentence (e.g., “Paul ate…” should be food) or evaluate the semantic congruence of the sentence with each of the response options.

S2-CHIT-CHAT: This task addressed sentence-level semantics within a conversation. Participants read a transcription of something a speaker could have said (e.g., “John got himself into trouble again”) and chose the most adequate reaction a hypothetical listener could make (e.g., “How happy he is!, He is so conflictual!, Very sensitive!, How vain!”, with the second option being the correct one). Since comments contained adjectives and exclamations, this was the original name of the task. The difficulty increased with increasing syntactic complexity. Additionally, the third level included options with analogies and proverbs, which increased abstraction. Each level contained 24 questions.

The task could be accomplished by first (1) extracting meaning from the speakers’ utterances and using theory of mind [49] skills to better understand their position and judgement concerning the situation. In this case, it was likely that the speaker was condemning John, even though this was not explicitly stated. Afterwards, participants could (2) either imagine what a good response from the listener would be and select it from the options (something like “How conflictual!”, agreeing with the speaker) or test response options one at a time. Inferences concerning the speaker’s intention (theory of mind) remained uncertain.

S3-MAKES ANY SENSE? Participants were asked to judge whether a sentence was semantically anomalous (originally named as “Anomalous sentences”). In the first level (18 sentences) participants only needed to answer “correct” or “incorrect”. In the second level (20 sentences), participants had to identify the word that made the sentence anomalous when applicable. In level 3, they were required to repair the incorrect sentence by choosing the word that would restore its meaning. In the easiest level, participants were asked to judge the semantic coherence of sentences as correct/incorrect (sentence-level semantic coherence), making this task potentially easier than S1. The medium and difficult levels imposed additional cognitive load.

S4-YOU TALK FOOLISHLY! This activity was originally named “Semantic context” and focused on identifying semantic anomalies within larger text segments (2-5 sentences). For instance, the sequence “Tomorrow I will cut my hair. Nevertheless, the market close” is anomalous because the first sentence is unrelated to the second. In level 1 (10 items), participants were asked to choose the correct text from two options. Levels 2 and 3 (16 items each) presented increasingly longer texts, one at a time, requiring participants to judge whether each text was correct or incorrect.

In this task, participants only needed to make simple right/wrong judgements, similar to the easy level of S3. Nevertheless, the increased amount of linguistic input made this task potentially more demanding.

The 8 activities, each with 3 levels, were structured as 24 different experiments using OpenSesame, version 3.3.9 Lentiform Loewenfeld. OpenSesame [50] is a free software that allows for online data collection once experiments are hosted on a server like Jatos [51]. At the time, online data collection was only possible with a computer (vs. tablet or smartphone).

To facilitate users’ access to activities and supporting information, we created a website (https://estimulacerup.wixsite.com/play/atividades) where links to the 8 x 3 experiments were organized on a single page. Along with a short description of each activity, the website also hosted an informed consent form, an overview of the project, as well as contact information. We created new names for the activities (as described in the materials) and added cover images (Figure 1) to make them more appealing. Response modes were mostly based on single clicks. Navigation instructions were provided at every step, and the instructions were written in simple language.

2.3. Procedure

We contacted several nursing homes and day centers nationwide (n = 112) by email, inviting them to participate in the usability study by accessing the website. We highlighted the confidentiality and anonymity of the data, ensuring that no unnecessary personal information, such as contact details or names, would be collected. Additionally, we made no specific requests regarding the modality of administration (group vs. individual). The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the Faculty of Psychology and Educational Sciences of the University of Porto, (authorization number: Ref.ª 2021/09-05b). If they agreed to participate, they were required to read thoroughly and provide written confirmation before starting to use the website.

Data collection was conducted online and lasted for 2 and a half months (19th April-4th June 2021). Upon accessing the website, participants were asked to read the study information and provide their consent to participate in the activities by clicking a checkbox. They were then directed to the main menu, which contained the links to the 8 tasks x 3 levels hosted on Jatos. Once opening a given link, participants were required to provide sociodemographic information such as age, education level, gender and professional status (active vs. non-active). This process was repeated each time participants entered a new level or task, allowing them to choose freely which activities/levels to engage with. Following this, they received task-specific instructions and completed two practice trials, with feedback and explanations provided in case of incorrect answers. Subsequently, participants answered the multiple-choice questions that comprised the block, receiving positive or negative feedback after each question. At the end of each task (block of questions), participants were informed on their overall performance, namely hit rates and average response time. Based on their performance levels, they were advised whether to increase the difficulty level or not.

Before leaving the block, participants were asked to rate the level of difficulty they experienced using a 5-point scale (1 = very easy; 5 very difficult) and the adequacy of the instructions using a 3-point scale (1 = inadequate/hard to understand; 3 = adequate/easy to understand). They were also asked if they had received any guidance when choosing the correct response and if they had experienced any problems related to using the computer (yes/no options for both questions).

2.4. Analysis

We first extracted measures of adherence (number of institutions showing interest in the program/invited institutions) and engagement (completed/opened tasks). Then, we analyzed participants’ accuracy and perceived difficulty by task type (word vs. sentence level), by task (8 tasks), and by difficulty level (easy vs. medium vs. difficult). The clarity of instructions was analyzed for each task. Finally, we analyzed the correlations between accuracy, perceived difficulty, and clarity of instructions. Due to the small sample size, parametric tests (ANOVA, Pearson correlations) were cross-checked with non-parametric alternatives. In addition, we calculated Bayes factors.

3. Results

3.1. Adherence and Engagement

A total of 112 institutions (nursing homes and day centers) were contacted. Of these, only 4 (3.6%) expressed interest in participating in the study. After 2.5 months, we obtained 68 groups of valid responses (completed blocks). Of these, 66% were answered collectively, while the remaining 34% were answered individually. In addition, we gathered a large amount of invalid responses, including blocks that were opened but not responded to, and non-completed blocks (Table 3).

3.2. Performance Accuracy, Perceived Difficulty and Adequacy of Instructions

Only one individual reported benefiting from external guidance provided by a neuropsychologist. All members of the collective received guidance from the group facilitator, and none reported any issues related to using the computer.

As shown in Figure 2, accuracy was above chance levels for both word-, t(37) = 16.31, p < .001, d = 2.64, and sentence-level tasks, t(29) = 14.96, p < .001, d = 2.73. A similar result was observed for the perceived adequacy of instructions, with values significantly above 2, the mid-point (word-level: t(28) = 6.77, p < .001, d = 1.26, sentence-level: t(26) = 7.32, p < .001, d = 1.41), indicating adequacy/ease of understanding. Regarding perceived difficulty, word-level tasks showed values that did not significantly differ from the mid-point (3, neither hard nor easy, p > .14), while sentence-level ones were rated with values significantly below it (easy), t(26) = -2.29, p < .030, d = -0.44. Word- vs- sentence-level tasks did not differ significantly in accuracy (p > .26), perceived difficulty (p > .38) or adequacy of instructions (p > .57).

Task type (8 levels) had a significant effect on accuracy, F(7,60) = 3.25, p = .005. ηρ² = .28. Post-hoc comparisons did not show significant local differences p > .10, but Bayesian analysis highlighted moderate evidence in favor of the alternative hypothesis for the comparisons between W1 and S1 (BF₁₀ = 6.27), W2 (BF₁₀ = 8.07), and W4 (BF₁₀ = 13.23), W1 showing lower performance levels. The same went for comparisons between W3 and S1, (BF₁₀ = 3.26), W2 (BF₁₀ = 3.18), and W4 (BF₁₀ = 4.37), with lower values for W3.

Perceived difficulty did not vary across tasks p > .42. The adequacy of instructions was rated between 2 and 3 for all tasks. Statistical tests could not be conducted due to insufficient values per cell. Accuracy did not change significantly according to the level of difficulty (Figure 3) we established p > .29. In contrast, perceived difficulty depended on the pre-established level F(2,42) = 15.17, p < .001. ηρ² = .42, though pairwise differences were limited to the contrast between difficult and easy (p < .001) and difficult and medium (p = .002). Accuracy showed a strong correlation with both perceived difficulty (ρ = −.541, p < .001) and adequacy of instructions (ρ = −.532, p < .001). However, the latter two showed no relevant association (p > .61).

4. Discussion

Firstly, it is important to highlight the minimal interest CerUp received from the contacted institutions and the low engagement observed after analyzing all the data. One possible explanation is that the present cognitive stimulation program is only functional on a computer, which could be an obstacle to its use in both institutional and domestic context/settings since access to this technology may often be limited or its use may be challenging (difficulties in its use). Therefore, adapting our program to a format that is functional on a tablet or smartphone could be beneficial. Another possible explanation relates to the large number of questions participants had to answer before and after each activity x level, which might have caused fatigue and discouraged them from finishing it. This obstacle is easily addressed, as it is related to the study’s requirements. Finally, the number of questions per level may have been too large, contributing to participants’ fatigue and subsequent abandonment of the activity. Reducing the number of questions might be useful.

Performance accuracy exceeded chance levels in all tasks. While differences across tasks were not striking, the small variations observed were compatible with the complexity indices we proposed. This suggests that our model map may be valid in guiding customized applications. The instructions were considered sufficiently clear, and the perceived levels of difficulty were compatible with our pre-assigned ratings, despite no significant differences in accuracy based on these ratings. Accuracy correlated strongly with both perceived difficulty and clarity of instructions.

Given the limited number of responses received at each level, it is important to emphasize the need for cautious analysis of the results, since they may not be representative of the population. It should also be noted that the bulk of the data originates from group participation, reflecting the views of the majority within these groups. Individual response profiles, which would offer more reliable conclusions, were not obtained. Furthermore, due to little interest from institutions, the extent of adherence to specific tasks is deemed irrelevant. This is because the collective group responded to all tasks twice and, in many instances, they compromised the sole sources of accesses and responses to the program. Although, the data acquired at this point indicate the necessity for adjustments in various aspects of CerUp, it is also evident that there have been no major hindrances to its functioning.

Taking all the above into consideration, future directions could focus on reviewing and potentially improving the instructions, structure, and difficulty levels of the CerUp program. Key areas for attention include reevaluating the instructions for clarity, comprehensiveness, and effectiveness in guiding users through the program. Moreover, adjustments to the segmentation of tasks and difficulty levels may be necessary to ensure the program is appropriately challenging without causing user fatigue or discouragement. These parameters are particularly important given recent findings suggesting that older individuals can think as divergently as younger ones in the verbal domain, provided there are no time constraints [52] and the workload is not excessive [53]. It is strongly recommended that future programs invest more in language production exercises (verbal tasks or written exercises) since recent studies indicate that problem-solving and creativity skills, verbal skills and verbal divergent thinking abilities appear to be preserved in older age [54,55,56]. Additionally, considering the relationship between divergent thinking and the construct of cognitive reserve [57,58], or even the effect of semantic abilities and language performance as predictors of developing dementia [59,60] it is important to utilize the preserved verbal divergent thinking abilities. Consequently, verbal divergent thinking should be considered a target for cognitive enhancement programs designed to support active aging and reduce the risk of dementia [61]. The CerUp program could serve as an example of how a tool can be systematically developed to emphasize the rehabilitation and enhancement of language production, thereby enhancing the general cognitive abilities in healthy elderly adults.