Version 1
: Received: 19 August 2019 / Approved: 20 August 2019 / Online: 20 August 2019 (09:50:03 CEST)
Version 2
: Received: 13 December 2021 / Approved: 14 December 2021 / Online: 14 December 2021 (10:40:37 CET)
Version 3
: Received: 12 January 2023 / Approved: 13 January 2023 / Online: 13 January 2023 (10:52:30 CET)
Version 4
: Received: 26 June 2023 / Approved: 26 June 2023 / Online: 26 June 2023 (10:56:59 CEST)
Németh, D.; Gerbier, E.; Born, J.; Rickard, T.; Diekelmann, S.; Fogel, S.; Genzel, L.; Prehn-Kristensen, A.; Payne, J.; Dresler, M.; et al. Optimizing the Methodology of Human Sleep and Memory Research. Nature Reviews Psychology 2023, doi:10.1038/s44159-023-00262-0.
Németh, D.; Gerbier, E.; Born, J.; Rickard, T.; Diekelmann, S.; Fogel, S.; Genzel, L.; Prehn-Kristensen, A.; Payne, J.; Dresler, M.; et al. Optimizing the Methodology of Human Sleep and Memory Research. Nature Reviews Psychology 2023, doi:10.1038/s44159-023-00262-0.
Németh, D.; Gerbier, E.; Born, J.; Rickard, T.; Diekelmann, S.; Fogel, S.; Genzel, L.; Prehn-Kristensen, A.; Payne, J.; Dresler, M.; et al. Optimizing the Methodology of Human Sleep and Memory Research. Nature Reviews Psychology 2023, doi:10.1038/s44159-023-00262-0.
Németh, D.; Gerbier, E.; Born, J.; Rickard, T.; Diekelmann, S.; Fogel, S.; Genzel, L.; Prehn-Kristensen, A.; Payne, J.; Dresler, M.; et al. Optimizing the Methodology of Human Sleep and Memory Research. Nature Reviews Psychology 2023, doi:10.1038/s44159-023-00262-0.
Abstract
Understanding the complex relationship between sleep and memory is one of the biggest challenges in neuroscience. Thousands of studies on memory consolidation suggest that sleep triggers offline memory processes, resulting in less forgetting in declarative memory and performance improvement in non-declarative memory. However, an increasing number of contradictory findings reveal potential issues with how research is conducted in this field, that weaken the reliability of these results. Here we describe four methodological pitfalls with respect to experimental designs and statistical analyses that should be avoided in order to unveil the true effect of sleep on memory consolidation: non-optimal experimental designs, task complexity, fatigue effect in repetitive tasks, and data analysis and availability. We then offer solutions that can be used in future research of sleep-dependent consolidation and also more broadly in memory research.
Keywords
sleep; memory; consolidation; napping; fatigue
Subject
Social Sciences, Cognitive Science
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Received:
13 September 2019
Commenter:
Shane Lindsay
The commenter has declared there is no conflict of interests.
Comment:
I like the article and find it helpful.
One bit of feedback though on power:
"Importantly, sample sizes are usually not being determined by a priori power analyses that are based on the expected effect sizes"
"It has long been recommended in guidelines (e.g.,American Psychological Association) that experimenters should determine the sample size before starting the experiment by computing power analyses based on the magnitude of the effect size estimated or found in previous studies that observed similar effects"
This might be true, but I would that current best thinking in response to the replication crisis in psychology would not support such a recommendation.
The problem is if you base your "expected effect size" estimates on inflated effect sizes from underpowered small n studies, it will perpetuate more under powered studies, with low power to detect smaller effect sizes than those that might be expected from previous research.
Different approaches to this problem exist. But a good one is to recommend researchers consider the "smallest effect size of interest" in determining power, and take published effect size estimates with a pinch of salt.
Commenter: Shane Lindsay
The commenter has declared there is no conflict of interests.
One bit of feedback though on power:
"Importantly, sample sizes are usually not being determined by a priori power analyses that are based on the expected effect sizes"
"It has long been recommended in guidelines (e.g.,American Psychological Association) that experimenters should determine the sample size before starting the experiment by computing power analyses based on the magnitude of the effect size estimated or found in previous studies that observed similar effects"
This might be true, but I would that current best thinking in response to the replication crisis in psychology would not support such a recommendation.
For example, see:
https://github.com/richarddmorey/psychology_resolution
And the section:
"Misconception 3: Sample size choice should be based on previous results".
The problem is if you base your "expected effect size" estimates on inflated effect sizes from underpowered small n studies, it will perpetuate more under powered studies, with low power to detect smaller effect sizes than those that might be expected from previous research.
Different approaches to this problem exist. But a good one is to recommend researchers consider the "smallest effect size of interest" in determining power, and take published effect size estimates with a pinch of salt.
e.g. see
http://daniellakens.blogspot.com/2017/05/how-power-analysis-implicitly-reveals.html
My suggestion might to be the revise the wording of the text quoted above and modify the advice.
Best wishes,
Shane