Version 1
: Received: 20 April 2020 / Approved: 21 April 2020 / Online: 21 April 2020 (05:42:47 CEST)
Version 2
: Received: 1 June 2020 / Approved: 3 June 2020 / Online: 3 June 2020 (05:10:05 CEST)
Hisaka, A.; Yoshioka, H.; Hatakeyama, H.; Sato, H.; Onouchi, Y.; Anzai, N. Global Comparison of Changes in the Number of Test-Positive Cases and Deaths by Coronavirus Infection (COVID-19) in the World. J. Clin. Med.2020, 9, 1904.
Hisaka, A.; Yoshioka, H.; Hatakeyama, H.; Sato, H.; Onouchi, Y.; Anzai, N. Global Comparison of Changes in the Number of Test-Positive Cases and Deaths by Coronavirus Infection (COVID-19) in the World. J. Clin. Med. 2020, 9, 1904.
Journal reference: Journal of Clinical Medicine 2020, 9, 1904 DOI: 10.3390/jcm9061904
Cite as:
Hisaka, A.; Yoshioka, H.; Hatakeyama, H.; Sato, H.; Onouchi, Y.; Anzai, N. Global Comparison of Changes in the Number of Test-Positive Cases and Deaths by Coronavirus Infection (COVID-19) in the World. J. Clin. Med.2020, 9, 1904.
Hisaka, A.; Yoshioka, H.; Hatakeyama, H.; Sato, H.; Onouchi, Y.; Anzai, N. Global Comparison of Changes in the Number of Test-Positive Cases and Deaths by Coronavirus Infection (COVID-19) in the World. J. Clin. Med. 2020, 9, 1904.
Abstract
Global differences in changes in the numbers of population-adjusted daily test-positive cases (NPDP) and deaths (NPDD) by COVID-19 were analyzed for 49 countries. The changes per population of a hundred million were compared, adjusting by the beginning of test-positive cases increase (BPI) or deaths increase (BDI). Notable regional differences of more than 100 times in NPDP and NPDD were observed. The trajectories of NPDD after BDI increased exponentially within 20 days in most countries. A machine learning analysis suggested that NPDD on 30 days after BDI was the highest in Western countries (1180), followed by the Middle East (128), Latin America (97), and then Asia (7), and furthermore that, NPDD in Western countries with a positive rate of the PCR test of less than 7.0% attenuated to only 15%. The cause behind differences between regions might be complex, however, investigation of the host genetic factors would be warranted. The lower positive rate would be caused by aggressive testing policy and associated with longer lag times between BPI and BDI. Our analysis suggested that the positive rate need to be 7% or less by extensive tests to reduce deaths effectively. As the number of infected people is growing rapidly, earlier expansion of the test capacity is indispensable.
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Received:
25 April 2020
Commenter:
Taylor Yamdif
The commenter has declared there is no conflict of interests.
Comment:
There are a few fatal flaws in this study.
First of all, you are confusing correlation with causation.
Secondly, there is no statistical control and verification of the target population.
Third, the results of machine learning, which is a classifier, are being misused without any verification.
The authors claim that, "You there! One lemon contains as much vitamin C as one lemon! ", is merely saying.
Each point will be addressed bellow.
1. misunderstanding of correlation and causation
The author states that there is no relationship between the number of tests and the number of deaths adjusted for, but argues that there is a correlation between the "positive rate" using the number of tests as the denominator and the "number of deaths". In other words, it is obvious that there is a correlation between the number of confirmed infections and the number of deaths in the numerator of the positive rate. However, the authors purport to be the first to find a correlation between the positive rate and the number of deaths, and the logical development is already broken and fatal at this point.
2. statistical controls are not in place and a lot of bias remains
The authors introduced an observational variable, the positivity rate, which does not go up or down by a single variable, the number of tests, but depends on the number of confirmed infections of the molecules tested. The actual status of the infection and the probability of positive results due to the testing method are still uncertain at present, and selection bias is very high. Furthermore, the authors deliberately exclude data from countries that are disadvantaged in deriving their results.
If the positive test rate is to be used as an observational variable, these baseline conditions should be statistically controlled to be within a certain range across countries, or they should be shown to be already within a certain range. However, there is no indication that the authors exercised any control over the inspection situation. It is a misreading because it does not take into account the vastly different testing regimes, hospitalization conditions, and timing of treatment interventions in each country into account at all, and it begins to develop its logic using the positive test rate figures as a goal.
3. given the relationship between the low positive rate and the low number of deaths in the data, isn't the correct interpretation in light of "Occam's Razor" to assume that the number of deaths is low because the infection is not widespread?
If we use the Directed acyclic graph (DAG) to organize the causal flow, we can see that the story of this paper is fatally wrong. The authors' logic, if presented in DAG, is as follows.
The authors' logic is that increased positive rates reduce deaths. But there is no causal relationship between the two numbers of positive rates and the number of deaths." It is reasonable to assume that the common cause of both the "positive rate" and the "number of deaths" is the "degree of spread of infection" in the first place, and that the two phenomena of "positive test" and "patient death" arise from this.
"Spread of infection," "more positive tests."
↓arrow
"The number of deaths will increase."
4. the authors have not verified the validity of the classification as to why the result is 7%, which is a typical example of misuse of statistics and machine learning.
The authors write that they used a gradient boosting decision tree. Machine learning returns some relationship as a result, with or without validity. We find "structures" and "features" from vast amounts of data that cannot be found in human cognition, but they are not causal relationships. The gradient boosting decision tree is a classifier and does not establish a causal relationship at all. I'm just categorizing the pattern of deaths and positive rates. The author has only categorized them into three groups, and there is no explanation for the cause there. Also, there is no statistical validation of anything. You need to interpret the fit with other validation data, not the training data used directly for the analysis. However, there is no indication of any such verification.
Received:
29 April 2020
Commenter: Akihiro Hisaka
The commenter has declared there is no conflict of interests.
Comment:
Comment:
There are a few fatal flaws in this study.
First of all, you are confusing correlation with causation.
Secondly, there is no statistical control and verification of the target population.
Third, the results of machine learning, which is a classifier, are being misused without any verification.
The authors claim that, "You there! One lemon contains as much vitamin C as one lemon! ", is merely saying.
Each point will be addressed bellow.
Response:
Thank you for your comments on our paper. There are two major findings in this study. One is the large global and regional differences in COVID-19 epidemics. Surprisingly the numbers of confirmed cases and deaths per unit of population differed approximately 100-fold between Western and Asian countries. The other is the relationship between rates of deaths and the positive PCR results in Western countries where the number of countries is relatively large. I understand that all the issues you pointed out are focused on the second finding, especially the soundness of our hypothesis and the statistical method used for the analysis. Since this study is an ecological one a causal relationship cannot be proven as you pointed out. It is natural that many factors are involved in the spread of infection in each country, and since reproducible experiments are not possible, a causal relationship cannot be proved strictly for any infection spread. In that respect, we agree that some of our descriptions were excessive and should be corrected. We should carefully examine the factors that found the correlation and the possibility of confounding.
We couldn’t establish a complete control group in this study simply because the situations in the countries differed so much. It is obvious that the analysis using full epidemiology information is ideal and should be done for preparation for the future pandemic of another emerging or re-emerging infectious disease. However such analysis can be made after the epidemic becomes controlled in many of the countries and data become available. We performed this study considering the current tragic situation of COVID-19 in the world, which needs various urgent decisions based on the available data at present. Machine learning was used to predict the progress of the infection in a relatively short period for each country group because the progress of the infection was not sufficient in some countries at the time of observation. Machine learning was not used as a classifier. We submitted the source code of analysis to the journal but it was not attached to the preprint. Therefore, we should describe it clearer in the manuscript. We thought that a comprehensive analysis to determine various affecting factors of the infection was yet difficult given that countries have so many attributes and that the progress of the infection was not sufficient in many countries.
Comment: 1. misunderstanding of correlation and causation
The author states that there is no relationship between the number of tests and the number of deaths adjusted for, but argues that there is a correlation between the "positive rate" using the number of tests as the denominator and the "number of deaths". In other words, it is obvious that there is a correlation between the number of confirmed infections and the number of deaths in the numerator of the positive rate. However, the authors purport to be the first to find a correlation between the positive rate and the number of deaths, and the logical development is already broken and fatal at this point.
Response 1:
Regarding this point, it is necessary to consider differences between an increase in the number of test-positive cases (NPDP) and an increase in deaths (NPDD), and it is true that sufficient explanation was not provided in the original paper, so we need to add it. As shown in the supplemental figure (added in version 2, This version was not changed other than an addition of this figure), there was no difference in the degree of increase in NPDP up to two weeks after BPI in countries with low and high positive rates. Your point is that since the positive rate and the number of test-positive cases were directly related, it is very natural that the positive rate and the number of deaths were likely to be related. However, in the early stage of the epidemic, changes in the number of test-positive cases were independent of the positive rate.
From the point of view of your comments, the positive rate of countries with a low positive rate had become low as a result of the smaller infection spread. Therefore, the difference between countries with a low positive rate and a high positive rate would be small before the infection spread. In this case, it may be natural that differences in NPDP were small between these countries in the early stages of infection. Or, from a conservative perspective, if countries with a low positive rate tended to perform a large number of tests from the beginning, it is possible that more positive cases were found. In this case, even if the number of test-positive cases was seemingly similar, the extent of infection spread would be low in countries with a low positive rate, and also the number of seriously infected people would be actually lower than in countries with a high positive rate.
However, in both cases, it cannot be explained that the slope of change in NPDP was attenuated only in countries with a low positive rate at two weeks after BPI. We think this attenuation of the slope at two weeks after BPI is very important and is actually the implicit target of our analysis. The slope after this attenuation was reflected over time in the slow increase in NPDD in countries with a low positive rate shown in Figure 5B in the paper. The change in the slope of NPDD would have caused the difference in the lag time between BPI and BDI which was also discussed in the paper. Therefore, the change in NPDD 30 days after BDI can be regarded as a reflection of the change in the slope of NPDP. Given this background, we thought that it is reasonable to compare NPDDs in countries with a different positive rate for assessing a depressing factor that influences the rate of spread of infection. In addition, considering that approximately two weeks are necessary for the detection of test-positive cases after the infection, the policy of testing in each country can be a possible causative factor that impacted the change of the spread of epidemic from the detection of test-positive cases.
As stated in the text, the absolute value of NPDP depends on the testing policy by each country (but please note that relative changes of NPDP in each country are yet informative). Therefore, NPDD, which is not affected by such differences, was considered more direct than discussing the change in NPDP. In addition, it can be only confusing to discuss differences in BPI and BDI when changes in NPDP is considered. On the other hand, when discussing the basis of causal relationships as you pointed out, the use of NPDD needs detailed explanations for specialists as provided here. I would like to consider which of these issues should be discussed, based on the reviewers' points of view. Your comment provided us a good opportunity to explain deeper into this issue. I thank you again.
An additional point to note is that if it is true that sufficient tests are effective in controlling the spread of infection, the positive rate is a good index of the sufficiency of the test and the number of population-adjusted deaths is a good index of the infection spread. Thus, it seems theoretically impossible to completely solve the causal problem which was pointed out by you in this case. Even if it is true, it cannot be proved eternally. Therefore, it is an important option for understanding the current situation to make comparisons as done in this study with careful considerations of causal relationships and confounding issues.
Successful isolation of infected cases from the city in countries that were well-tested from the outbreak of infection may result in the subsequent spread of infection being suppressed as suggested by the slope of NPDP. It seems reasonable to assume that the decreased seriously infected cases reduced the overburden on the medical system and provided sufficient medical treatment, leading to a significant reduction in deaths. As you pointed out, these are only an interpretation and we also need to describe that, in countries where the positive rate was low, the possibility that confounding factors such as high medical capacity or different national policies or characteristics, age composition, and ethnic composition may also be involved in this result. In addition, we think that 7% of a positive rate may be only a snapshot of various changes during the epidemic. The positive rate of the PCR test is informative but the absolute target value is difficult to determine.
Comment: 2. statistical controls are not in place and a lot of bias remains
The authors introduced an observational variable, the positivity rate, which does not go up or down by a single variable, the number of tests, but depends on the number of confirmed infections of the molecules tested. The actual status of the infection and the probability of positive results due to the testing method are still uncertain at present, and selection bias is very high. Furthermore, the authors deliberately exclude data from countries that are disadvantaged in deriving their results.
If the positive test rate is to be used as an observational variable, these baseline conditions should be statistically controlled to be within a certain range across countries, or they should be shown to be already within a certain range. However, there is no indication that the authors exercised any control over the inspection situation. It is a misreading because it does not take into account the vastly different testing regimes, hospitalization conditions, and timing of treatment interventions in each country into account at all, and it begins to develop its logic using the positive test rate figures as a goal.
Response 2:
As far as we know, many countries are using the PCR test as the basis for final judgment, and even when other methods such as the antibody test or image diagnosis are used together, there is no major conflict with the PCR test result. Therefore, I do not think that the selection bias due to the difference in the inspection method is very high for COVID-19. Since this study targeted the fatalities, we had no choice but to limit the countries subject to analysis to those with a certain degree of spread of infection. The selection of the country this time is based on an objective indicator that the number of infected persons is above a certain level, and there is no fact that a specific country was not intentionally analyzed. Of course, infections are progressing rapidly and the number of countries that can be analyzed is increasing at present.
There is no baseline value before the infection because the positive rate is the number that appears as responses to the infection. Therefore such control is impossible. I actually think that the change in the positive rate during the spread of infection should also be included in the complete analysis in the future. But at present, detailed information is not available in many countries in the world. I would like to make it when the information is obtained.
Comment: 3. given the relationship between the low positive rate and the low number of deaths in the data, isn't the correct interpretation in light of "Occam's Razor" to assume that the number of deaths is low because the infection is not widespread?
If we use the Directed acyclic graph (DAG) to organize the causal flow, we can see that the story of this paper is fatally wrong. The authors' logic, if presented in DAG, is as follows.
The authors' logic is that increased positive rates reduce deaths. But there is no causal relationship between the two numbers of positive rates and the number of deaths." It is reasonable to assume that the common cause of both the "positive rate" and the "number of deaths" is the "degree of spread of infection" in the first place, and that the two phenomena of "positive test" and "patient death" arise from this.
"Spread of infection," "more positive tests."
↓arrow
"The number of deaths will increase."
Response 3:
I think our response to comment 1 explains the answer to this comment.
Our logic is like this.
"More test" -> "effective isolation" -> "reduce epidemic" and "lower positive rate"
->" fewer death"
Of course, this is one possible interpretation of the current observation.
Comment: 4. the authors have not verified the validity of the classification as to why the result is 7%, which is a typical example of misuse of statistics and machine learning.
The authors write that they used a gradient boosting decision tree. Machine learning returns some relationship as a result, with or without validity. We find "structures" and "features" from vast amounts of data that cannot be found in human cognition, but they are not causal relationships. The gradient boosting decision tree is a classifier and does not establish a causal relationship at all. I'm just categorizing the pattern of deaths and positive rates. The author has only categorized them into three groups, and there is no explanation for the cause there. Also, there is no statistical validation of anything. You need to interpret the fit with other validation data, not the training data used directly for the analysis. However, there is no indication of any such verification.
Response 4:
As mentioned at the beginning, the use of machine learning in this study is not for identifying the influencing factors, but to predict the number of deaths as a group because the spread of infection was still insufficient to assess the number at 30 days after BDI in many countries. The reliability of the prediction is evaluated by the bootstrap method. It was judged that there is a clear difference in the estimated values as a group because the confidence interval does not intersect at all. The thresholds of 7% and 17% were adopted as values close to the quartile. We will mention these explanations as well in the paper.
Given the myriad of attributes in a country, it is not possible without a hypothesis to find the factors that contribute to the observed reduction in deaths. Thus, it is necessary to select a rational hypothesis and consider its superiority or inferiority, but since it constantly fluctuates depending on the spread of infection, it is unavoidable to make the best of the current available information.
I found a sentence in the Discussion that "machine learning analysis classified ..." and this was misleading. It should be corrected and at the same time, we would like to add an explanation in the appropriate place to clarify why we used machine learning.
Response 2 to
Comment 1
Received:
30 April 2020
Commenter: Akihiro Hisaka
The commenter has declared there is no conflict of interests.
Comment:
We added the supplemental Figure 1 to version 1, not to version 2. This was in accord with a suggestion by Preprints.
Comment 2
Received:
30 April 2020
The commenter has declared there is no conflict of interests.
Comment:
Hi, Prof. Hisaka,
(1) Sometimes I wish I could. However, you cannot claim causality out of correlation, no matter how strong it is.
A couple of examples: "cities with more trees are safer" does not mean trees make cities safer.
"Countries with more Mangos have more Malaria patients" does Not imply that cutting down mango trees will reduce Malaria patients. You need to exterminate Mosquitos. Not Mangos.
I wish More PCR tests indeed have a causal impact on the reduction of NPDD, however I cannot claim that just because countries with more PCR tests have lower NPDD.
(2) You will have to find a good exogenous variation in the NPDP, hopefully at the country level, but if it is difficult, at least at the community level, to avoid possible between-unit spillover effects. One possible, however, never-the-best way would be to start out with DID, or DID-PSM.
Received:
6 May 2020
Commenter:
Yasuhiro Ishikawa
The commenter has declared there is no conflict of interests.
Comment:
If we calculate the initial effective infection rate R0 (R zero), which was a plateau when the contact rate was 0.6, to 1.7, then the plateau would be 1.7 * 0.6≈1, so it seems that R0 is considered to be 1.7 instead of 2. Even R0 = 1.7 is quite large.
The rest seems to be inferred from the differential equations of a simple SIR model.
By the way, PCR is intentionally narrowed down, so it is said that it is not possible to predict the future because the rate of community transmission is unknown on TV.
Professor Yamanaka also says that a 40% PCR positive rate in Tokyo is dangerous. The rationale is not given.
Therefore, I estimated the rate of community-acquired infection in Tokyo.
In Bayesian statistics, the posterior probability changes depending on the prior probability. For Covid-19 infections, the PCR positive rate is the posterior probability. On the contrary, from the posterior probability (positive rate), try to predict the prior probability (community infection rate).
PCR test sensitivity Se = 0.7
Specificity Sp = 0.97
Posterior probability (positive rate) Po = 0.4 (PCR positive rate).
Obtain the prior probability (community infection rate) Pr.
Pr = Po * (1-Sp) / (Se * (1-Po) + Po * (1-Sp)) Pr (community infection rate) is the left formula.
The positive PCR test rate in Tokyo is 0.4.
Pr = 0.4 * (1-0.97) / (0.7 * (1-0.4) + 0.4 * (1-0.97))
= 0.028 When the PCR test positive rate is 40%, the local infection rate is 2.8%
If the population of Tokyo is 10 million, the number of infected people is 280,000.
(Currently, a little less than 4000 confirmed cases are infected by PCR.)
Even if the PCR test is negative, the probability of being infected with Covid-19 is 23.6%.
Even with PCR negatives, 1 in 4 is infected.
If the sensitivity of PCR test is a little worse and Se = 0.6,
When PCR test positive rate is 40%, Pr = 0.032, local infection rate is 3.2%
If the population of Tokyo is 10 million, the number of infected people will be 320,000.
It is said that 80% of asymptomatic infected carriers (carriers) and mild patients do not need elementary inhalation, but in Tokyo, 98% are carriers or mild patients, and further stay at home is required.
Here, the prior probability and community-acquired infection rate are used, but since PCR testing is the target, it is better to think of it as the probability of those who have a coronavirus. (April 29, 2nd year of Reiwa)
Addendum On May 5, 2012, Governor Yoshimura of Osaka Prefecture indicated a PCR test positive rate of 7% or less as one of the numerical criteria for canceling the emergency declaration.
If the population of Osaka Prefecture is 8.8 million, the prior probability is 0.0032, and 28,000 people or 0.32% will be the number of people who have coronavirus. --------------
M.D. Yasuhiro Ishikawa
E-Mail: [email protected][email protected] WWW href="http://www.uinet.or.jp/~ishiyasu/" target="_blank">http://www.uinet.or.jp/~ishiyasu/
Akihiro Hisaka
*
,
Hiroto Hatakeyama
,
Commenter: Taylor Yamdif
The commenter has declared there is no conflict of interests.
First of all, you are confusing correlation with causation.
Secondly, there is no statistical control and verification of the target population.
Third, the results of machine learning, which is a classifier, are being misused without any verification.
The authors claim that, "You there! One lemon contains as much vitamin C as one lemon! ", is merely saying.
Each point will be addressed bellow.
1. misunderstanding of correlation and causation
The author states that there is no relationship between the number of tests and the number of deaths adjusted for, but argues that there is a correlation between the "positive rate" using the number of tests as the denominator and the "number of deaths". In other words, it is obvious that there is a correlation between the number of confirmed infections and the number of deaths in the numerator of the positive rate. However, the authors purport to be the first to find a correlation between the positive rate and the number of deaths, and the logical development is already broken and fatal at this point.
2. statistical controls are not in place and a lot of bias remains
The authors introduced an observational variable, the positivity rate, which does not go up or down by a single variable, the number of tests, but depends on the number of confirmed infections of the molecules tested. The actual status of the infection and the probability of positive results due to the testing method are still uncertain at present, and selection bias is very high. Furthermore, the authors deliberately exclude data from countries that are disadvantaged in deriving their results.
If the positive test rate is to be used as an observational variable, these baseline conditions should be statistically controlled to be within a certain range across countries, or they should be shown to be already within a certain range. However, there is no indication that the authors exercised any control over the inspection situation. It is a misreading because it does not take into account the vastly different testing regimes, hospitalization conditions, and timing of treatment interventions in each country into account at all, and it begins to develop its logic using the positive test rate figures as a goal.
3. given the relationship between the low positive rate and the low number of deaths in the data, isn't the correct interpretation in light of "Occam's Razor" to assume that the number of deaths is low because the infection is not widespread?
If we use the Directed acyclic graph (DAG) to organize the causal flow, we can see that the story of this paper is fatally wrong. The authors' logic, if presented in DAG, is as follows.
"More tests" → "lower positive rate" → "fewer deaths"
The authors' logic is that increased positive rates reduce deaths. But there is no causal relationship between the two numbers of positive rates and the number of deaths." It is reasonable to assume that the common cause of both the "positive rate" and the "number of deaths" is the "degree of spread of infection" in the first place, and that the two phenomena of "positive test" and "patient death" arise from this.
"Spread of infection," "more positive tests."
↓arrow
"The number of deaths will increase."
4. the authors have not verified the validity of the classification as to why the result is 7%, which is a typical example of misuse of statistics and machine learning.
The authors write that they used a gradient boosting decision tree. Machine learning returns some relationship as a result, with or without validity. We find "structures" and "features" from vast amounts of data that cannot be found in human cognition, but they are not causal relationships. The gradient boosting decision tree is a classifier and does not establish a causal relationship at all. I'm just categorizing the pattern of deaths and positive rates. The author has only categorized them into three groups, and there is no explanation for the cause there. Also, there is no statistical validation of anything. You need to interpret the fit with other validation data, not the training data used directly for the analysis. However, there is no indication of any such verification.
Commenter:
The commenter has declared there is no conflict of interests.
There are a few fatal flaws in this study.
First of all, you are confusing correlation with causation.
Secondly, there is no statistical control and verification of the target population.
Third, the results of machine learning, which is a classifier, are being misused without any verification.
The authors claim that, "You there! One lemon contains as much vitamin C as one lemon! ", is merely saying.
Each point will be addressed bellow.
Response:
Thank you for your comments on our paper. There are two major findings in this study. One is the large global and regional differences in COVID-19 epidemics. Surprisingly the numbers of confirmed cases and deaths per unit of population differed approximately 100-fold between Western and Asian countries. The other is the relationship between rates of deaths and the positive PCR results in Western countries where the number of countries is relatively large. I understand that all the issues you pointed out are focused on the second finding, especially the soundness of our hypothesis and the statistical method used for the analysis. Since this study is an ecological one a causal relationship cannot be proven as you pointed out. It is natural that many factors are involved in the spread of infection in each country, and since reproducible experiments are not possible, a causal relationship cannot be proved strictly for any infection spread. In that respect, we agree that some of our descriptions were excessive and should be corrected. We should carefully examine the factors that found the correlation and the possibility of confounding.
We couldn’t establish a complete control group in this study simply because the situations in the countries differed so much. It is obvious that the analysis using full epidemiology information is ideal and should be done for preparation for the future pandemic of another emerging or re-emerging infectious disease. However such analysis can be made after the epidemic becomes controlled in many of the countries and data become available. We performed this study considering the current tragic situation of COVID-19 in the world, which needs various urgent decisions based on the available data at present. Machine learning was used to predict the progress of the infection in a relatively short period for each country group because the progress of the infection was not sufficient in some countries at the time of observation. Machine learning was not used as a classifier. We submitted the source code of analysis to the journal but it was not attached to the preprint. Therefore, we should describe it clearer in the manuscript. We thought that a comprehensive analysis to determine various affecting factors of the infection was yet difficult given that countries have so many attributes and that the progress of the infection was not sufficient in many countries.
Comment: 1. misunderstanding of correlation and causation
The author states that there is no relationship between the number of tests and the number of deaths adjusted for, but argues that there is a correlation between the "positive rate" using the number of tests as the denominator and the "number of deaths". In other words, it is obvious that there is a correlation between the number of confirmed infections and the number of deaths in the numerator of the positive rate. However, the authors purport to be the first to find a correlation between the positive rate and the number of deaths, and the logical development is already broken and fatal at this point.
Response 1:
Regarding this point, it is necessary to consider differences between an increase in the number of test-positive cases (NPDP) and an increase in deaths (NPDD), and it is true that sufficient explanation was not provided in the original paper, so we need to add it. As shown in the supplemental figure (added in version 2, This version was not changed other than an addition of this figure), there was no difference in the degree of increase in NPDP up to two weeks after BPI in countries with low and high positive rates. Your point is that since the positive rate and the number of test-positive cases were directly related, it is very natural that the positive rate and the number of deaths were likely to be related. However, in the early stage of the epidemic, changes in the number of test-positive cases were independent of the positive rate.
From the point of view of your comments, the positive rate of countries with a low positive rate had become low as a result of the smaller infection spread. Therefore, the difference between countries with a low positive rate and a high positive rate would be small before the infection spread. In this case, it may be natural that differences in NPDP were small between these countries in the early stages of infection. Or, from a conservative perspective, if countries with a low positive rate tended to perform a large number of tests from the beginning, it is possible that more positive cases were found. In this case, even if the number of test-positive cases was seemingly similar, the extent of infection spread would be low in countries with a low positive rate, and also the number of seriously infected people would be actually lower than in countries with a high positive rate.
However, in both cases, it cannot be explained that the slope of change in NPDP was attenuated only in countries with a low positive rate at two weeks after BPI. We think this attenuation of the slope at two weeks after BPI is very important and is actually the implicit target of our analysis. The slope after this attenuation was reflected over time in the slow increase in NPDD in countries with a low positive rate shown in Figure 5B in the paper. The change in the slope of NPDD would have caused the difference in the lag time between BPI and BDI which was also discussed in the paper. Therefore, the change in NPDD 30 days after BDI can be regarded as a reflection of the change in the slope of NPDP. Given this background, we thought that it is reasonable to compare NPDDs in countries with a different positive rate for assessing a depressing factor that influences the rate of spread of infection. In addition, considering that approximately two weeks are necessary for the detection of test-positive cases after the infection, the policy of testing in each country can be a possible causative factor that impacted the change of the spread of epidemic from the detection of test-positive cases.
As stated in the text, the absolute value of NPDP depends on the testing policy by each country (but please note that relative changes of NPDP in each country are yet informative). Therefore, NPDD, which is not affected by such differences, was considered more direct than discussing the change in NPDP. In addition, it can be only confusing to discuss differences in BPI and BDI when changes in NPDP is considered. On the other hand, when discussing the basis of causal relationships as you pointed out, the use of NPDD needs detailed explanations for specialists as provided here. I would like to consider which of these issues should be discussed, based on the reviewers' points of view. Your comment provided us a good opportunity to explain deeper into this issue. I thank you again.
An additional point to note is that if it is true that sufficient tests are effective in controlling the spread of infection, the positive rate is a good index of the sufficiency of the test and the number of population-adjusted deaths is a good index of the infection spread. Thus, it seems theoretically impossible to completely solve the causal problem which was pointed out by you in this case. Even if it is true, it cannot be proved eternally. Therefore, it is an important option for understanding the current situation to make comparisons as done in this study with careful considerations of causal relationships and confounding issues.
Successful isolation of infected cases from the city in countries that were well-tested from the outbreak of infection may result in the subsequent spread of infection being suppressed as suggested by the slope of NPDP. It seems reasonable to assume that the decreased seriously infected cases reduced the overburden on the medical system and provided sufficient medical treatment, leading to a significant reduction in deaths. As you pointed out, these are only an interpretation and we also need to describe that, in countries where the positive rate was low, the possibility that confounding factors such as high medical capacity or different national policies or characteristics, age composition, and ethnic composition may also be involved in this result. In addition, we think that 7% of a positive rate may be only a snapshot of various changes during the epidemic. The positive rate of the PCR test is informative but the absolute target value is difficult to determine.
Comment: 2. statistical controls are not in place and a lot of bias remains
The authors introduced an observational variable, the positivity rate, which does not go up or down by a single variable, the number of tests, but depends on the number of confirmed infections of the molecules tested. The actual status of the infection and the probability of positive results due to the testing method are still uncertain at present, and selection bias is very high. Furthermore, the authors deliberately exclude data from countries that are disadvantaged in deriving their results.
If the positive test rate is to be used as an observational variable, these baseline conditions should be statistically controlled to be within a certain range across countries, or they should be shown to be already within a certain range. However, there is no indication that the authors exercised any control over the inspection situation. It is a misreading because it does not take into account the vastly different testing regimes, hospitalization conditions, and timing of treatment interventions in each country into account at all, and it begins to develop its logic using the positive test rate figures as a goal.
Response 2:
As far as we know, many countries are using the PCR test as the basis for final judgment, and even when other methods such as the antibody test or image diagnosis are used together, there is no major conflict with the PCR test result. Therefore, I do not think that the selection bias due to the difference in the inspection method is very high for COVID-19. Since this study targeted the fatalities, we had no choice but to limit the countries subject to analysis to those with a certain degree of spread of infection. The selection of the country this time is based on an objective indicator that the number of infected persons is above a certain level, and there is no fact that a specific country was not intentionally analyzed. Of course, infections are progressing rapidly and the number of countries that can be analyzed is increasing at present.
There is no baseline value before the infection because the positive rate is the number that appears as responses to the infection. Therefore such control is impossible. I actually think that the change in the positive rate during the spread of infection should also be included in the complete analysis in the future. But at present, detailed information is not available in many countries in the world. I would like to make it when the information is obtained.
Comment: 3. given the relationship between the low positive rate and the low number of deaths in the data, isn't the correct interpretation in light of "Occam's Razor" to assume that the number of deaths is low because the infection is not widespread?
If we use the Directed acyclic graph (DAG) to organize the causal flow, we can see that the story of this paper is fatally wrong. The authors' logic, if presented in DAG, is as follows.
"More tests" → "lower positive rate" → "fewer deaths"
The authors' logic is that increased positive rates reduce deaths. But there is no causal relationship between the two numbers of positive rates and the number of deaths." It is reasonable to assume that the common cause of both the "positive rate" and the "number of deaths" is the "degree of spread of infection" in the first place, and that the two phenomena of "positive test" and "patient death" arise from this.
"Spread of infection," "more positive tests."
↓arrow
"The number of deaths will increase."
Response 3:
I think our response to comment 1 explains the answer to this comment.
Our logic is like this.
"More test" -> "effective isolation" -> "reduce epidemic" and "lower positive rate"
->" fewer death" Of course, this is one possible interpretation of the current observation.
Comment: 4. the authors have not verified the validity of the classification as to why the result is 7%, which is a typical example of misuse of statistics and machine learning.
The authors write that they used a gradient boosting decision tree. Machine learning returns some relationship as a result, with or without validity. We find "structures" and "features" from vast amounts of data that cannot be found in human cognition, but they are not causal relationships. The gradient boosting decision tree is a classifier and does not establish a causal relationship at all. I'm just categorizing the pattern of deaths and positive rates. The author has only categorized them into three groups, and there is no explanation for the cause there. Also, there is no statistical validation of anything. You need to interpret the fit with other validation data, not the training data used directly for the analysis. However, there is no indication of any such verification.
Response 4:
As mentioned at the beginning, the use of machine learning in this study is not for identifying the influencing factors, but to predict the number of deaths as a group because the spread of infection was still insufficient to assess the number at 30 days after BDI in many countries. The reliability of the prediction is evaluated by the bootstrap method. It was judged that there is a clear difference in the estimated values as a group because the confidence interval does not intersect at all. The thresholds of 7% and 17% were adopted as values close to the quartile. We will mention these explanations as well in the paper.
Given the myriad of attributes in a country, it is not possible without a hypothesis to find the factors that contribute to the observed reduction in deaths. Thus, it is necessary to select a rational hypothesis and consider its superiority or inferiority, but since it constantly fluctuates depending on the spread of infection, it is unavoidable to make the best of the current available information.
I found a sentence in the Discussion that "machine learning analysis classified ..." and this was misleading. It should be corrected and at the same time, we would like to add an explanation in the appropriate place to clarify why we used machine learning.
Commenter:
The commenter has declared there is no conflict of interests.
The commenter has declared there is no conflict of interests.
(1) Sometimes I wish I could. However, you cannot claim causality out of correlation, no matter how strong it is.
A couple of examples: "cities with more trees are safer" does not mean trees make cities safer.
"Countries with more Mangos have more Malaria patients" does Not imply that cutting down mango trees will reduce Malaria patients. You need to exterminate Mosquitos. Not Mangos.
I wish More PCR tests indeed have a causal impact on the reduction of NPDD, however I cannot claim that just because countries with more PCR tests have lower NPDD.
(2) You will have to find a good exogenous variation in the NPDP, hopefully at the country level, but if it is difficult, at least at the community level, to avoid possible between-unit spillover effects. One possible, however, never-the-best way would be to start out with DID, or DID-PSM.
I wish you a fruitful research in your future!
Commenter: Yasuhiro Ishikawa
The commenter has declared there is no conflict of interests.
The rest seems to be inferred from the differential equations of a simple SIR model.
By the way, PCR is intentionally narrowed down, so it is said that it is not possible to predict the future because the rate of community transmission is unknown on TV.
Professor Yamanaka also says that a 40% PCR positive rate in Tokyo is dangerous. The rationale is not given.
Therefore, I estimated the rate of community-acquired infection in Tokyo.
In Bayesian statistics, the posterior probability changes depending on the prior probability. For Covid-19 infections, the PCR positive rate is the posterior probability. On the contrary, from the posterior probability (positive rate), try to predict the prior probability (community infection rate).
PCR test sensitivity Se = 0.7
Specificity Sp = 0.97
Posterior probability (positive rate) Po = 0.4 (PCR positive rate).
Obtain the prior probability (community infection rate) Pr.
Pr = Po * (1-Sp) / (Se * (1-Po) + Po * (1-Sp)) Pr (community infection rate) is the left formula.
The positive PCR test rate in Tokyo is 0.4.
Pr = 0.4 * (1-0.97) / (0.7 * (1-0.4) + 0.4 * (1-0.97))
= 0.028 When the PCR test positive rate is 40%, the local infection rate is 2.8%
If the population of Tokyo is 10 million, the number of infected people is 280,000.
(Currently, a little less than 4000 confirmed cases are infected by PCR.)
Even if the PCR test is negative, the probability of being infected with Covid-19 is 23.6%.
Even with PCR negatives, 1 in 4 is infected.
If the sensitivity of PCR test is a little worse and Se = 0.6,
When PCR test positive rate is 40%, Pr = 0.032, local infection rate is 3.2%
If the population of Tokyo is 10 million, the number of infected people will be 320,000.
It is said that 80% of asymptomatic infected carriers (carriers) and mild patients do not need elementary inhalation, but in Tokyo, 98% are carriers or mild patients, and further stay at home is required.
Here, the prior probability and community-acquired infection rate are used, but since PCR testing is the target, it is better to think of it as the probability of those who have a coronavirus. (April 29, 2nd year of Reiwa)
Addendum On May 5, 2012, Governor Yoshimura of Osaka Prefecture indicated a PCR test positive rate of 7% or less as one of the numerical criteria for canceling the emergency declaration.
If the population of Osaka Prefecture is 8.8 million, the prior probability is 0.0032, and 28,000 people or 0.32% will be the number of people who have coronavirus.
-------------- M.D. Yasuhiro Ishikawa
E-Mail: [email protected] [email protected]
WWW