Disconnect between published ac magnetic susceptibility of a room temperature superconductor and measured raw data

In ref. [1], we pointed out that certain anomalies observed in the published data for ac magnetic susceptibility of a room temperature superconductor reported in Nature 586, 373 (2020) [2] would be cleared up once the measured raw data were made available. Part of the measured raw data were recently posted in arXiv:2111.15017 [3]. Here we report the results of our analysis of these raw data and our conclusion that they are incompatible with the published data. Implications of these results to the claim that the material is a room temperature superconductor are discussed.


I. INTRODUCTION
On October 14, 2020, Snider et al reported the discovery of the first room temperature superconductor, carbonaceous sulfur hydride, hereafter called CSH [2]. If this is true, it represents a major scientific breakthrough. "A superior test of superconductivity" [2] demonstrating superconductivity was claimed to be the detection of sharp drops in the ac magnetic susceptibility. Figure  1 shows the results published in that paper in Figs. 2a and Extended Data Fig. 7d giving susceptibility versus temperature for 5 different pressures.
The curves shown in Fig. 1 were obtained from the subtraction of two independent measurements, namely "raw data" and "background signal", according to the equation data = raw data − background signal. (1) According to the caption of Fig. 2a of [2], "The background signal, determined from a non-superconducting C-S-H sample at 108 GPa, has been subtracted from the data". Neither of these independent measurements were given in the paper nor in the supplemental material for the 5 pressures shown in Fig. 1.
In addition, the inset of Extended Data Fig. 7d, shown in the inset in the right panel of Fig. 1, reportedly presented "raw data" for still another value of the pressure, 138 GPa, according to the caption of the figure [2].
For more than one year, starting on November 12, 2020, we have attempted to obtain the raw data and background signal that were used to obtain the measurements shown in Fig. 1 from the corresponding author and coauthors. Details of this saga are described in ref. [1]. Finally, on December 1, 2021, part of those data, namely the measured raw data for the three curves shown on the left panel of Fig. 1, as well as for the inset on the right panel of Fig. 1, were made public in ref. [3] by two of the authors of ref. [2]. Neither the background signal data for the 5 curves shown in Fig. 1, nor the raw data for the two curves shown on the right panel of Fig. 1, that we also requested more than a year ago, have been made available yet. respectively. The inset in the right panel shows raw data reported in ref. [2] for another pressure value. The lower left and right panels show enlarged versions of the upper panels, enclosed by small rectangles in the upper panels. They will be discussed later in the text.
Nevertheless, given the raw data and the data, we can extract the background signal from the relation background signal = raw data − data.
raised in that paper and in an earlier paper [4] about the validity of the magnetic susceptibility measurements reported in ref. [2] would find answers once the authors released the underlying data. In this paper we report our analysis of the data reported in ref. [3] and the conclusions that this analysis leads to.

II. COINCIDENCES IN THE PUBLISHED DATA
In ref. [1] we analyzed the vector graphics image embedded in the published figure 2a of ref. [2] to extract the data points used to produce the figure to 6 digits accuracy. Fig. 3 show graphs of the obtained figures, both with the low resolution shown in Fig. 1 above in the left panel and with higher resolution in the right panel. From these data we learned [1] that there were strange coincidences in the data, where numerical values for susceptibility at widely different temperature coincided to 6 digits accuracy. Some of those coincidences are shown as the red lines on the right panels of Fig. 4.
The authors of ref. [3] countered our analysis by claiming that in fact the image in fig. 2a of their paper [2] "is a raster image and not a vector image. There is not infinite precision and scalability in this form of image, relying on discretization of the image along pixels to assign values, which leads to a limited precision for the values extracted. Analysis of such figures is certainly not accurate to 6 decimal places." Their statement is incorrect. The on-line version of their paper [2] has a vector image for Fig. 2a, and a raster image for Extended Fig. 7d. This can be clearly seen in the lower two panels of Fig. 1, where we expanded portions of the upper panels (enclosed in small rectangles) by 6400%. The reader can clearly see that the left panel remains sharp and the right panel becomes blurred. That is a hallmark of vector versus raster images. We have verified that fig. 2a in their paper [2] !"#$%&'$ χ($)*+,  [2], discussed in ref. [1]. The left panels show the curves with the resolution given in the published fig. 2a of [2], as shown here in Fig. 1 top left panel, the right panels show the same curves with higher resolution. The regions enclosed in the rectangles for 166 GPa and 189 GPa are shown with even higher resolution in Fig. 4.
(left panel of fig. 1 above) is a vector image by analyzing it with two different software packages, inkscape and adobe illustrator, and getting identical answers to 6 digit accuracy.
On the left panels of Fig. 4 we show what the authors claim are the real numerical values [3] for some of the points where we claimed in ref. [1] that there are 6 digits coincidences: they claim [3] that the coincidences are only two digits, with the numbers given on the left panels of Fig. 4. This is not so.
Be that as it may however, it is not important for the purposes of this paper. What is important is that by publishing the left panels of Fig. 4  The red lines show anomalous coincidences discussed in ref. [1]. The left panels show the corresponding portions of the curves claimed to be true by the authors in ref. [3]. Note that the shapes of the curves on the left and right panels are identical, the claimed disagreement is in the third significant figure.

in ref. [3] the authors
confirm that our results [1] for the fine structure of their curves published in Fig. 2a of [2] are indeed correct to at least two digits accuracy. That is sufficient to establish the points made in this paper discussed in the following sections.

III. THE RAW DATA AND THE BACKGROUND SIGNAL
The authors of ref. [2] have not released the background signal used to obtain the data shown in their Fig.  2a of ref. [2] according to Eq. (1), but they have released the numerical values for the measured raw data to at least 6 digits accuracy in ref. [3]. Using those raw data, and the numerical values for the published data of Fig.  2a of ref. [2] obtained from the vector graphics image, we can infer the background signal from Eq. (2) and expect the background signal thus obtained to be accurate to 6 digits.
Using this procedure, we plot the raw data from ref. [3], the data published in fig. 2a of ref. [2], and the background signal from Eq. (2), for the three values of the pressure 166, 178 and 189 GPa, in Figs. 5, 6 and 7.
The scale on the vertical axis of Figs. 5, 6 and 7 give the susceptibility in nV as given by the raw data of [3]. The insets in Figs. 5, 6 and 7 show portions of the raw data and background signal amplified, for the regions enclosed by rectangles in the main panels.
What should be apparent to the reader from just looking at the main panels, and becomes even more apparent  [2], and the red points are the background signal inferred from the raw data and published data according to Eq. (2). The insets show the regions of the curves enclosed in rectangles in the main panel, with the red points shifted uniformy down to be close to the black points so as to allow comparison of the noise structure. In the upper part of the curves the near identity of the noise structure in the raw data and background signal is apparent in the main panel, so we have not included an amplified inset for that region. when looking also at the amplified insets, is that the random noise structure in the raw data and the background signal is nearly identical for all cases shown.
Finally, we show in Fig. 8 comparison of the noise structures for the regions that we focused on in our Physica C paper [1], that the authors of ref. [3] also discussed in their figures 8 and 9 of [3], reproduced in the right and left panels of Fig. 4 respectively. For these regions, similarly to the other cases, a high degree of coincidence exists in the noise structures of raw data and background signal, as seen in Fig. 8. Given that the authors have confirmed in their paper [3] that our curves on the left  Fig. 4 for pressures 166 GPa and 189 GPa, the right panels show comparison of the noise structure for raw data (black dots) and background signal (red dots). The left panels show the data in that region, from our Physica C paper [1]. panels of Fig. 8 are accurate to at least 2 significant digits (see Fig. 4 in this paper), it is significant that the noise structure is very similar on a much larger scale on the right panels of Fig. 8.
To assess the significance of these results, it should be remembered that the background signal, that has not been released by the authors of [2], was reportedly obtained from independent measurements at a much lower pressure, 108 GPa. What is the probability that the noise structure obtained in those independent measurements is identical to the noise found in the measurements of the raw data at pressures 166 GPa, 178 GPa, and 189 GPa, as shown in figs. 5-8? The reader can make his/her own best estimate.
In our view, it is impossible that the raw data and background signal have the similar noise structure shown in Figs. 5-8 resulting from independent measurements. This then leads us to consider the following possible explanations for this conundrum: (1) The raw data reported in [3] are not the raw data corresponding to the published data in ref. [2]. If that was the case, our procedure for obtaining the background signal Eq. (2) would be invalidated. However, the authors of ref. [3] claim they are. In addition, it can be seen in Figs. 5-7 that in the region of the transitions there is a good coincidence between the published data (green curves) and raw data (black curves). Therefore, we have to discard this explanation.
(2) The published data in ref. [2] resulted from a smoothing procedure performed on the difference between measured raw data and measured background signal that were independently noisy, that eliminated the noise structure. If so, in obtaining the background signal from the subtraction in Eq. (2), we would artificially introduce the raw data noise into the extracted background signal. However it would not make sense to smoothen the data to eliminate noise in the raw data of the magnitude shown in the figures and at the same time retain the fine structure in the data displayed in fig. 4, that the authors themselves acknowledge [3] exists in the published data [2]. Therefore, we have to discard this explanation.
(3) Either the raw data given in ref. [3], or the published data in fig. 2a of ref. [2], or both, do not display the reality of what the papers claim they display.

IV. FURTHER COMPARISON OF RAW DATA AND PUBLISHED DATA
From the analysis of the previous section we concluded that there is an unexpected disconnect between the published data for magnetic susceptibility in ref. [2] and the raw data for the same measurements posted in ref. [3]. Our analysis in the previous section relied on extracting the background signal. Here we do further comparison between raw data and published data without relying on an inferred background signal.
In fig. 9 we plot the temperature increments between subsequent points, ∆T , for the raw data given in [3] and the corresponding published data from [2], versus temperature. It is apparent that they are substantially different for all cases, with the published data showing significantly more values of ∆T than the raw data, and values that don't coincide with any of the values seen for the raw data. This of course also implies that the temperature values given in the raw data and in the published data don't coincide. It is difficult to understand why the published data would show measurements at different temperatures than the measured raw data.
In fig. 10 we plot susceptibility increments ∆χ between subsequent temperature points for the raw data given in [3] and the corresponding published data from [2], versus temperature. It is apparent that they are substantially different. Namely, away from the transition region the published data show significantly smaller ∆χ increments than the raw data. It is impossible to understand why the published data would show increments in susceptibility measurements for neighboring temperature values so substantially smaller than seen in the raw data.
In fig. 11 we show the data of fig. 10 for a small range of susceptibility increments around zero. The qualitatively different nature of the published and raw data is apparent. The published data show a quantification of the susceptibility increments in discrete steps. A closer look shows that in fact there are steps and much smaller substeps on the right panels of fig. 10. Instead, for the left panels of fig. 11, i.e. the raw data, no quantification whatsoever is apparent.
We have recently learned [5] that the quantification of measured voltages shown on the right panel of fig. 11 is in fact expected when measurements are obtained using a digital lock-in amplifier, according to information given to us by S. Weir. However the most common situation is to have a single step and a single substep [5], while an analysis of the published data shows a more complicated step structure.
Be that as it may, the important point for the purposes of this paper is that the increments in temperature and susceptibility shown in figs. 9-11 exhibit a complete disconnect between what is published in ref. [2] and the raw data reported in ref. [3]. In particular, it is impossible that measured raw data don't show steps in ∆χ, as seen on the left panels of Fig. 11, yet the data obtained from them and an independently measured background signal using Eq. (1) would show the steps seen on the right panels of Fig. 11.
In Appendix A, we discussed other points addressed in the recently posted paper ref. [3].

V. DISCUSSION
For over one year we have been waiting [1] to receive the underlying data associated with the published susceptibility curves in ref. [2], and for an explanation of the anomalous change in slope in the raw data susceptibility curve shown in the inset of Fig. 1 [4]. Now we have finally been provided with some answers to these questions in the paper [3] recently posted in arxiv. Unfortunately, the answers provided do not answer the questions and instead raise additional troubling questions.
As discussed in Appendix A, no physical explanation was provided in [3] for the anomalous rise of χ below the jump in the raw data shown in the inset of Fig. 1 noted in refs. [1,4]. None of the references cited in [3] (refs. 20, 21 and 22 in [3]), claimed in [3] to show such anomalous rise, do in fact show such behavior within a range of less than 2% of the presumed critical temperature as the inset of Fig. 1 shows, as readers can easily verify by reading those references. Nor has it been explained why the anomalous raw data of the inset of Fig. 1 were chosen to be shown in ref. [2] instead of any of the typical raw data shown in Fig. 2 that don't show such an anomalous rise.
More troubling is the fact that the raw data provided in ref. [3] that purportedly underly the published susceptibility data in ref. [2] exhibit a complete disconnect with the published curves they are supposed to represent. In sections II and III of this paper we have provided extensive conclusive evidence that the raw data presented in [3] cannot possibly give rise to the numerical values for susceptibility that we extracted from the vector graphics figure 2a published in ref. [2].
The authors of [3] have disputed our claim [1] that the published Fig. 2a of their paper [2] is a vector image that allows for extraction of the numerical values used to create the figure to 6 digits accuracy. Knowledgeable readers will know to check this out for themselves and understand who is right and who is wrong in this respect. But even if one were to accept the authors' claim that our numerical values are not accurate to 6 digits accuracy, the authors themselves have accepted that they are correct to at least 2 digit accuracy, as comparison of the left and right panels of fig. 4 shows, at the very least for the temperature ranges shown in fig. 4. Given that fact, the fact that in the lower right panel of fig. 8 the noise of the background and the signal closely track each other in the same temperature range provides incontrovertible proof that the background signal and raw data cannot have originated in independent physical measurements of susceptibility at pressure values 189 GPa and 108 GPa, as references [2] and [3] claim.
Furthermore, the presence of steps in the ∆χ incre-ments of the published data seen on the right panels of Fig. 11, together with the absence of such steps in the purportedly measured raw data seen on the left panels, also provide incontrovertible evidence that the published data could not have originated from those raw data and an independently measured background, with or without steps. Examination of the susceptiblity increments ∆χ for the background signals that we obtained through Eq.
(2) shows (as expected) exactly the same pattern shown on the left panel of Fig. 11 for the raw data, namely no step pattern at all. This suggests consideration of the following equation that follows from Eq. (1) or Eq. (2): In words, that a physically reasonable approximately linear background signal with random noise, as given by the red curves in Figs. 5 to 7, added to the data published in Fig. 2a of ref. [2], would give rise to 'raw data' identical to the black curves shown in Figs. 5 to 7, obtained from ref. [3]. It would of course be a misnomer to call such numbers obtained through Eq. (3) 'raw data'.
In conclusion, we have shown in this paper that the ac magnetic susceptibility curves reported in Fig. 2a of ref. [2] claimed to provide a "superior test of superconductivity" of CSH cannot have been obtained from Eq. (1) with the raw data published by two authors of ref. [2] in ref. [3]. Consequently, those susceptibility curves provide no evidence for the claimed room temperature superconductivity of CSH [2], since they are unsupported by valid raw data. Why the authors of ref. [3] chose to present as raw data underlying Fig. 2a of ref. [2] numbers [3] that are not consistent with the published data in Fig. 2a is a question for each reader to answer for themselves. Figure 4 of [3] compares the raw data for CSH shown in the inset of fig. 1 top right panel of this paper with similar-looking data for susceptibility of europium metal reported in ref. [6]. Such a comparison was made by us in refs. [4] and [1]. Ref. [3] states "Remarkably, the measured signal strength is different in two samples, indicating different sample sizes." Indeed, fig. 4 of [3] shows a susceptibility jump of approximately -20nV for CSH and -40nV for Eu. It should be pointed out however that the susceptibility jump published in ref. [6] for exactly the same case showed a jump of -20nV and not -40nV. Ref. [3] makes the cryptic statement in the caption of fig. 4 "Note that drop in signal in Eu is ∼ -40nV as observed before scaling due to different warming rates.'' There is however no statement in ref. [6] indicating that the warming rate would require scaling of the signal, quite the contrary, ref. [6] only stated that "the observed ∆χ ∼ 20nV jump at T c is consistent with perfect diamagnetism, the hallmark of a superconductor." There is in fact no physical reason for why the warming rate would cause a susceptibility drop due to onset of superconductivity to change by a factor of 2.
Ref. [3] then shows in fig. 7 two plots of susceptibility of CSH at 138 GPa. The bottom curve is the same as the inset of fig. 1 top right panel, which the caption of Extended Data fig. 7d in ref. [2] says are "raw data". Yet fig. 7 of [3] shows a curve at the top of the figure which it calls "raw data" in the caption, and explains that the bottom curve is obtained after subtraction of a linear background. This is in contradiction with the figure caption in ref. [2], which now we are supposed to assume are not raw data but instead data obtained after subtraction of a linear background? So it is not clear what those data are, nor is it clear why they show a large change in slope between above and below T c , an anomaly first noted in refs. [4] and [1]. It should also be pointed out that, as ref. [6] explains in connection with the Eu curve shown in fig. 4 of [3], "The inset shows the raw data at 118 GPa before this background subtraction." Quite generally, "raw data" are understood to mean data measured before background subtraction, in contradiction to the figure caption of fig. 7 in [3]. If the inset of Extended Data fig. 7d of ref. [2] had background subtracted as ref. [3] states, it is perplexing that it looks qualitatively different from the other 5 curves for susceptibility shown in fig. 1 that reportedly also had background subtracted.
Note also that the raw data for pressures 166 GPa, 178 GPa and 189 GPa shown on the left panel of Fig. 2 (graphed from the data tables provided in ref. [3]) look typical and qualitatively different from the so-called raw data for pressure 138 GPa shown in the inset of Fig. 1. Given those typical raw data it is incomprehensible that the authors would have chosen to show in ref. [2] the highly atypical raw data shown in the inset of Fig. 1.
Continuing with ref. [3], it shows in figure 5 a curve for "AC susceptibility of a sample which superconducts at 235K", that shows a drop in susceptibility of approximately 60nV, which is four to eight times larger than the drops shown in fig. 1. Since the magnitude of the jump attributed to superconductivity is supposed to be proportional to the volume of the sample [7], it is perplexing that the same sample would show jumps that differ by factors of up to eight as these results would indicate.
Finally, the explanation given in ref. [3] for the process of background subtraction associated with fig. 6 of ref. [3] does not provide a physical explanation for the large change in slope from above to below the jump in susceptibility shown in the inset of fig. 1, the anomalous behavior that raised our concern with these measurements originally [4].