Comparing Self-Administered Web-Based to Interviewer-Led 24-Hour Dietary Recall: An Italian Pilot Case Study

1. Introduction

National food consumption surveys are crucial for monitoring the nutritional status of the population, defining nutrition policies, estimating dietary exposure, and and, in conjunction with environmental impact indicators, for exploring the environmental impacts associated with different dietary scenarios. In 2009, the European Food Safety Authority (EFSA) underlined the importance of these studies launching at the European level, the EU Menu project with the aim of making the collection of more harmonised food consumption data among the EU Member States to use them for dietary exposure assessments of food-borne hazards and nutrient intake estimations [1].

In Italy, national dietary surveys are conducted about every 10 years, and the fourth, IV SCAI study, has now been completed and was carried out following the above EU Menu methodology [2]. The first Italian survey on food consumption dates back to 1980-1984 [3]; subsequently, two other surveys were carried out respectively in 1994-96 [4] and 2005-06 (INRAN-SCAI) [5]. All four surveys used different tools and methods depending on both the type of information that was considered important at the time of the survey, the availability of effective and adequate tools for data collection and economic resource disposability.

According to the EU Menu guideline, the recommended technique to record the food consumed are face to face interview using the 24h recall for adolescent, adult and elderly population (10-74 years) and the dietary food records in the case of children up 10 years old. Moreover, a software validated is necessary to collect, record, manage and analyse the dietary data [1,2].

The 24-hour recall technique requires the involvement of dieticians or nutritionists or, at least, the well-trained personnel to carry out the interviews and to manage the specially developed software to entering the food items consumed. Typically, dietary surveys are conducted at national level and have duration covering the four seasons, and an adequate sample size is needed to estimate longer-term or usual intake, and multiple non-consecutive 24h recalls on the same individual are also necessary to capture daily variability [6,7,8]. In addition, the huge amount of time required for volunteers to record their food consumed and fill out questionnaires causes a high level of dropout, prolonging the recruitment phase of the survey. Therefore, it is important identify alternative methods that could tackle the challenges encountered by national dietary surveys [9]. The growing use of technology and web-based platforms can help to reduce the logistical burden and cost of conventional methods and maximizing the response rate compared to more traditional paper-based methods or interviewer-led survey [10,11].

In this context, the scientific literature has produced a large quantity of studies concerning software and training tools that can offer the possibility for the participants’ survey of recording a 24h recall autonomously, meaning, without the support of the trained staff such as ASA-24 in the United States [12], the Canadian R24W [13] and the UK Intake24 [11]. At the same time, another segment of the literature has focused on validation of self-administrated 24h recall with the interviewer-led methods, considering also different target of population [14,15,16,17,18]. Since, the self-administrated recall can be completed at a time and place convenient to the participant, without the need for a trained interviewer, this can decrease the respondent burden by reducing barriers to participation [19].

EFSA, in its updated EU MENU guidance, also underlines that conducting interviews in person has become less relevant, especially since the COVID 19 pandemic, as the use of videoconferences has been replaced by CATI (Computer Assisted Telephone Interview) and CAPI (Computer Assisted Personal Interviewing) methods. And it concludes that in the next round of EU MENU, will be possible to adopt a self-administered 24-hour dietary recall or smart-phone food record to implement the national dietary survey for the adult population [20].

FOODCONS is a web-based software and has been designed to collect food and nutrient intake data for the Italian population. It was developed by the Research Center for Food and Nutrition of Council for Agricultural Research and Economics, permitting the data entry with interviewer-led multiple pass 24h recalls according to the EU Menu guidelines [2]. Over the years, interviews and data-entry through the software were carried out by trained personnel with a nutritional background. In view of future technological changes in food consumption data collection tools, we wanted to check whether the current FOODCONS features could be suitable for autonomous use by individuals involved in nutritional studies. Therefore, the aim of this study is to undertake a comparison of FOODCONS self-administered 24h recall (the test method) with FOODCONS interviewer-led 24h recall (the reference method) in at least on 40 subjects aged 18-64 years, by examining food items, food group and nutrient intakes derived from both methods for collecting consumption data.

2. Materials and Methods

2.1. The Software FOODCONS

The software FOODCONS and all connected databases such as food composition, food nomenclatures, portion size data and food picture atlas, were developed by the Research Center for Food and Nutrition of Council for Agricultural Research and Economics (https: //www.crea.gov.it/en/web/alimenti-e-nutrizione) for use in nutritional and epidemiological studies.

The first software version dated back to 1999 and in 2007 it was developed to be continuously updated until 2014. It is in Italian language and has been used in several food consumption surveys conducted both by the Research Centre for Food and Nutrition and by other research institutions [5,21,22,24]. To date, it has been updated for online use ‘on stand-alone computer’ through a virtual machine (compatible on all platform Windows, MAC and Linus). The current and complete version includes two data entry modules: 24-hour recall and Food diary and one module for data management. The 24-hour recall module was built permitting the data entry with the Multiple-Pass Method according to the EU Menu guidelines [2]. It consists of five steps: (1) quick list, which is an uninterrupted listing by the subject of foods and beverages consumed; (2) the forgotten foods list, which queries the subject on categories of foods that have been documented as frequently forgotten; (3) the time and place at which foods were consumed; (4) followed by probing questions about quantities consumed and further information on the foods and drinks coded; and finally (5) reviewer of all the foods and drinks entered and, at last, the opportunity to add any forgotten items. [23]. The software is designed to guide the user through all the recall process. Anyway, participants do need some level of computer literacy and basic food knowledge. FOODCONS output provides other than the description of the food consumed and the amount in grams, the energy intake (EI), macronutrients (water, proteins, fat, saturated (SFA), monounsaturated (MUFA) and polyunsaturated fatty acids (PUFA), available starch and soluble carbohydrates (CHO), fibers, alcohol, cholesterol and minerals such as calcium (Ca), phosphorus (P), magnesium (Mg), potassium (K), iron (Fe), zinc (Zn) and vitamins such as vitamin C, thiamine, riboflavin, niacin, vitamin B₆, vitamin B₁₂, vitamin D vitamin E , vitamin K , retinol, β-carotene, vitamin A (expressed in retinol equivalents or REs) and dietary folate equivalents (DFE), natural folate and folic acid (from fortified foods and supplements). In addition, the output also provides the amount consumed at the level of food groups considering the categorization used in the consumption database of Italian population [5].

Protection of the data entered in FOODCONS software was guaranteed by daily backup and positioning the file in a password protected computer. Each fieldworker was committed to confidentiality and data protection is under the responsibility of CREA. Cryptography was applied to appropriate database fields.

2.2. Subjects’ Recruitment

The recruitment of a convenience sample was conducted by sending an invitation letter to the administrative staff of Council for Agricultural Research and Economics and National Institute of Geophysics and Volcanology, explaining the activities and purpose of the pilot study and an estimation of the time needed to complete it. Those who expressed an interest in participating in the study were screened for eligibility. The inclusion criteria were adults aged 18-64 years, which had regular access to the Internet. The exclusion criteria were pregnancy or breastfeeding, having health condition requiring nutritional or medical treatment, having academic or professional background in food and/or nutrition.

The study protocol was drawn up based on previous similar studies [10,11,18] and it was approved by the ethical committee LAZIO 2 of the ASL ROMA2 located in Rome, Italy (Studio 115.22), and all participants signed the informed consent prior to being enrolled in the study.

2.3. Study Design

Data collection took place between January and March 2023. For the two non-consecutive days, the participants had to complete both a self-administered and an interviewer-led 24h recall using FOODCONS software as a data entry, for each day. The two study days included at least one weekend day. To investigate the impact of order of administration, the study design requires that 75% of participants complete the web-based self-administered 24h recall as first, and approximately 3 hours after the first one, the interviewer-led 24h recall. The remaining 25% will complete the two recalls in the opposite order. About two weeks later, the participants recorded the consumption of two more recall days using the same methods, but in the opposite order (Figure 1) [10]. The 24-hour recall interview was conducted using the FOODCONS software as CAPI method. During data entry, each food, recipe and beverage consumed could be automatically described, retrieved and quantified using food atlas or selecting the photo of the portion available in the data entry software. The interviewer asked the respondent to recall the food and drink consumed on the previous day, including the type of meal, the time and place of consumption, the name of the food, whether it was a recipe or not, and the amount consumed. The interviewer-led recalls were all conducted by the experts of the CREA, face to face or online (on videoconferencing platforms, such as Microsoft Teams), after no less than 3 hours for those completing self-administered 24h recall as first one

For the self-administered 24h recall, participants entered all foods and drinks consumed the previous day into the FOODCONS software; once the food was selected, the participant estimated the food quantity consumed with the same visual aids; the software automatically coded the food consumed and assigned the nutritional composition.

Previously, all participants signed the informed consent and received two ad hoc video tutorials containing instructions, lasting approximately 40 minutes. The first one explained, in as much detail as possible, the 24h recall method and the critical aspects related to describing foods, linking foods to those in the software databases and the quantification of the consumption. The second video taught how to use the FOODCONS software to enter own consumption data.

After completing the study, volunteers filled out a short anonymous online questionnaire to evaluate the tool used for both methods tested (self-administered and led).

2.4. Statistical Analysis

Descriptive statistics mean, standard deviation (SD), median and Interquartile Range (IR) were used to summarize the sample in terms of daily energy and a selection of nutrient intakes. Mann-Whitney U test was performed to identify significant differences between estimates. Spearman’s rank order correlation was also calculated to assess the relationship between estimates of nutrient intake between the self-administered and the interviewer-led recall. A p-value < 0.05 was considered statistically significant. Bland and Altman analysis was used to plot the agreement of nutrient intake, considering the standard error s of the mean difference d of the two methods and d±1.96s as the 95% CI of agreement limits. Match, omission, and intrusion rates were also calculated to assess the agreement of the self-administered dietary recall with the interviewer-led recall. All statistical analyses were conducted with the software SAS version 9.4 (SAS Institute Inc., NC, USA) an R for the Blant Altman Plot.

3. Results

The sample was recruited on a voluntary basis between March and May 2023; the total number of subjects was 41 (each subject filled in four 24h dietary recalls). The mean age was 51 years and the 66% were women. The 24-hour recall of two subjects was excluded from the analysis because they did not complete the expected 5 steps. One day's recall was rejected because the subjects replaced the amount in grams to the amount in servings and reported unrealistic energy values such as 213880 and 34567 kcal respectively. The analyses were carried out on 39 participants who completed a total of n.156 recall interviews.

The difference in the two-day mean of energy and macro and micronutrients intakes between the two methods was not statistically significant, except for Linolenic acid (Table 1).

Mean intakes reported with the self-administered method were very close to the intakes reported during the interviewer-led recall for energy and the macronutrients. For energy all participants are within the limit of agreement; for carbohydrates just one is lied out of lower limit whilst for the dietary fibre two are out of upper limit; the worst case is the mean distribution of the protein intake that shows as 8% of the volunteers is out of upper limit.

Figure 2. Bland-Altman plots represent the agreement between the average of nutrient intakes measured from the self-administered and the interviewer-led recall. The solid line represents the average difference between the two methods used (self-administered and the interviewer-led recall), while the dashed line represents the distance between the limits of agreement (± 2SD). (a) energy intake (kcal); (b) protein intake (g); (c) fat intake (g); (d) carbohydrates intake (g); (e) dietary fiber intake (g); (f) alcohol intake (g)

Figure 2. Bland-Altman plots represent the agreement between the average of nutrient intakes measured from the self-administered and the interviewer-led recall. The solid line represents the average difference between the two methods used (self-administered and the interviewer-led recall), while the dashed line represents the distance between the limits of agreement (± 2SD). (a) energy intake (kcal); (b) protein intake (g); (c) fat intake (g); (d) carbohydrates intake (g); (e) dietary fiber intake (g); (f) alcohol intake (g)

At level of food groups, the correlation coefficients indicate a good concordance between the two methods, except for meat and pulses (Table 2) see also Supplementary materials.

It should be noted that the mean difference is positive for most of the food groups, indicating that the amount of food consumed in the self-administrated 24h recall is greater than that reported in the interviewer-led mode. At the level of registered foods, the exact or approximate matches were measured; moreover, foods omitted and added in the self-administered 24-hour recall vs interview were also evaluated. The results of this analysis are presented as a percentage based on the total number of foods consumed by the participants, including omitted and added foods. Out of the foods consumed 73.5% agreed between the two modes, of these 16.8% agreed only approximately (i.e. foods that differed slightly in the case of the self-administered recall compared to the one carried out by interview, such as semi-skimmed milk and skimmed milk); the foods omitted during the interview mode were 15.7% of the total foods while the percentage of those added in the self-administered recording was 10.8%. However, an improvement was noted on the second day of the survey. In fact, there was an increase in the matched foods (77.2%) and a decrease in the omitted foods (11.3%) (Table 3).

All participants completed the evaluation questionnaire, also those excluded from the analysis of results. The questionnaire was completed online and anonymously at the end of the second day of the study, so it was not possible to identify the two excluded subjects. The volunteers rated the software and the difficulty of entering the 24-h dietary recalls for both modalities. 17 out of 41 respondents indicated an average execution time of 30-45 minutes (both modalities), and only 6 individuals took more than an hour with the self-administered 24-h recall. The instructions received were adequate for understanding the required information for 95% of the sample, and most participants had no difficulty using the food atlas to identify the portion of food consumed (80%). The 93% of volunteers indicated that the software interface facilitate the data entry. Overall, most respondents considered FOODCONS suitable for use in research projects, although 66% rated the interviewer-led 24h recall as more suitable for recording data on food consumption (Table 4).

4. Discussion

This pilot study evaluates the agreement between two modes of food consumption data collection (administered and self-reported), using the software (FOODCONS) previously used in several nutritional studies as a Computer-Assisted Personal Interview (CAPI) with an interviewer.

The comparison of energy and macro and micronutrients intake is good, more precisely there is not significance difference between the means of the two methods and the level of agreement is satisfactory. The number of individuals out of the interval of acceptability in the Blant Altmann plot is low but at the same time such interval is large mainly due to the difference in the amount reported as underlined also in the comparison of mean intake of food groups.

In fact, with the exception of Non alcoholic beverages, all food groups show a higher mean intake in the self-administered than in the interviewer-led method. The lowest correlation was found in the Pulses and Meat Products and Substitutes groups.

Other studies looking at the comparability of web-based recall tools and interviewer-led recalls have reported estimated lower intakes of energy and some nutrients compared to the interviewer-led 24h recall, with energy, total fat, and monounsaturated fats significantly different between both methods; this was the case of Foodbook24 study [10]. INTAKE24 study [18] provided estimates of energy intake that were 3% lower on average than the interviewer-led recall for the younger age group, with the limits of agreement ranging from minus 48% to plus 82%. Mean intakes of all macronutrients and micronutrients were within 10% of the interviewer-led recall. For the older age group, estimates of energy intake were in agreement on average for both methods, with limits of agreement ranging from minus 50% to plus 98%. Mean intakes of all macronutrients and micronutrients were within 3% of the interviewer-led recall. In Myfood24 [25] also underestimated energy intakes by 3% when compared with interviewer-led recalls in 11–18 years old, with the limits of agreement ranging from an underestimation of 39% to an overestimation of 34%.

Another study involving adolescents aged 12-17 years assessed the relative validity of the self-administered web-based 24-h dietary recall (the R24W) for evaluating energy and nutrient intakes, respect to the interviewer-led 24-d recall following the Automated Multipass Method [26]. Mean energy intake from the self-administered mode was significantly higher than from the interview-administered 24-h dietary recall. Significant differences in mean nutrient intake between the R24W and the interview-administered mode ranged from 6.5% for % E from fat (p<0.05) to 25.2% for saturated fat (p<0.001), i.e., higher values with R24W.

Several factors may explain our results. Firstly, it should be born in mind, that the description of the food items in the food list also reflects the various declinations of the food, since they are different in nutrient composition even though the name is almost the same but with specific details that define its quality.

This is the case with cow's skim milk, which is different in fat content from cow's whole milk and completely different in nutrients from oat milk; the same rule applies to yogurt as well. Therefore, careful reading of the definition of food is necessary to select the right one in the food list. Some volunteers ignored this warning with the consequence that the food intake was far from the "true" value. Secondly, the food items have different typologies, for example the biscuits could be with chocolate or without and the portion change, this affects the amount and the nutritional intake. Thirdly, some participants confuse the number of portions with the amount causing an abnormal value of food intake. This is a limitation of the software since no alert is present to advise the unrealistic number of portions. Anyway, most participants (93%) found the software interface easy to use and a good proportion (73%) found it easy or very easy to self-administer the 24h recall, 66% considered the interviewer-led mode more suitable than the self-administered one. Identification of the most similar food to the food consumed and the breakdown of composite foods into individual ingredients, were the most common problems encountered during self-administration. One of the main difficulties faced was navigating the drop-down menu to search for the food consumed from the food database, probably due to the large number of food items available, about 1200. This often led to discrepancies in the quantification of consumption. The software food database offers also composite dishes, many of which are typical Italian recipes. However, because of the above difficulty of searching for these from the software interface, when self-reporting the participants often choose to break the dish down into its individual ingredients and enter the quantities at ingredient level, which led to an overestimation of the total quantities consumed. In view of this, the activity of searching for foods by recalling them from the database is definitely an aspect to improve and deepen in the video tutorials teaching for self-administering 24h dietary recall, enriching them with more examples and clues to facilitate the search. A small percentage of users (22%) indicated a lack of clarity about how to correct the information recorded, and about the same percentage expressed an overall neutral opinion about the use of the software (neither satisfied nor dissatisfied). This also suggests the need to improve the learning tools developed for the use of the software.

The limitations of this study are both the low sample size and the social characteristics of the participants, who have medium to high levels of education and, above all, good computer skills. It is necessary to conduct more case studies, expanding the range of participants by including people with different computer skills and sociocultural backgrounds, as well as other age groups such as the elderly and adolescents. This would contribute to better understanding what aspects of the software need to be improved to maintain its specificity and potential while making it more user-friendly.

Even though this technological innovation has already been underway for decades. Most of the evaluated and validated tools are self-administered variations on the conventional dietary assessment methods, such as online 24-hour dietary recalls and smart-phone food records [27].

In a recent report [20], EFSA pointed out the gradual spread of tools for online 24-hour dietary recalls. In a 2021 review, two image-assisted 24-hour dietary recall tools were identified that were applied on a large scale, i.e. ASA24 (Automated self-administered 24-hour dietary assessment tool) and the CAAFE tool (Food Intake and Physical Activity of Schoolchildren tool). Recently, online, and self-administered tools were introduced for national dietary surveys in a few countries. The UK and Sweden use self-administered 24-hour dietary recalls and Denmark an online food record using Intake24 [28], RiksmatenFlex [29], and WebDASC [30], respectively). France also decided to use Intake24 for future surveys and a pilot study is planned

5. Conclusions

The results obtained demonstrate that the self-administrated 24h recall could be a good alternative to face-to-face interviews for recording data on foods consumed and this would allow a higher participation rate in food consumption studies.

Other studies are needed for filling the gaps highlighted in the association between food consumed and food items listed in the software. In addition, a protocol validation must be developed to implement this procedure in the national food consumption survey.