A Machine Learning Algorithm Predicting Infant Psychomotor Developmental Delay Using Medical and Social Determinants

1. Introduction

Children’s progress in achieving developmental milestones in infancy and childhood is dependent on a large number of factors. These include growth in utero, size at birth, maternal health, socioeconomic position, genetically inherited developmental patterns, and many family and social factors [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]. This makes predicting developmental delay in advance so that steps can be taken to avoid it difficult, as there are so many potentially important causes and the relative importance of each is not clear. For an increasing number of health conditions with complex aetiologies, artificial intelligence (AI) has been successfully applied to identify when an individual is at high risk for a future adverse health outcome e.g., [18,19]. In the discipline of developmental psychology, the machine learning approach Random Forests (RF) has been applied to predict future psychiatric conditions [20] and to predict infant growth using inflammatory markers [21]. The present study applied RF to predict psychomotor developmental delay in 9-month-old infants using data on a wide array of factors in pregnancy, birth and early infancy. The intent was to achieve higher sensitivity and specificity than has been achieved in prior studies approaching similar problems using regression methods, which rarely have greater than 80% sensitivity [15].

Other than machine learning approaches, several statistical techniques are potentially appropriate for classification problems including predicting developmental delay. Van Dokkum et al. [15] used logistic regression to predict developmental delay at age four, producing an algorithm with 73% sensitivity and 80% specificity. Another promising linear modelling approach when there is a large number of predictor variables is principal component analysis (PCA). However, both statistical techniques assume linear relationships between values of the predictor variable and the outcome: PCA is based on linear transformation using orthogonal matrices, and logistic regression assumes that the log-odds of the relationship between each predictor and the outcome is linear. There is no reason to believe that predictors have linear associations with developmental delay: for example, birthweight has negative associations with developmental delay at both low and very high levels [15,22], and socioeconomic position may not be important for health outcomes above a threshold level [23]. For the present research, Random Forest (RF), which is an ensemble decision-tree classifier was chosen. RF can handle large numbers of predictors (features) simultaneously and does not assume linear or monotonous relationships between predictors and an outcome [24,25].

2. Methods

Population and sample

The UK Millennium cohort sample (henceforth MCS) consists of infants born in the United Kingdom from September 2000 to August 2001, identified using Universal Child Benefit records and NHS Health Visitors [26]. In the British healthcare system, Health Visitors are usually registered nurses who provide ante- and post-natal care and advice in the home. The sample was not a random sample: ethnic minority and low socio-economic groups were oversampled to compensate for loss to follow-up of these segments of the population that occurred in Britain’s earlier longitudinal cohort studies. Here, data were analysed using the first survey of the cohort, which took place when the infants were around 9 months old. The maximum possible sample size for analysis using this cohort is 18,467. A cohort profile is available providing far more detail about the sample and sampling methods [27].

Outcome variable

Developmental delay is typically identified in clinical settings using parental questionnaires. The 9-month MCS interview with parents or the main care giver included questions about infant psychomotor development which are very similar in content and format to the Ages and Stages 12-month questionnaire [26]. The aim in creating the dependent variable was to capture infant development across a number of cognitive and motor skill domains. Second, variation in reaching developmental milestones has the most practical or clinical significance if a statistical model is created to predict substantial delay versus the range of normal development. With these aims in mind, a dependent variable was created using parental or main caregiver reports of achievement of developmental milestones. The interview contained 12 questions on cognitive and motor skills development. Responses to the 12 questions were on three-point scales, coded as “1” for the infant frequently demonstrates the developmental milestone, “2” for sometimes, and “3” for the infant has not yet demonstrated the milestone. The 12 items were: sits up; smiles; stands up holding on, puts hands together; grabs objects; holds small objects; passes a toy; walks a few steps; gives toy; waves bye-bye, extends arms; nods for yes. The responses were summed into a single score, followed by splitting into a binary variable with the cut point at the fifth percentile.

Predictor variables (features)

The first MCS survey was broad in scope, covering aspects of pregnancy, labour, birth, and children’s and their parents’ social, work, and economic situations. Many of the variables included in the MCS have been demonstrated to be or could plausibly be associated with child development. Covariates were selected by reading through the MCS variable list and selecting all that appeared appropriate for analysis. The variable selection process is illustrated in Figure 1. Some additive combining of variables was performed where two or more variables were repeated information about a single concept. For example, paternal involvement in infant care was represented in the original data as questions about each individual act of care, such as nappy changing, getting up in the night, etc. These were additively combined to create a single variable. Of note, a decision was made to combine medical problems in pregnancy into a single variable. In descending order of their prevalence in the dataset the most common were: bleeding in pregnancy, eclampsia, hyperemesis, urinary tract infections, anaemia, and non-trivial infections. These were combined because conceptually they should all affect foetal nutrition, and because in initial testing of algorithms they performed poorly as predictors of developmental delay when included separately. Fifty-two variables were included in total. For ease of reading, variables were classified into groupings based on the concept that each represented: family & social support; socioeconomic indicators; infant characteristics; beliefs about parenting; medical circumstances in pregnancy and birth; maternal factors; and paternal and family factors. Supplementary material Table S1 includes details of variable coding, the MCS names and any changes made to the original MCS variables.

Data analysis

The MCS data were analysed using random forests (RF), a supervised machine learning decision tree algorithm easily implemented in statistical software such as Stata. In building each decision tree, the RF algorithm used half of the data (the training set), and with bootstrapping created sets of decision trees with the bootstrapped subsets of the data which comprise of a decision rule at each branch node. The remaining half of the data for each tree (the test set) was used to test how well the algorithm performed classifying observations correctly. Missing data occurred due to unanswered interview items on a small number of variables, particularly paternal support. The RF algorithm contained a proximity algorithm to handle missing observations for features. Observations with a missing value for the outcome variable were dropped from the analysis, and continuous predictors were transformed to z-scores.

All analyses were carried out in Stata 16. For the RF model, the plug-in Rforest was used [29]. Algorithm hypertuning of the number of variables included at each split and number of iterations were performed using Stata code developed by Schonlau and Zou [29]. A backward elimination wrapper method was applied to produce a reduced model which maximised number correctly classified using the fewest variables.

3. Results

Descriptive statistics

Descriptive statistics are displayed for all variables in Table 1, split into groups of variables as described above.

RF algorithms

The RF algorithm for all 52 predictors had an out of bag error rate of 0.0389. Hypertuned values for number of iterations and number of variables at each split were 25 and 14 respectively. Only 29 infants were classified incorrectly (0.16% of observations). The left side of Table 2 displays a classification table of correctly classified cases in the 52-predictor algorithm, and the right-side correct classification with the reduced model with the fewest incorrectly classified cases by exhaustive backwards selection. The reduced model was hypertuned to 80 iterations and 5 variables at each split. This algorithm incorrectly classified only 6 cases using five features: birthweight, the infant’s age in days at the time of the interview with the main caregiver, maternal and paternal ages, and equivalised income (McClement’s score, adjusted for household size and ages of children in the household). Out of bag error (oob) for this algorithm was 0.0393, or only marginally worse than for the 52-feature algorithm.

Figure 2 displays the variable importance plot for the 52-predictor RF algorithm. The variable with the highest feature importance score was birthweight, hence all of the importance scores are relative to the importance of birthweight for predicting being in the bottom 5% for development scores. It should be noted that the model did not include gestational age at birth, hence birthweight is almost certain to incorporate effects of premature delivery. This is desirable so that predictors are compared relative to a statistically (and clinically) very important variable. None of the variables had feature importance scores close to zero, meaning that all had some predictive utility in the RF algorithm.

Decision tree algorithms do not produce a statistic or parameter estimate showing the direction of association, as they are not linear models. To overcome this, two-way prediction plots are displayed for the reduced (5-variable) algorithm in Figure 3, and for all features in Supplementary material Figure S1. The plots shown are two-way prediction plots with either a Lowess smooth fit line, a quadratic fit line, or as a linear plot for binary predictors (whichever best described the observed relationship). The direction or shape of relationships between developmental delay and all predictors are described in writing in Figure 2.

4. Discussion

The RF machine learning approach allowed simultaneous analysis of a large number of maternal, paternal, social and health-related factors. The algorithm performed very well when applied to the test dataset, with sensitivity at the level of a good diagnostic medical test. The results were consistent with developmental delay having a complex aetiology: 45 variables had importance scores above 0.2. However, prediction measured as number of observations correctly classified by the algorithm was maximised using only five predictors, one of which was simply the infants exact age in days. The algorithm suggested that a typical developmentally delayed child is likely to have been born low birthweight to older parents. Household income had an important but nonlinear relationship with developmental delay. While low birthweight and socioeconomic position are well-established predictors of developmental delay, the importance of both maternal and paternal age was surprising. Maternal age has previously been found to have the opposite relationship to developmental delay to what was found here: there was a monotonic trend towards lower risk of delay beginning with the youngest mothers (see Figure 3). In prior research infants of teenage mothers had an increased risk of delay [13]. Prior research additionally highlighted the importance of maternal education [16,17]. Here, income had a higher importance score than maternal education.

Study limitations

A prospective longitudinal study design would be necessary to confirm algorithm performance in a clinical setting. Psychomotor delay in the MCS 9-month interview was measured using fewer items than are typically found in established scales such as Age and Stages. In addition, Ages and Stages and similar scales identify more infants as showing delay than the bottom 5% analysed in the current study. The same data quality issue applies more generally to most of the concepts in this analysis: national cohort study data allows for large analysis sample sizes and the potential for high statistical power, but this comes at a cost to the level of detail gathered about each concept: for example, family support variables were from interview rather than methods which directly measure social support. Methods that directly measure or change social support would be preferable.

Conclusion

RF can be easily implemented in statistical software such as Stata, as well as in open source software such as BlueSky Statistics. It is preferable to regression when there is a large number of potentially important predictors of an outcome. A disadvantage is that other than producing sensitivity and specificity values, the underlying concepts and results interpretation are not familiar to the majority of medical and social science researchers. The results of the RF modelling here showed remarkably high sensitivity and specificity which were far in excess of existing regression-based algorithms predicting developmental delay [15]. The features with the highest importance scores: birthweight, household income, maternal and paternal ages and duration of labour can all be discerned at birth. This implies that screening for developmental delay can be successfully implemented in the neonatal period. Features representing early infant environment and parenting all had lower importance scores. Maternal health problems during pregnancy, including eclampsia, bleeding and non-trivial infections also had lower importance scores than expected.