JMIR Publications

ABSTRACT

Background:

Fetal Alcohol Syndrome (FAS) is a lifelong developmental disability which occurs among individuals with prenatal alcohol exposure. With improved prediction models, FAS can be diagnosed/treated early, if not completely prevented.

Objective:

In this study, we sought to compare different machine learning algorithms and their FAS predictive performance among women who consumed alcohol during pregnancy. We also aimed to identify which variables (e.g., timing of exposure to alcohol during pregnancy, type of alcohol consumed, etc.) were most influential in generating an accurate model.

Methods:

Data from the Collaborative Initiative on Fetal Alcohol Spectrum Disorders (CIFASD) from 2007-2017 were used to gather information about 595 women who consumed alcohol during pregnancy at five hospital sites around the United States. To obtain information about prenatal alcohol exposure, questionnaires/in-person interviews as well as reviews of medical, legal, and/or social service records regarding alcohol-related problems/a diagnosis of alcohol abuse were employed to gather information about alcohol consumption. Four different machine learning algorithms (logistic regression, XGBoost, light GBM, and CatBoost) were trained to predict the prevalence of FAS at birth, and model performance was measured by analyzing the area under the receiver operating characteristics curve (AUROC). 80% of total cases were randomly selected for training, while 20% remained as test datasets for predicting FAS. Feature importance was also analyzed using Shapley Values for the best-performing algorithm.

Results:

Overall, there were 20 cases of FAS within a total population of 595 individuals with prenatal alcohol exposure. Most of the drinking occurred in the 1st trimester only (n=491) or throughout all three trimesters (n=95); however, there were also reports of drinking in the 1st and 2nd trimesters only (n=8), and one case of drinking in the third trimester only (n=1). The CatBoost method delivered the best performance in terms of AUROC (0.92) and AUPRC (0.51), followed by the logistic regression method (AUROC: 0.90; AUPRC: 0.59), light GBM (AUROC: 0.89; AUPRC: 0.52), and XGBoost (AUROC: 0.86; AURPC: 0.45). Shapley values in the CatBoost model revealed that 12 variables were considered important in FAS prediction, with drinking throughout all three trimesters of pregnancy, maternal age, race, and type of alcoholic beverage consumed (e.g., beer, wine, or liquor) scoring highly in overall feature importance. For most predictive measures, the best performance was obtained by the CatBoost algorithm, with an AUROC of 0.92, precision of 0.50, specificity of 0.29, F1-score of 0.29, and accuracy of 0.96.

Conclusions:

Machine learning algorithms were able to identify FAS risk with a prediction performance higher than that of previous models, among pregnant drinkers. For small training sets which are common with FAS, boosting mechanisms like CatBoost may formulate discriminative classifiers for unclear optimality criterion; helping physicians identify high-risk populations and predict/prevent FAS in a timely manner. Clinical Trial: Institutional Review Boards (IRBs) at all CIFASD sites approved this study, and the Harvard T.H. Chan School of Public Health IRB approved analyses of these secondary data (Protocol #: IRB21-1261).