JMIR Publications

Background:

Blindness or vision impairment, one of the top ten disabilities among men and women, targets more than 7 million Americans of almost all ages. Accessible visual information is of paramount importance to improve independence and safety of blind and visually-impaired people, and there is a pressing need to develop smart automated systems to assist their navigation, particularly in unfamiliar healthcare environments, such as clinics, hospitals, and urgent cares.

Objective:

The main objective of the current contribution was to develop computational vision algorithms composed with deep neural network to assist visually impaired individual's mobility in clinical environments by accurately detecting doors, stairs, and signages, the most remarkable landmarks.

Methods:

A dataset of indoor landmarks of interests was collected from Marshfield Clinic in Marshfield. To increase the accuracy and stability of predictive model, an augmented dataset was generated by adding several variations to the collected one. A deep convolutional neural network (CNN), namely “AlexNet” was then utilized and trained with both datasets. The CNN model was configured in two different strategies: (1) Transfer Learning (TL), (2) Feature Extraction (FEx) combined with three different classifiers, namely Support Vector Machine (SVM), Naïve Bayesian (NB) and K-Nearest Neighbor (KNN). To speed up the training process and make a real-time decision, NVIDIA GPU Quadro M5000 was employed.

Results:

The CNN is trained with TL and FEx approaches using 10%, 20% and 30% of both collected and augmented datasets. While training time of FEx model is in the order of a few minutes, TL model requires 10-20 times more time. The average object recognition time of both approaches is less than 0.3 seconds. The accuracy of recognizing objects of interest is more than 98% regardless of training size. However, the accuracy of TL model varies from about 60% to 99% with respect to the training size.

Conclusions:

Even though GPU processors extremely increase the speed of computing compared with CPUs, training time of TL approach is unbearable. However, both approaches on the stage of test sample evaluation are fast enough to be used in real-time applications. FEx accompanied with either NB, SVM or KNN classifiers achieves accuracy of greater than 98% even with small training set size. To compete with FEx approach with respect to accuracy, recall and precision, TL requires larger training set. For detecting three objects of interest that are common in any health care facility, the CNN model based on FEx is a pragmatic option. The proposed framework has the potential to recognize a variety of objects in the indoor environment, and ultimately to help visually-impaired people navigate their way in the clinic independently.