JMIR Publications

ABSTRACT

Background:

Significant progress in reducing maternal and infant mortality worldwide has not fully eliminated inequalities, which persist both between and within countries [1,2]. Each year, more than 4.5 million women and newborns die during pregnancy, childbirth, or the first weeks of life. The vast majority of these largely preventable deaths occur in low- and middle-income countries (LMICs) [1–3]. Mobile health (mHealth) applications can be used to wellness management, behavior change, health data collection, disease management, self-management, self-diagnosis, and rehabilitation. They can also serve as electronic patient portals and medication reminders [4,5]. Research studies on mHealth apps and their outcomes have shown that well-designed mHealth applications can empower patients, improve medication adherence, and reduce the cost of healthcare [6–9]. Furthermore, consuevaluates the usability and user satisfaction of HADA Educación, a mobile application developed to deliver digital health education through the Mujer Saludable (Healthy Woman) program, targeting women in fertile age, pregnancy, and postpartum stages in an underserved region of Colombia.mers appear to prefer health apps developed by private companies or trusted healthcare providers, such as hospitals or health systems that operate independently of insurers. Future work by researchers and app developers should focus on qualitative methods and usability testing to better define quality app requirements and ensure they meet end-user expectations [10]. Therefore, it is essential to evaluate the usability of any digital development before its implementation in healthcare settings, ensuring that it meets the needs and expectations of both patients and healthcare professionals. Five essential usability characteristics should be considered when developing software: learnability, so that users can quickly start working with the system; efficiency, to achieve a high level of productivity; memorability, which allows occasional users to return after a period of inactivity without having to relearn everything; a low and easily correctable error rate; and finally, satisfaction, making the system pleasant to use [11,12].

Objective:

Evaluate the usability and user satisfaction of HADA Educación, a mobile application developed to deliver digital health education through the Mujer Saludable (Healthy Woman) program, targeting women in fertile age, pregnancy, and postpartum stages in an underserved region of Colombia.

Methods:

A mobile application (HADA Educación) was designed based on an existing educational program woman during their reproductive stage (fertile age, pregnancy, and postpartum), following user-centered design principles. A prospective observational study was carried out by conducting a pilot test between May and November 2024. This zone is in the northwest of Colombia, and limits with Panamá, here is where the Darien Gap is located, and common immigration route to Central and North America. This zone has an area of 11,664 km2 (4,503.5 sq miles), with an approximately permanent population of 525,685 inhabitants, distributed in 11 municipalities, 51% of population are women, and the general fertility rate is 46.0. [13,14]. We recruited and consent women of reproductive age in the region of interest, inclusion criteria: women between 18 and 49 years old, residents of the Urabá subregion in Antioquia, Colombia with access to a mobile or Wi-Fi network. HADA Educación offers personalized health education content to women through a learning pathway tailored to the user's reproductive stage. During the pilot test, participants explored the application for a period of 6 months, they completed the validated System Usability Scale (SUS) questionnaire and a custom-designed satisfaction survey. Educational Program The Mujer Saludable (Healthy Woman) educational program was developed with the goal of providing digital health education through the app HADA Educación, covering topics related to disease prevention, recognition of warning signs, and the promotion of healthy habits. It is intended for women during their reproductive stage, including preconception, pregnancy, and postpartum. The program follows the public policy developed in Colombia that promotes comprehensive care and adaptation for women during pregnancy and postpartum The development process of the program began with the identification of educational needs through a descriptive cross-sectional study, which surveyed 1,478 women between the ages of 18 and 44 in prioritized municipalities, allowing for the definition of program emphases. [15] Based on the results, the micro-curricular structure of the program was established, defining its scope and main topics. Subsequently, a team of expert healthcare professionals validated the relevance of the topics, drawing from literature and evidence, and consolidating the information with bibliographic references for each subject. Finally, a content committee—comprising health professionals, social communicators experienced in health education, graphic designers, and illustrators—was responsible for adapting the language and representing the information in various formats such as podcasts, videos, reels, infographics, images, and carousels. Software Development The mobile app was developed with the collaboration of systems engineering professionals and healthcare personnel, using agile methodologies with an iterative approach in which the user is the central focus of the design and development. The UFuRT framework (User, Function, Representation, and Task analyses) was integrated to ensure an optimal user experience. [16,17] This framework enabled a detailed analysis of user characteristics, needs, and behaviors, ensuring that the information structure was aligned with their knowledge, skills, and context. The development process was structured into four key phases: • Discovery Phase: This began with user analysis to understand their needs and behaviors. Co-creation committees were established to collect key information. From this process, two models were developed: a technical design model and an initial user model based on mockups, which were combined to create a comprehensive functional model. • Requirements Specification Phase: Identified functions were prioritized, and operational flows were defined. Functions were also classified according to their importance. • Validation and Verification Phase: Representation analysis strategies were implemented to evaluate the presentation and use of data within the system. Testing and validation processes were carried out to ensure proper system performance. • Development Phase: Both secondary and prioritized functions were implemented, ensuring alignment with the established requirements and validations. The software architecture was based on the Onion Architecture model, ensuring a clear separation between infrastructure, business logic, and application layers. The technologies used were: • .NET Core for the backend • React and React Native for the frontend • PostgreSQL for the database For development management and control: • Azure DevOps was used to support continuous integration, agile planning, and team task tracking • Enterprise Architect was used to model system architecture, document workflows, and define the information structure • Visual Studio was the main development platform The software’s development and execution flow follows a defined scheme: HTTP requests reach the API, which delegates the requests to the application layer, where use cases are processed, and business rules are validated in the domain layer. Finally, the processed data is returned to the client, ensuring a robust, scalable system aligned with users' needs. Pilot test Social work professional recruited participants in the field, approaching women on waiting rooms of various healthcare institutions within the municipalities selected for the study through convenience sampling. Interested women were informed about the study and later contacted to receive an e-card outlining the participation requirements, which included a registration link. Through this link, participants could voluntarily register by first accepting the terms and conditions and then completing the registration process. A total of 104 women registered and were contacted by phone to coordinate the delivery of a tablet with the Android operating system, pre-configured to allow only the use of the HADA Educación application. Participants signed Informed consent form to participate in the study and the device handover form upon delivery of the device. Several locations were made available for this process. However, 15 women did not attend, leaving a total of 89 active participants in the pilot test. The social worker supported participants in registering within the application, which was an essential step to ensure that participants could interact with the system. System Usability The Spanish version of the System Usability Scale (SUS) questionnaire was applied at the and of the pilot test (see Table 1). This is a 10-item questionnaire rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Additionally, participants were allowed to add comments at the end of the questionnaire. Its advantages include versatility, simplicity, low cost, accuracy, and validity, with a high reliability (Cronbach's α = .85). The usability analysis of the system was carried out using the System Usability Scale (SUS). The questionnaire is designed to be answered after the user interacts with the system. It is organized to alternate between positive (odd) and negative (even) statements to avoid the usual bias of the respondent. The score contribution for odd-numbered items (positive statements) is the scale position minus 1, and the contribution for even-numbered items (negative statements) is 5 minus the scale position. The overall score is calculated by summing the scores of all the items and multiplying by 2.5, with the overall score ranging from 0 to 100. A system with a score above 85 is considered to have excellent usability, while a system with a score between 68 and 84 is considered to have good usability [18]. Satisfaction The Mujer Saludable (Healthy Woman) educational program satisfaction was evaluated using a custom-designed survey. The survey consisted of 10 questions on a 5-point Likert scale (1 = Not at all, 5 = Very much). Items 1 to 5 assessed participants' experiences with the program, while items 6 to 10 focused on their learning process (See annex 2).

Results:

The educational program was structured around three thematic cores: Women of Fertile Age, Pregnant Women, and Postpartum, designed according to the specific educational needs of each stage. It includes 6 pathways, divided into 16 formative experiences, 39 learning units, 167 educational contents, and 190 graphic resources. To participate in HADA educación, total of 89 women consented to participate in the study. From the municipalities of Apartadó, Turbo, Chigorodó, and Carepa participated. They were categorized according to their reproductive stage (fertile age, pregnant, and postpartum). At the end of the pilot test, usability and satisfaction surveys were administered to evaluate interaction with the system. Responses were obtained from 77 participants. Regarding the place of residence, the analysis by municipality shows that Carepa recorded the highest number of women in the fertile age group (22), while the number of pregnant women was significantly lower (3) compared to the other municipalities, and there were no postpartum participants. In Turbo, the distribution was more balanced, with 8 women of fertile age, 9 pregnant women, and 4 postpartum, indicating a higher presence of women in pregnancy compared to other municipalities. In Apartadó, the majority of participants were also in the fertile age group (14), followed by 4 pregnant women, with 2 postpartum participants. Finally, Chigorodó had the lowest number of participants in the fertile age group (5), with 2 pregnant women and 3 postpartum. The geographical distribution of the participants showed a concentration in the municipalities of Carepa (33%), Turbo (27%), and Apartadó (25%), with lower participation in Chigorodó (12%). However, when compared to the estimated population distribution —Apartadó (34.4%), Carepa (13.9%), Chigorodó (16.9%), and Turbo (34.7%)— there is an overrepresentation of Carepa and an underrepresentation of Apartadó, Chigorodó, and Turbo. This variation can be attributed to greater accessibility for participant recruitment in Carepa, facilitated by the existing networks and relationships of the operational staff based in the region, which eased community engagement and outreach efforts in this municipality. Analysis of the satisfaction survey results. A total of 77 people participated in the usability analysis, with 12 excluded for not answering the survey. The average age was 32 years, and the education levels varied among participants, with the majority having completed secondary education (22 cases). Most of the participants are in their fertile age (49), followed by pregnant (18) and postpartum (10). All survey respondents were classified as belonging to low socioeconomic stratification framework. System Usability Scale (SUS) The usability classification based on the scoring obtained an average of 82.4 points, with 44% of participants rating the usability as excellent (>=85), 48% as good (between 68 and 84), and the remaining 8% rated it as acceptable (<68) (table 4. SUS survey results). The results indicate a positive perception of the system's usability. The statement "I think I would like to use this system frequently" received an average of 4.43 (SD = 0.52), suggesting a high level of acceptance of the system by the users. The items related to ease of use reflect a favorable perception. The statement, "I think the system is easy to use," received a high average rating of 4.39 (SD = 0.65), closely followed by "I think people can learn to use this system very quickly" at 4.38 (SD = 0.61). Notably, "I feel confident using the system" received the highest score of 4.51 (SD = 0.55), indicating that users feel secure interacting with the platform. Conversely, the negatively worded items—where lower scores indicate positive usability—were rated favorably. For instance, "I find this system unnecessarily complex" received a low average of 1.88 (SD = 0.86), and "I think I would need technical help to use this system" averaged 1.91 (SD = 0.86). This suggests that most users do not perceive the system as complicated or requiring technical support. Other aspects were also rated positively. The integration of functions was well-regarded; with the statement "I think the functions in this system are well-integrated" scoring 4.38 (SD = 0.54). Perceptions of inconsistency were low, as evidenced by the average rating of 1.79 (SD = 0.66) for "I think there are many inconsistencies in this system." Finally, the statement "I feel I would need to learn a lot before being able to use this system" received a low average score of 1.86 (SD = 0.76), reinforcing the conclusion that users find the system's learning curve to be minimal. A total of 10 SUS items were evaluated to assess perceived system usability. The results indicate a high level of usability and user satisfaction. For the positive statements, a majority of participants agreed or strongly agreed with the system’s ease of use (91%), functional integration (97%), and quick learnability (93%). 97% reported feeling confident using the system, and 99% indicated they would like to use it frequently. Conversely, for negatively worded items, the majority of users disagreed or strongly disagreed, suggesting a low perception of complexity. 80% disagreed that the system was unnecessarily complex, and 85% disagreed with needing technical support. Similarly, 89% disagreed with the presence of inconsistencies, and 91% disagreed with the idea that the system was cumbersome. Finally, 87% disagreed with the notion that prior knowledge was needed before using the system. 3.2 Analysis of the Satisfaction Survey Results. A satisfaction survey, designed specifically for the study, was applied to the 77 participants in the controlled test. It included 10 questions evaluated using a Likert scale from 1 to 5 (1: "Not at all" – 5: "Too much"). The last question asked whether the participant considers it necessary to have prior knowledge to understand the content of the program. The results of the satisfaction survey reflect a highly positive educational experience among participants. Overall, the majority of responses were concentrated in the higher levels of the Likert-type scale used (from 1 to 5, where 5 is "Too much"), demonstrating a favorable perception of the program's content, methodology, and impact. A particularly notable aspect is the program's accessibility. When asked about the need for prior knowledge to understand the content, 34% responded "Not at all" and 52% "A little." This question, formulated in reverse, indicates that 86% of participants believe that prior knowledge is not necessary to understand the program, which is a highly positive result. This reflects that the instructional design is clear, understandable, and accessible even to those without technical training or prior experience in the subject. Regarding the learning experience, the results are also encouraging. Sixty-eight percent of respondents indicated that learning through digital media was "Too much" enjoyable, and 65% expressed their willingness to implement at least one of the recommendations received. This suggests not only a pleasant experience but also a high level of appropriation and applicability of the knowledge acquired. Another strong point of the program is its usefulness in increasing health knowledge: 77% of participants considered the information extremely useful. Similarly, 77% indicated they would recommend this program to others, reflecting high overall satisfaction and an experience of sufficient value to be replicated or shared. Regarding the ease of understanding the information, although 62% responded "Too much," 36% chose "Quite a lot." While these are positive results, they suggest that there is room for further work to improve the clarity or accessibility of some content.

Conclusions:

Technological advancements have facilitated the development of mHealth tools that enhance access to health education and promote self-management of well-being, particularly among women in their reproductive years [19,20]. Some interventions demonstrated improved maternal health, such as reduced adverse pregnancy outcomes through mobile-based prenatal education programs and remote monitoring [21, 22, 23, 24], and reduced perinatal mortality using telehealth and education models [25]. Digital prenatal education delivered through mHealth applications and virtual online platforms efectively enhanced knowledge and encouraged healthy pregnancy practices among antenatal mothers, as evidenced by multiple studies [26,27-32]. That is why this program was designed with a focus on educating women of reproductive age, during pregnancy, and in the postpartum period, aiming to enhance health. In this context, evaluating mobile applications through usability and satisfaction surveys is essential—not only to assess the ease of use but also to determine the extent to which these tools meet users’ expectations and needs. This study assessed both the usability of the HADA Educación mobile application and the satisfaction levels of its participants (n = 77). Results demonstrated high acceptance, with an average satisfaction score of 4.55 out of 5 and a mean System Usability Scale (SUS) score of 82.4. These findings suggest that HADA Educación is a technically viable and well-received digital tool for health education. Satisfaction, in this context, is as the relationship between users’ prior expectations and their actual experience with the system [33]. The SUS results are consistent with existing literature on the usability of mobile health applications. Several studies have reported similarly favorable scores, supporting the utility of such tools. For example, one study developed, validated, and assessed a mobile application for oral health promotion, reporting a SUS score of 73.75, with validation by pregnant users and expert reviewers [34]. Another study, which evaluated a mobile application for employment counseling during pregnancy, obtained an average score of 68. Similarly, a self-care application scored 67.82 on the SUS [35,36]. Furthermore, a study evaluating a mobile app designed for women undergoing assisted reproduction reported SUS scores of 76 (SD = 13.54) at baseline, 75 (SD = 17.16) at three months, and 84 (SD = 11.21) at six months [37]. Collectively, these studies demonstrate a generally positive trend in the usability of health-related mobile apps, underscoring their effectiveness in delivering user-friendly, functionally appropriate digital solutions. Analyzing sociodemographic data alongside usability and satisfaction outcomes provided additional insights into user engagement. The average participant age was 32 years, with a distribution that included women of reproductive age (63.64%), pregnant (23.38%), and postpartum (12.99%). This group typically expresses a strong interest in accessible, timely, and reliable health information, which may have contributed to their positive reception of the application. Notably, 63.64% of participants had attained higher education, which likely facilitated interaction with the digital platform and may have enhanced their perceptions of its utility, accessibility, and ease of use. Similarly, another study conducted in a rural area with participants who had mostly completed secondary or higher education reported a SUS score of 73 and an average participant age of 29 [33]. Importantly, this study was conducted in a context of socioeconomic vulnerability in Colombia, with the majority of participants belonging to lower income strata. This highlights the relevance of the application as a tool for reducing structural barriers to health information. As a free and accessible digital resource, HADA Educación contributes to the reduction of health inequities by delivering useful, tailored content. Its context-sensitive design promotes user engagement, appropriation, and empowerment, supporting broader goals in public health and digital inclusion.