JMIR Publications

ABSTRACT

Background:

This study aims to determine the attitudes and perceived competence level of 64 Filipino registered psychometricians with prior experience in psychological tele-assessment to conduct this service for their present and future clientele. Researchers used the unified theory of acceptance and use of technology (UTAUT) that connects attitude and mental health services and a five-theme framework that discusses the factors or needs in facilitating successful tele-assessment. The results revealed a predominantly positive attitude toward psychological tele-assessment yet with the emergence of contradicting appraisals. Most tended to agree on the benefits of tele-assessment in addressing mental health concerns, but concerns about its flexibility, privacy, confidentiality, rapport, and communication were salient. Respondents reported high openness to conducting tele-assessment in the future but recognized that this was more challenging than the traditional assessment. They evaluated themselves with low confidence and were somewhat knowledgeable about the ethical issues involved in this service. Results further indicated low perceived competence to conduct tele-assessment. They expressed the need for more training, specific local guidelines, access to technology, and available online tests for tele-assessment.

Objective:

This study aimed to determine the' attitudes and perceived competence levels of Filipino registered psychometricians toward psychological tele-assessment. Furthermore, it aimed to explore their needs in conducting psychological tele-assessment. Specifically, this research aimed to answer the following questions: What are the attitudes of Filipino registered psychometricians toward psychological tele-assessment? What is the perceived competence level of Filipino registered psychometricians toward psychological tele-assessment? What are the needs of Filipino registered psychometricians that will help them conduct psychological tele-assessment?

Methods:

The research used a mixed-method design.

Results:

The scores of items in the attitude section of the questionnaire are presented in Table 2. Respondents strongly agreed that tele-assessment in in line with the modern times (M=3.37, SD=.604) and it will reach more individuals with mental health concerns (M=3.44, SD=.560) items. Also, moderate mean scores ranging from 2.63 to 3.22, which were interpreted as “agree” by the respondents, were observed from the majority of the items. On the other hand, respondents reported disagreement on several items with mean scores ranging from 1.97 to 2.45. They reported the major disagreement on the confidence in data security during tele-assessment (M=1.97, SD=.776) and on the preference of tele-assessment over a conventional face-to-face assessment (M=2.13, SD=.701). The summary of scores for the perceived competence toward psychological tele-assessment is presented in Table 3. High level of openness was observed when asked about their participation in tele-assessment in the future (M=3.16, SD=.695). In terms of perception of difficulty level, respondents deemed it was more challenging than the traditional testing and assessment (M=1.42, SD=.529). They have low level of confidence in participating in tele-assessment in the future (M=2.20, SD=.694) and reported being somewhat knowledgeable about the ethical issues involved in conducting tele-assessment to clients (M=2.31, SD=.687). Mean scores of the variables and verbal interpretations are presented in Table 4. The attitudes of the psychometricians to psychological tele-assessment were deemed positive (M=2.66, SD=.34) while perceived competence was deemed low (M=2.27, SD=.31). The responses of the respondents with regards to their additional needs and resources to effectively conduct psychological tele-assessment are presented in Table 5. The majority of the respondents (f=31) requested more training, in which responses included webinars, workshops, “support from the organizations in conducting psychological tele-assessment”, and “proper training and knowledge on how to handle tele-assessment well.” Following this is the need for specific local guidelines (f=16), such as clear and updated ethical rules that are fit for Filipino psychometricians and their clientele. One respondent stated, “I’m not sure if there is one but local guidelines for telehealth practices… Currently, we’re only following/adapting guidelines from the APA, but not everything applies to us because the laws are different.” Technological accessibility (f=12), where the majority of the respondents cited stronger internet connection as one of their necessities. Some of the responses pertained to the availability of effective computer programs and applications, and the “need to be immersed in tech-related discussions so test admins would be more adept in navigations." Session security (f=9), pertaining to the need for a safe and confidential online session should be a priority according to the respondents. This concern was about cybersecurity and data privacy. Some of the responses were: “test content is not secured because it can be recorded by the clients” and “secured website that protects the data of the client.” Lastly, the availability of tests that are fit for online sessions (f=7) was a concern because not all psychological tests can be administered online. Specifically, respondents stated that “online version of valid and reliable psychological tests” and “psychological tests modified for tele-assessment” are needed.

Conclusions:

Attitudes toward tele-assessment Our findings indicate an inclination to a primarily positive attitude, yet with the emergence of contradicting appraisals in some aspects of tele-assessment. Consistent with the definition of Khan et al. (2017), the researchers evaluate the psychometricians’ attitude in continua. Hence, we describe it as primarily positive yet with an ambivalent attitude. Positive attitudes involve their strong agreement toward the benefits of tele-assessment, which include reaching out to more clients in need of professional help, thus addressing more mental health concerns. Psychometricians strongly agree that the facilitation of tele-assessment is in line with modern times. Most of them agree that tele-assessment can necessitate early interventions. These findings are consistent with the technology acceptance model (TAM) and the unified theory of acceptance and use of technology (UTAUT) by Venkatesh et al. (2003), which emphasizes that perceived usefulness is significant for usage intentions. The study of Apolinario-Hagent et al. (2017) supports these findings, explaining the connection between a positive attitude and a sense of helpfulness to mental health services using technology. A more recent framework consistent with the findings is the integrated theory of Park (2013), which accentuates the role of awareness and positive attitudes in the increased use of telepresence systems. Although the respondents are highly aware of the high cost of tele-assessment, they agree on having a lesser expense than the usual price for a face-to-face assessment. These optimistic viewpoints are, however, challenged by other findings. Interestingly, most psychometricians agree that technology to conduct assessment can be an effective alternative to conventional assessment. However, they would still personally opt to have the face-to-face assessment than online. Another concept in the integrated theory of Park (2013) regarding the use of high-quality technology that can enhance performance appears less evident among the respondents. Most respondents disagree that rapport can be just as quickly built on the internet, similar to the face-to-face assessment. For them, the physical distance serves as a barrier that decreases the client’s opportunity to speak openly and honestly. In connection with this, we identify that psychometricians recognize that difference exists in the success and rapport between a client and test user when using the internet or face-to-face setting. Analyses reveal that they predominantly agree about communication issues in an online platform occurring as often as in the traditional setup. The lowest agreement is found for the statement concerning data security in the practice of tele-assessment. Perceived competence in tele-assessment Another point to consider is the perceived competence of psychometricians toward tele-assessment that pertains to one’s evaluation of confidence, openness, and knowledge in task accomplishment, as described by Kremer et al. (2012). Although this study lacks a clear-cut operational definition of perceived competence in tele-assessment, the abovementioned description applies to the context. Psychometricians present high openness to participate in tele-assessment in the future. Despite the scant literature about the practice of tele-assessment in the Philippines, many psychometricians express their high interest and intent to participate in webinars about remote testing and assessment. These are evident in the different posts and comments sections of the known psychology groups or pages on social media. The 64 respondents find conducting tele-assessment more challenging than face-to-face testing, supporting the general impression that fewer psychometricians and psychologists practice tele-assessment in the Philippines. This appears congruent with the explanation that tele-assessment requires more complex preparation, procedural memory skills, and actual hands-on practice (Suhr, 2021). Interestingly, psychometricians give a low rating in their confidence to participate in this specialized yet equally important psychological service. In terms of knowledge about ethical issues involved in conducting tele-assessment of clients, psychometricians describe that they are somewhat knowledgeable about these aspects. Overall, the psychometricians rate themselves with a low level of perceived competence. Needs of psychometricians in conducting psychological tele-assessment In general, psychometricians interpreted their needs in conducting psychological tele-assessment as the different resources that will prepare and equip them in their professional practice. Psychometricians should be competent in the field where continuing education and training are needed to effectively provide psychological services as stipulated in the Code of Ethics and Professional Standards for Psychology Professionals (Professional Regulation Commission [PRC], 2017, Section II.C). This affirms the study of Cabrera (2019), which emphasized the need for psychology professionals for Continuing Professional Development (CPD) activities that are effective and will meet the international standards to enhance their abilities. Tarroja et al. (2020) reported that the majority of the psychometricians were able to receive training from their school and supervised internship about various psychological tests, test administration, and assessment-related topics in a face-to-face setting. Due to the pandemic and the quarantine restrictions set by the Philippine government, many schools offering graduate programs in psychology find alternative ways of teaching assessment and providing internships. Some schools opted to teach psychological assessment in the context of telehealth, while face-to-face internship was forcefully stopped to ensure the safety of the students. Students were asked to prepare and conduct webinars about mental health and write case studies as alternative activities for their internship. These activities, although beneficial, seemed to limit the competency of the students professionally or practically. The challenges mentioned above are consistent in other countries. Sayegh (2021) explained that the pandemic has disproportionately affected teaching assessment courses and training the students remotely. These concerns necessitate local resources and having clear guidelines in tele-assessment, as Wright et al. (2020) asserted that remote assessment is here to stay and should be incorporated into teaching and training, similarly to in-person assessment. Although the PAP has issued telepsychology guidelines, most of its contents focus on registered psychologists' roles. The role of psychometricians in telepsychology may vary depending on the context (PAP, 2020). Accordingly, psychometricians can only administer levels A and B psychological tests (PAP, 2009; Philippine Psychology Act, 2009; PRC, 2017). Moreover, the country's small number of registered psychologists has inadvertently prompted psychometricians to adapt to the guidelines with few adjustments to facilitate their work effectively and avoid ethical violations. Psychometricians may experience ethical dilemmas. While they abide by the Code of Ethics for Psychologists, they are not psychologists because their license is only for psychometricians. Although there are foreign guidelines available in conducting tele-assessment, having clear guidelines specifically for psychometricians on psychological tele-assessment might minimize this concern. Technological accessibility, such as stronger internet connection, is another concern of the psychometricians in conducting psychological tele-assessment. A news article revealed that the Philippines’ internet is the second slowest in the ASEAN countries (Porcalla, 2020). On the other hand, a recent report highlighted that it has improved. The country was ranked 17th for fixed broadband and 23rd for mobile internet out of the 50 countries in Asia (Ookla’s Speedtest Global Index as cited by Mercurio, 2021). Despite these advancements, most of the respondents are still heavily concerned with the unstable internet connection in conducting psychological tele-assessment because it affects the success of the sessions. In support of this, having sufficient knowledge about computer programs or software is needed. Psychometricians are expected to be adept in using technology, considering remote assessment as an alternative to a face-to-face assessment that includes technological competence, equipment, tools, and comfort with technology (Eichstadt et al., 2013). Psychometricians are also concerned about the privacy and confidentiality of sessions. These principles are some of the focuses of the Code of Ethics. Similarly, it is mandated in the law (Data Privacy Act, 2012) that any data, which includes personal and sensitive data, from an individual should be carefully secured and protected. The use of technology, which applies to other services like counseling, requires precaution. In the study of the adoption of psychologists toward information and communication technologies (ICT), technology problems (e.g., equipment adjustments) and technical failures (e.g., internet connection), as well as ethical concerns, reduced privacy, and lack of security, are the factors that need to be considered (Dores et al., 2020). As psychometricians who conduct testing for psychological assessment, standardized tests modified for online testing are necessary. There are available tests, but the psychometric properties during psychological tele-assessment are in question due to standardization issues. The majority of the available tools for assessment are made only for a face-to-face assessment. Hence, there is a need for psychometrically valid tests for tele-assessment. Some test publishers have modified and provided tests for digital testing (PARiConnect, n.d.; Pearson Assessments, 2021). The American Psychiatric Association (APA, n.d.) provided emerging measures in section III of the Diagnostic and Statistical Manual of Mental Disorders Fifth Edition (DSM-5) for test users to use as initial tools in their assessment. However, the standardization and psychometric properties should be prioritized in administering those tests. Moreover, having extensive training regarding these tests should be prioritized to minimize errors because conducting psychological tele-assessment. Clinical Trial: n/a