Evaluating PSAU Mobile Application Based on People at the Center of Mobile Application Development (PACMAD) Usability Model: Empirical Investigation

: This study investigates the extent to which the usability attributes, namely, effectiveness, efficiency; learnability and memorability, satisfaction, errors, and cognitive load of PSAU mobile application exist from students’ point of view who were enrolling at the academic year 2019-2020 in College of Business Administration (CBA) at Prince Sattam Bin Abdulaziz University. The study employs the People at the Center of Mobile Application Development (PAMCAD) usability model to determine the extent to which the usability attributes are available of PSAU mobile application. A survey-based methodology is used to collect data from a random sample size of 137 enrolled students in the College of Business Administration (CBA) at Prince Sattam bin Abdulaziz University. The results demonstrate the state of usability attributes of PSAU mobile application is acceptable; the highest mean was 3.3 for the cognitive load dimension, after that, the learnability and memorability dimensions with mean 3.0. The lowest mean is 2.4 for the Efficiency dimension. The overall mean for usability is 2.8 which reflect the level of usability for the PSAU mobile application. The results of this study should be useful to IT deanships and related policymakers at the university level with empirical evidence about the issues and problems that faced users of mobile applications in higher educational institutions in KSA; and helping in developing high-quality mobile application.


Introduction
Mobile application (app) is software that designs to work on mobile computing platform. The usability is the major issue that may affect the usage of the mobile applications. The concept of usability refers to the fact that an application must be friendly to users, which implies that the software must be easy to use. Software usability is defined as the ease of use of the software. ISO 9241-11 defines usability as ''the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use'' (Moumane et al., 2016;Sagar and Saha, 2017;Seffah et al., 2006). In software engineering, usability is considered as one of most important software quality attributes. Many research studies have revealed that usability problems indirectly or directly have caused problems related to software quality such as lower efficiency and effectiveness, in addition to, poor usability is causes for failure of software application and product (Sagar and Saha, 2017;Bertoa et al., 2006;Seffah and Metzker, 2004;Seffah et al., 2006). Besides, some studies have proved that the software with good usability can lead to increase the productivity and the revenues (Jake-Schoffman et al., 2017). Where it is found that in E-commerce applications improving usability by 5% leads to increase the revenues by 10-35%. Therefore, evaluating the usability of mobile applications is very important, due to proliferation the usage of mobile devices, determining the level of acceptance by its users and discovering the major issues that faced the users of that application and to gain feedback of interface design. Many methods are suggested for evaluating the usability for mobile applications (Moumane et al., 2016;Kortum and Sorber, 2015;Hashim and Ahmad, 2016;Kumar and Goundar, 2019;Nielsen and Mack, 1994;Nielsen and Molich, 1990;Harrison et al., 2013;Parsazadeh et al., 2018;Az-zahra et al., 2019).
This study attempts to examine the extent to which the usability attributes, namely; effectiveness, efficiency, learnability and memorability, satisfaction, errors and cognitive load of PSAU smartphones application are available. To achieve this goal, the PACDAM model which is suggested in 2013 by Harrison, Flood and Duce (Harrison et al., 2013) is employed.
To the researcher's best knowledge, no researches have been conducted at PSAU to evaluate the usability of mobile applications in the institution. Consequently, to enhance the usability of PSAU mobile application by students, understanding the current situation of the usability attributes among CBA students is necessary. Based on a review of the existing literature, limited research has been conducted on university students' evaluation usability of mobile applications. Theoretically, the importance of this study is its contribution to add new empirical evidence to the issues related to the usability of smartphones' applications in higher educational institutions. Further, since this study offers theoretical and empirical results to smartphones' applications, developers' and designer's issues that are related to enhancing the usability of mobile applications, as well as, highlights the major issues related to usability of smartphones' applications in higher educational institutions in KSA.
University administrators at information technology section can use the findings of this study to either improve the usability of mobile applications by overcoming the problems that face users of PSAU app or take into account these issues related to the usability attributes when developing a new mobile application.

Literature Review and Hypotheses Development
Usability of the interface is the core area in the field of Human Computer Interaction (HCI) and an important attribute of any user interface that measures ease at which interfaces can be used. The field of usability evaluation has been widely investigated over the last years; due to proliferation of using smartphones around the world. The usability of smartphones applications is an emerging as a hot topic area of research; many researchers have presented a number of methods for evaluating the usability of smartphones applications for assessing the usability, where more than one model has been used for evaluation the usability. Harrison et al. (2013) proposed the PACMAD model for assessing the usability of smartphones applications. PACMAD identifies seven dimensions that reflect the usability of smartphones applications: Effectiveness, efficiency, satisfaction, learnability, memorability, errors and cognitive load. Kortum and Sorber (2015) described the usability of a large number of mobile apps for both tablets and phones across Android and Apple operating systems. The System Usability Scale (SUS) is the model for evaluating usability, proposed by (Brooke, 1996).
Authors used SUS as a survey instrument, to measure the usability of mobile application. The study performed on more than nine mobile applications; more than 3000 users were participated in the study. Moumane et al. (2016) performed an empirical study for evaluating the usability mobile applications that working or operating on different mobile operating systems such as iOS, Android and Symbian. The software Quality standard ISO 9126 for mobile environment was used. The main obtained outcomes of the study determined some issues related to the usability that must take into account during the developing the mobile application to enhance the usability of mobile application (Bertoa et al., 2006). Kumar and Goundar (2019) used a heuristics method for evaluating the usability of mobile learning application. The suggested method relaying on (Nielsen and Mack, 1994;1990) heuristic approaches, which used to evaluate the usability based on preparing list of criteria and check for their availability, the study was carried out on two applications at University of the South Pacific. Goal Question Metrics (GQM), is a method for evaluating usability of mobile applications suggested by (Saleh et al., 2015). Its major objective is to assess the usability, to determine the main issues in user interface, that may lead to human errors during interaction with mobile application. The PACMA model is used for evaluating the usability of two mobile applications for two universities portals. Az-zahra et al. (2019) used the PACMAD usability model for evaluating the usability of three mobile applications used in Indonesia: Tokopedia, Bukalapak and Shopee. The evaluation performed for seven aspects as in PACMAD, which include effectiveness, efficiency, learnability, memorability, errors, satisfaction and cognitive load. The obtained results showed that Bukalapak was good in almost all aspects, then Tokopedia and Shopee (Az-zahra et al., 2019). Parsazadeh et al. (2018) proposed a usability evaluation model with the inclusion of timeliness to evaluate the usability of mobile learning application; the aim of the study was to construct and validate a usability evaluation for mobile environment. Questionnaire was used as a data collection method.
To achieve the study goals the following hypotheses are developed: H1: The availability of "Efficiently" attribute of usability in the PSAU mobile application used by CBA students is high. H2: The PSAU mobile application used by CBA students has high level availability of Effectiveness attribute of usability. H3: The PSAU mobile application used by CBA students has high level availability of Learnability attribute of usability. H4: The PSAU mobile application used by CBA students has high level availability of Memorability attribute of usability.
H5: The level of Errors attribute of usability in the PSAU mobile application used by CBA students is high. H6: The PSAU mobile application used by CBA students has high level availability of Cognitive Load attribute of usability. H7: The PSAU mobile application used by CBA students has significant level of Satisfaction attribute of usability.

Research Methodology
To achieve the study goals, a questionnaire has been designed based on the dimension of PACMAD usability model that presented by (Harrison et al., 2013) and used by (Az-zahra et al., 2019;Saleh et al., 2015), which measures the usability dimensions that are based on 7 attributes: effectiveness, efficiency, learnability, memorability, errors, satisfaction and cognitive load. The study uses a survey-based methodology to obtain data from the respondents. The questionnaire is divided into two sections, namely, section A and section B. In specific, section A comprised 5 statements designed to collect data about the respondents like the study level, department, gender, age and type of smartphone. Section B ascertains the views of the College of Business Administration students on availability of factors influencing their usability of the university PSAU mobile application (PSAU App). A five-point and four-point Likert scale have been used in this section and the respondents have been required to state the extent to which they agreed or disagreed with the statements in the questionnaire. In order to give interpretations for the obtained results after processing, Table 1 lists the interval and its weights and description (Dawes, 2008;Pimentel, 2010). The sample of the study comprised 137 College of Business Administration students enrolling for the academic year 2019/2020 at Prince Sattam bin Abdulaziz University in Saudi Arabia. The questionnaire was published via web using Google forms to College of Business Administration students using a simple random sampling. The Cronbach's alpha test was used to measure the reliability of questionnaire. SPSS 26 software was used to conduct the reliability test and T test for statistical testing. Table 2 provides the value for Cronbach alpha was 0.901 and reflects high reliability of the measuring instrument (Taherdoost, 2016).

Results and Discussion
Demographic characteristics of students in this study are level of study, department, gender, age, level of study and type of smartphone they use as shown in Table 3. Based on the demographic information, the majority of students in second and forth level with ratio (32.8%) and (29.9%) respectively. Regarding the departments, the majority of the students (41.6%) and 32.11% are in public preparation (general department) and in Law department respectively. In terms of gender and age, the majority of students (54%) were male, while the age, majority of students (84%) between 20 and 21-year-old. In terms of type of smartphones used, the majority of students (91%) used iPhone.
As mention the previous sections the PACMAD model has been employed in this study which depends on six attributes for determine the usability of mobile application. Usability is generally a relative measure, means that the usability may different from user to another, according to context of use, user knowledge and environment as listed in (Seffah et al., 2006). The obtained results showed the status of the usability attributes as reported in the next section.

Efficiency
To reflect the user's ability to complete goals based on speed and accuracy (Harrison et al., 2013;Az-zahra et al., 2014), the efficiency task completion time was indicated (Frøkjaer et al., 2000). To measure the efficiency three questions are used as listed in Table 4. As a response to question1 37% of respondents requires more than 180 seco completing a task; while 34% of respondents requires 60-120 sec completing a task using the PSAU App. When the participants were asked about the time spent on each page, the majority commented that 38% need 60 to 120 sec, while 26% of responses need more than 180 sec. When asked about the number of touches that they need to perform a task, 44% of the respondents reported that they need 2 to 3 clicks or touch; while 26% of respondents need more than 4 clicks to complete a given task. Descriptive statistics such as the mean, standard deviation and standard error mean were calculated for each of the 3 items of the efficiency as illustrated in Table 4. As shown in Table 4, means of the items are between 2.14 and 2.54 and the standard deviation ranges between 1.0091 and 1.051.
Statistically, T test is performed to test the H1 hypothesis "The availability of "Efficiently" attribute of usability in the PSAU mobile application used by CBA students is high ". Table 5 shows the P value for testing. The value is less than 0.05 which means there are significant differences for all items of efficiency; therefore, the H1 hypothesis is rejected; and can state the alternative hypothesis which defined as "The availability of "Efficiently" attribute of usability in the PSAU mobile application used by CBA students is not high".    In summary, these results show that the efficient attribute in the PSAU app is available; but the level of availability is low. The diversity in obtained results may due some reasons such as the tasks may different, level of experience in dealing with the PSAU app. In addition to quality of connection to network, type of the mobile advice and its specification.

Effectiveness
Effectiveness reflects the ability of users to accomplish goals in certain contexts (Harrison et al., 2013;Az-zahra et al., 2019). According to PAMCAD model, the effectiveness has been measured using three questions as shown in Table 6. The first one is related to the number of steps required to complete a given task. The obtained results show that 39% of respondents required 2-3 steps, while 28% of respondents needs steps, 4-5; and 20% required 3-4 steps to perform a given task. The second question which is related to the number of tasks that were performing during specific time; the produced results are as follows: 46% can perform 1 to 2 task at a given or specific time, while 31% can perform 2 to 3 tasks, the others solved more than 3 tasks at specific time (23%). The last question is related to the number of errors that may occur during performing a task, the highest ratio is as follows: 41% of the responses, faces 1 to 2 errors, while 31% shows that the number of errors that may appear are 4 to 5 errors, while 19% shows that the number of errors that may occur 3-4 and zero error has faced 9% of the respondents. Table 7 shows the descriptive statistics such as the mean, standard deviation and standard error mean for each of the 3 items of the effectiveness. The mean for items is between 2.28 and 3.09 and the standard deviation ranges between 1.000 and 1.029. The produced P value of statistical testing is less than 0.005, this leads to reject the hypothesis H2," The PSAU mobile application used by CBA students has high level availability of Effectiveness attribute of usability" and accepting the alternative hypothesis which can be restated as "The PSAU mobile application used by CBA students has acceptable level of Effectiveness attribute of usability ". The diversity in obtained results can be justified by to the natural of the mobile applications environments as listed in the previous section; where many factors that may affect the usability of mobile application such as the quality of connection, device type and its specification, time of using the application and the experience of users, all these factors may have their impact the effectiveness of the mobile application.

Learnability
Learnability attribute measures how simple or easy is for the user to perform or implement a task for the first time [12,14]. There are two questions that are used to measure the learnability as listed in Table 8. In response to question 1, most of those surveyed have indicated that 26% of the responses are neutral and 20% for (agree, disagree and strongly disagree) and only 15% of the respondents positively agree. The second question that is related to the necessary time to learn, in response to this question the highest rate is 28 and 18% of the respondents agree and strongly agree with the time which required is long; while 25% of the respondents are neutral, while others are 19% disagree and 10% strongly disagree. Table  9 displays the results of statistical test including descriptive summary and P value as noted the mean for learnability attribute is 3.06 and 0.61 for standard deviation. The obtained P value is less than 0.05 this leads to prove that the hypothesis H3 is rejected; and accepting the alternative hypothesis which is stated as" The PSAU mobile application used by CBA students has good level availability of Learnability attribute of usability. Overall, these results indicate that the learnability of the PSAU app is acceptable according to the natural of mobile computing platform and its limitations.

Memorability
Memorability refers to that user interface is easy to be memorized if an interaction happened after a period of inactivity similar to the one happened before the period (Fenu et al., 2017;Harrison et al., 2013;Az-zahra et al., 2019) defined the memorability as a measure reflects the user's ability to master the use of app effectively. To measure memorability using PACMAD model two questions were used the first one was related to the reuse of the application and second was about the number of tasks that a user can performed at the first time. Table 10 lists results that obtained how the application is easy to reuse. 28 and 21% agree and strongly agree respectively, while 25% from responses are neutral and 15 and 12% strongly disagree and disagree respectively. Regarding the responses about the number of tasks that may be achieved at the first time use, the neutral and disagree get the highest ratio with 26% of respondents, while 20 and 10% agree and strongly agree and 18% from responses strongly disagree.      Table 11 reported the descriptive statistical of the memorability factors. The P value obtained is less than 0.05 this leads to reject the H4 hypothesis which was "The PSAU mobile application used by CBA students has high level availability of Memorability attribute of usability" and accepting the alternative hypothesis by restate it as "The PSAU mobile application used by CBA students has accepted level of Memorability attribute of usability". In addition, the diversity in responses may be related to the level of experience as well as to other factors that affect the usage of mobile applications in general like the context of use and tasks that perform. Thus, the memorability PSAU App is not bad.

Errors
Error's attribute is related to mistakes made by users during interact with the application (Harrison et al., 2013;Az-zahra et al., 2019). Three questions have been used to measure this attribute as they listed in Table 12, the first one regards the recovering from error; as display in table; 42% from responses neutral, while 23 and 16% disagree and strongly disagree. The second question is related to the error messages that clearly inform user how to recover the highest response for this question. 31% strongly disagree. The third question is regarding to the user response easily to recover errors; the highest response is 29% strongly disagree. The overall average for this attribute is 2.5 with standard deviation 1.06 and 0.000 for P value as illustrated in Table 13. The produced P value is less than 0.05 this reflects a significant difference and leads to reject the H5 hypothesis" The level of Errors attribute of usability in the PSAU mobile application used by CBA students is high"; and accepting the alternative hypothesis that can be rested as "The level of Errors attribute of usability in the PSAU mobile application used by CBA students is low".

Cognitive Load
This attribute determines the level of cognitive processing that the user needs to use the application. (Harrison et al., 2013;Az-zahra et al., 2019). To measure the cognitive load five questions have been used, as listed in Table 14. The highest response is neutral with 24% for the first question and the next is strongly disagree with ratio 22%, where the question is related to achieve the task within a given time. The second question is about affection the errors user workload, response to this question indicated that 32 and 30% they agree and strongly agree, while other responses are 25% for neutral and 13% for disagree and strongly disagree. The response to question three indicated that 33 and 19% from respondents agree and strongly agree with the question, while 23% from respondents are neutral. In response to question four the highest response is 32% neutral and 28% agree with this question. The last question, which is related to the time taken to response from application, in response to this question; the highest response is 37% indicated that the time is between 60-120 sec, while 28% of the responses indicated that the time less than 60 sec, these are the highest responses that have been obtained. Table 15 reported the results of statistical test including descriptive summary and P value. The mean for cognitive load attribute is 3.14 and 0.69552 for standard deviation. The P value that obtained is less than 0.05 which indicated that there is a significant difference, this leads to reject the H6 hypothesis which was "The PSAU mobile application used by CBA students has high level availability of Cognitive Load attribute of usability.; and accepting the alternative hypothesis that indicated that the cognitive load of the PSAU app is acceptable.

Satisfaction
This attribute measures the comfort and acceptability of the work systems to its users; two questions are used to measure this attribute (Harrison et al., 2013). Table 16 lists the obtained results after processing the data. In response to question 1 has indicated that 25% from responses agrees with using the application, while the rest of responses are diverse or varies between disagree and strongly disagree and neutral and strongly agree respectively. In response to question 2 the majority of participants are neutral and strongly disagree. The descriptive summary about the obtained results and P value are illustrate in Table 17. The P value that produced indicated that there is significant differences, which leads to rejected the H7 hypothesis that was "The PSAU mobile application used by CBA students has significant level of Satisfaction attribute of usability"; and accepting the alternative hypothesis by restated it as "The PSAU mobile application used by CBA students has acceptable level of Satisfaction attribute of usability". Overall, these results have indicated that the satisfaction of students to PSAU app is not bad.        Table 18 indicates a summary of the results. The highest mean is 3.3 for the cognitive load dimension, the learnability and memorability dimensions are with the mean 3.0. The lowest mean is 2.4 for the efficiency dimension. Therefore, the obtained outcomes have demonstrated that the status of each attribute with a vary level of availability. Obtained results have proved that the usability is a relative measure, where the usability varies from user to user according variety of factors.
The main contribution of this study can be summarized as the followings:  First: The status of usability attributes of PSAU mobile applications  Highlights the major issues that face the usability of mobile applications in higher educational institutions in KSA, providing feedback to mobile applications' designers and developers, regarding the issues that are needed to be enhanced and improved, guidelines that assist the mobile applications developers and designers to develop applications with good quality

Conclusion
The main goal of the current study is to examine the extent to which the usability attributes, namely; effectiveness, efficiency, learnability and memorability, satisfaction, errors and cognitive load of PSAU mobile application exist among students who were enrolled in College of Business Administration (CBA) at Prince Sattam Bin Abdullaziz University for the academic year 2019-2020. To achieve the goals of the study a surveybased methodology was used to obtain data from the respondents based on the dimensions of PACMAD usability model. The results of this investigation have stated the current status of the usability of PSAU mobile applications is acceptable; furthermore, the results of this research supported the idea that the usability is a relative measure, as it varies from user to another user according several factors. This study contributes to the literature of usability evaluation of mobile application at higher educational institutions in KSA. In addition to this, the results of the study provide developers and designers of mobile with insights on the usability issues of the mobile applications. Future studies may consider other various samples such as different colleges and universities as well as developing new methodologies to evaluate the usability.