Using Expert Evaluation to Assess the Implementation of Persuasive Design in e-Commerce

Corresponding Author: Majed Alshamari Department of Information Systems, King Faisal University, Hofuf, Kingdom of Saudi Arabia Email: support@thescipub.com Abstract: It has been argued that the use of persuasive design in ecommerce applications has a positive impact on user behavior and on converting browsers into buyers. Effective e-commerce applications successfully gain customers’ trust. Thus, it is important for designers to know how to apply and embed persuasive design models in e-commerce applications. This study aimed to assess the extent to which persuasive designs were implemented in e-commerce applications in Saudi Arabia using an experts’ evaluation method. In order to answer this question, one of the most commonly used models, namely the Persuasive Systems Design (PSD) Model, was utilized as a benchmark against which the adoption of PSD guidelines in e-commerce was measured. Several usability experts were approached to conduct the evaluation of a variety of e-commerce applications using the PSD guidelines. The findings reported the number of criteria that were highly utilized, such as primary task support, credibility support and dialogue support. The results also revealed a lack of implementation of a number of persuasive design guidelines, such as the social support principles. The authority and verifiability principles were applied in a limited way in the selected e-commerce applications. Further directions and recommendations for better implementation of persuasive design guidelines in e-commerce are discussed.


Introduction
This template, Technology has made a tremendous impact on the daily lives of individuals. The Internet is arguably one of the greatest achievements of the technological revolution. E-commerce is just one of a variety of helpful services provided via the Internet. Many individuals prefer to utilize e-commerce over traditional commerce (Ali, 2017). E-commerce applications, therefore, compete in providing their services, making it necessary for these applications to be user-friendly in order to achieve satisfaction and meet users' needs. However, despite the efforts of businesses to implement and adopt various usability criteria in the design of e-commerce software, many users still do not utilize these applications. Therefore, a persuasive design is proposed. It has been described as a beyond usability approach. It is a technique that is used to achieve a positive influence on customers, persuading them to take a specific action. Persuasive design helps users make a decision, it helps designers to communicate, it guides users in the right direction and it helps users to improve their skills (Kim, 2018). Persuasive design is considered beyond the usability concept because it improves the user experience during the creation of applications and websites in order to make them easy to use. This is achieved by understanding the behavior of users and engaging those (Ferraro et al., 2017). Applying persuasive design in e-commerce applications helps encourage people to make online purchases, to promote user satisfaction and to increase consumers' trust. Therefore, e-commerce applications with high-quality features tend to attract more users (Alhammad and Gulliver, 2014). This paper critically reviews different persuasive design models. Then, the Persuasive System Design (PSD) model, the most commonly used model, is employed to assess how a number of e-commerce applications have implemented the various guidelines. This paper then reports to what extent the PSD model guidelines have been adopted and concludes with a number of findings and recommendations.

Selecting a Template
Using persuasive design in e-commerce websites and applications is considered industrial standards for the success of e-commerce. Therefore, there is a significant need to understand how to design and develop persuasive e-commerce websites and applications (Oinas-Kukkonen and Harjumaa, 2008). This advancement can help e-commerce providers achieve their business targets and increase revenue by converting users from browsers to buyers (Capetola, 2018). Persuasive design is a technique that focuses on influencing a user's behavior through service and product characteristics in order to persuade him/her to take a specific action. Therefore, persuasive design in e-commerce can play a significant role in increasing revenue and maintaining customer loyalty (Ahmad and Ali, 2018).
Several persuasive models have been developed that designers can embed and follow in order to take websites beyond usable to persuasive. One of these is the Fogg Behavior Model (FBM). This model aims to understand users' behavior. It depends on the following three factors: Triggers, ability and motivation. The target behavior is displayed when all of these factors occur simultaneously; the user has sufficient motivation, effective triggers and ability. This model attempts to influence only people's behaviors, without changing their attitudes. However, FBM can also be employed to recognize potential problems where persuasive systems may fail to achieve the intended outcomes. Though this model is a simple approach to changing behavior, it has not been validated (Fogg, 2009). Another existing persuasive model is the Behavior Wizard Model, which is a matrix that matches target behaviors with solutions. This model defines 15 types of behaviors. Proposed for academic studies and commercial products, this model does not provide a complete analysis of the target user or the behavior. Also, the information provided by the model informs designers about the behavior, but not the user (Wiafe, 2012). On the other hand, this model is more helpful if a designer can analyze the problem by identifying the behavior and the user so precise methods can be provided for effective persuasion. The categorization of behavior into "flavors" and durations also eliminates the fuzzy thinking involved in the development of behavior change support systems and persuasive technologies (Fogg and Hreha, 2010).
The Design With Intent (DWI) framework is an additional design tool for influencing user behavior. This framework helps to clarify how to apply different persuasive techniques, as well as how to modify behavior based on the design. Like previously mentioned models, this model also attempts to influence people's behaviors, not their attitudes. However, this model does allow designers to look at how others might approach a problem (Lockton et al., 2010).
The Persuasive Systems Design (PSD) model is a comprehensive model for designing and evaluating a persuasive system. This framework is comprised of 28 persuasive principles that guide the designer toward building a persuasive system. These principles are categorized into four groups. The first is primary task support, which concerns the primary strategies that make the main task easier for the users. The second is dialogue support, which centers around the verbal information given to consumers. The third is credibility support, which helps to promote higher credibility and achieve greater levels of persuasion. The last group is social support, through which the principles aim to encourage users by increasing social influences. The PSD model consists of three main phases. First, the designer should understand seven postulates before the analysis and development of the persuasion context. Next, the persuasion context is analyzed. Finally, the designer checks to see if the system achieves a particular set of persuasive system principles (Oinas-Kukkonen and Harjumaa, 2008). Table 1 describes the principles in detail.
There are many research studies that evaluate the persuasiveness of e-commerce websites by using the Persuasive Design model (PSD). Alhammad and Gulliver (2014) clarified how the persuasive features of the PSD model can be used to develop a bridge supporting the evaluation and extraction of persuasive principles in these e-commerce websites. By using a checklist of persuasive design features that included the principles of all four categories in the PSD model, they evaluated 10 e-commerce websites, five of which were based on ranked popularity (Amazon, eBay, Netflix, Walmart and Ikea) with the remaining five based on their ranked sales conversion rate as defined by the Nielsen Company to include: Schwan's (41.7%), ProFlowers (26.5%), Vitacost (24.0%), Woman Within (22.4%) and Blair (20.5%). The results showed intensive use of persuasive features in the selected websites; especially credibility support, dialogue support and primary task support. Yet, these websites displayed weaknesses in the use of social support features. Further, the utilization of dialogue support in e-commerce improved persuasiveness by providing the necessary support for effective interaction between sellers and shoppers. A limitation of this article was that it showed only percentages in the results, without additional details. Also, the researchers did not mention highly utilized principles for the 10 selected e-commerce websites. In addition, they did not include users' opinions in order to achieve a comprehensive assessment. Reduce complex tasks into simpler tasks to help the users to perform the target behavior Tunneling Guide users in the attitude change process by getting them nearer to the target behavior Tailoring Information based on user's potential interest and usage pattern related to a user group Personalization Make more personalized services and content available in a system for persuasion Self-monitoring Allow users to track their behavior Simulation Enable users to see effect relationships Rehearsal A system could change people's behavior through users rehearsal of a target behavior Praise A system that uses images and words to provide positive feedback to the user Rewards A system that provides a reward to the users Reminders To remind users to carry out a target behavior Suggestion A system that provides suggestions to help users to achieve a target behavior Similarity The system behaves similar to user behavior Liking Being attractive for its users Social Role When the system adopts a social role, users will be more likely to use it Trustworthiness fair, truthful and unbiased system Expertise A system that is viewed as incorporating experience, knowledge and competence Surface credibility Users make an initial evaluation of system credibility based on first-hand inspection Real-world feel A system that highlights the organization or users behind its services will have credibility Authority A system that leverages roles of authority will have an improved power of persuasion Third-party endorsements Third-party endorsements, particularly from respected and well-known sources, will boost the perception of system credibility Verifiability Credibility will be improved if a system makes it easy to verify the accuracy of the content by outside sources Social learning Allowing users to learn from each other's Social comparison A system that allows users to compare their performance with the performance of others Normative influence A system should leverage peer pressure in order to increase the likelihood that a user will adopt a target behavior Social facilitation A system that allows users to discern other users performing the same behavior Cooperation A system that allows customers to cooperate Competition A system that provides competition among users to adopt a target attitude or behavior Recognition The system should offer public recognition to users who perform their target behavior Adaji and Vassileva (2016a) evaluated the personalization methods implemented using an existing process framework to examine how all the principles of the three categories (dialogue support system, primary task support and social support) were applied to a selected website. These researchers applied the PSD model to one e-commerce website, Amazon, to evaluate the persuasive principles. The research reported that all the principles in these three categories in the PSD model were successfully implemented on Amazon. However, this study can be criticized because it did not evaluate Amazon with regard to credibility support principles and did not report a detailed analysis. In addition, it did not involve experts' views since this model heavily involves usability experts. Nkwo and Orji (2018) evaluated the application of the persuasive design principles on two e-commerce websites-the western site, Amazon and the indigenous African site, Jumia-by examining how the principles in all the categories in the PSD model (social support, dialogue support, primary task support and credibility support) were applied to the selected websites. They also identified the differences and similarities between Amazon and Jumia based on how the sites have enhanced their customers' shopping experiences by applying various persuasive strategies. Researchers explained how persuasive principles that were applied effectively on Amazon were also implemented in the indigenous African e-commerce website in order to optimize customer engagement and grow sales. This research showed only one example for each feature in the evaluated websites. It again did not involve experts in the evaluation.
Adaji and Vassileva (2016b) used Persuasive System Design (PSD) to evaluate the persuasiveness of the Netflix website by assessing how it utilized the principles of dialogue support and primary task support. The results showed how these principles were executed on Netflix, but the study was limited to the principles of dialogue support and primary task support. Not evaluating the social support and credibility support principle categories on Netflix can be considered a limitation of this research. The report also did not mention the analysis of the persuasion context.
Persuasive design in e-commerce applications plays an important role in increasing sales revenue, as it attempts to convince users to purchase from the applications. The persuasiveness of e-commerce applications can be improved by providing support to users while moving them closer to the targeted behavior and reducing the required effort. All the existing models were proposed to evaluate and design a persuasive design. The PSD model has been shown to be a comprehensive model, especially in the e-commerce domain. There is a lack of studies discussing how to utilize and implement a persuasive design within the ecommerce domain. There appears to be no study currently available related to persuasive design for e-commerce in the Kingdom of Saudi Arabia. Thus, this study aimed to evaluate the use of persuasive design with the most used applications in the Kingdom of Saudi Arabia. This study used the PSD model, which is the most commonly used model within the e-commerce domain.

Research Design
This research followed an experimental approach where a checklist was developed based on the PSD model. The data was then examined by a number of usability experts in order to offer more valid results. To further improve the process, findings were sent to a different set of usability experts to conduct the assessment. The following sections describe the research steps in detail.

Persuasive Design Principle Model
In this study, the Persuasive Systems Design (PSD) model was used as defined by (Oinas-Kukkonen and Harjumaa, 2008). A checklist of 28 persuasion principles was created that was explained and developed based on the PSD model (Table 1).

Selected Applications
There are several e-commerce applications available in iTunes (Apple, 2019) and Google Play (Google, 2019). This study focused on the three most used ecommerce applications in Saudi Arabia based on the rankings in SimilarWeb, a website that provides statistics on the most downloaded applications from a variety of app stores (e.g., Apple Store, Google Play), as well as statistics about countries and types of application (e.g., shopping) (SimilarWeb, 2018). The study was also concerned with the functional features that were available on the selected applications. After examining the options, the selected applications for this study were (SHEIN Group Ltd, 2018;ASOS, 2018) and NOON (Noon E Commerce, 2017).

Expert Selection Criteria
According to past literature, the recommended number of expert reviewers was two to five when using the expert evaluation method (Hall, 2017). In addition, the experts needed to have knowledge and experience in the usability field, persuasive design and the e-commerce domain. It was highly recommended to utilize five experts in order to provide a variety of insights and paint a more complete picture (Hall, 2017).

Pilot Study
The aim of a pilot study is to perform preliminary studies to investigate whether assessments can be conducted in the way that has been planned (Cadete, 2017). The 28 persuasive principles provided by (Oinas-Kukkonen and Harjumaa, 2008) were categorized into four groups (primary task support, system credibility support, dialogue support and social support). To evaluate these principles of the PSD model, a checklist template with a description of the principles was provided to two experts who were asked to review the given checklist and report whether they faced any issues. All the comments received from the experts were reviewed by the authors and used to update the assessment procedures and materials where appropriate.

Experimental Procedure
To begin the process, five experts were chosen based on their experience with design and the e-commerce domain. Researchers then contacted the experts via email and provided them with three attachment files. The invitation file contained a statement of the study's aim and provided the individuals with links to the selected applications, a description of the process and contact information. An additional file was provided as a guideline for performing the evaluation correctly. This file included explanatory and illustrative examples for each principle. Lastly, an evaluation file contained an evaluation form checklist for performing the assessment. The experts assessed the applications against each principle independently and provided their comments about how each principle was applied. Then, the data was gathered, which included the ratings and comments from the experts.

Data Analysis
This section discusses the general findings for the SHEIN, ASOS and NOON applications. It also explores the most interesting findings while applying the PSD model including more details for each application. Table 2 presents the general findings from the experts' evaluation results for the selected applications. It illustrates that SHEIN has adopted the highest level of the PSD principles among the three selected applications, whereas ASOS saw the lowest number of adopted PSD principles. A total of eight principles were reported as highly supported by SHEIN and the majority of the remaining principles were moderately supported in SHEIN. In contrast, only three principles were reported as highly supported by NOON out of the 28. In ASOS, the majority of the principles were minimally supported and only five principles were reported as highly supported. As explained earlier, the 28 PSD principles have already been classified into four categories (primary task support, dialogue support, credibility support and social support). Figure 1 shows that the primary task support principles were the most commonly utilized, having been implemented by all the selected applications, representing more than 41%. Therefore, in this study, it can be claimed that the three applications have implemented 41% of primary task support principles. This category was followed by the credibility support and dialogue support principles, which were implemented by 30 and 31%, respectively. Social support principles were the least often implemented at only 11%.
Furthermore, the findings report that SHEIN had the highest implementation of primary task support among the selected applications. More than 75% of the primary task support principles were implemented in SHEIN. Further, 66 and 58% of the primary task support principles were adopted by ASOS and NOON, respectively. This data demonstrates that primary task support was considered as the primary task in all the selected applications. As a result, these applications supported their users to clearly achieve the target behaviors. Among those principles, the tailoring and personalization principles appeared to be more supported in ASOS, while self-monitoring was seen as the least supported principle. Rehearsal and tailoring were found to be the principles that were most adopted in SHEIN and NOON. In contrast, the simulations principle was reported to be the least applied principle in SHEIN and NOON. However, ASOS employed a more convincing use of the simulation principle by allowing users to watch a 10-sec video about the items. This tool helps users since it allows them to view a virtual catwalk that is similar to a real one.
In addition, the results showed that the implementation of dialogue support principles was more focused on achieving increased interaction between shoppers and sellers. The findings demonstrate that dialogue support principles were highly utilized in SHEIN (61%), which was higher than what was noted in other applications. The suggestion principle was found to be the most supported in ASOS and NOON, whereas reword was the highest supported principle in SHEIN. In contrast, the reword principle was applied the least in ASOS and the liking principle was the least applied in NOON. Credibility support, which is also referred to as believability, had a significant influence on persuasion in website design. The results showed that SHEIN implemented almost 58% of the credibility support principles. The expertise principle in credibility was found to be the most frequently utilized principle in all the selected applications. In contrast, the selected applications showed limited utilization of authority and verifiability and these principles could poorly affect sellers' credibility. The social support category clearly contained the least commonly utilized principles in the selected applications. The findings show that SHEIN (47%) was the application that most applied the social support principles. In contrast, ASOS and NOON showed a minimum utilization of social support principles. Normative influence and social learning were the least commonly used social support principles in the selected applications.
Moreover, Table 3 presents the top five principles that were implemented in SHEIN, NOON and ASOS from among the 28 principles based on the experts' evaluations. Their results reported that the rehearsal principle was adopted in all the selected applications as it allows users to search for and browse items and view the product descriptions/images, without having accounts. In addition, the expertise principle was found to be one of the most implemented principles in all three applications. The tailoring principle was also reported as one of the top principles, since it allows users to change the language and select a preferred payment method. In addition, the reduction principle enables users to sign up and log in with their social media accounts, such as Google and Facebook. Moreover, this principle is found to help users pay using Apple Pay or cash on delivery, instead of entering card payment details. This feature increases its value to users and reviewers, making it one of the most frequently implemented principles in the selected applications. The tunneling principle is also implemented in several ways. For example, ASOS provides a fit assistant feature that is aimed at helping the users enter their weight, height and other details in order to calculate their perfect size. It is also supported in SHEIN by giving the users the option to mark shipped products as delivered and review the products. Additionally, tunneling is employed to allow the users to join free trial programs.
Furthermore, Table 4 shows the five principles that were implemented the least in the selected applications based on the experts' evaluations. Although authority was implemented in SHEIN, it was their least supported principle because it was only utilized to display products by famous people through videos and images, which did help to enhance the persuasion. Moreover, the experts did not find the implementation of the social learning principle in ASOS. It was used in NOON and SHEIN in limited situations, such as when providing a rating for products and when a user learns from the reviews of other users. Experts found that social facilitation did not apply in ASOS and NOON, but it was adopted in SHEIN through providing a product review section that made the process of purchasing faster since users were influenced by those products which had good reviews from other users. The cooperation principle was one of the least applied principles in ASOS and NOON, but it is demonstrated in SHEIN when a user cooperates in product or outfit reviews. In the experts' evaluation results, the reword principle was observed as the principle least applied in ASOS, even though offering rewards for users does help to affect their loyalty and encourage them to purchase online.
Generally, the results found a limited application of authority and verifiability principles in selected applications, even though they could have considerably helped to achieve persuasion. This is consistent with the findings of (Alhammad and Gulliver, 2014), who reported that top websites showed limited utilization of authority and verifiability. In addition, the results referred to a weakness in the implementation of the simulations principle in the dialogue support category. This is in line with (Nkwo and Orji, 2018), as they also reported that the implementation of the simulations principle was not taken into consideration in the targeted websites in their study. Moreover, (Alhammad and Gulliver, 2014) clearly stated that e-commerce applications did not pay attention to simulations, although they have an important role in website persuasiveness.
The literature states that the implementation of dialogue support in the e-commerce domain enhances persuasiveness by providing the support needed for effective interaction among buyers and owners, as confirmed and clearly reported by (Alhammad and Gulliver, 2014). The dialogue support principles were most highly utilized within SHEIN more than the other applications. For example, SHEIN allows shoppers to write reviews about their products to benefit other users. In addition, this study found high utilization of primary task support, credibility support and dialogue support. However, it was reported that there was a clear lack of implementation of social support principles, although they were expected to influence a user's behavior. This fact is supported by (Alhammad and Gulliver, 2014), who reported weaknesses in the implementation of social support principles and wider use of dialogue support, than credibility support and primary task support, when they evaluated 10 successful e-commerce websites.

Recommendations and Further Work
This section aims to provide recommendations for designers and developers about the importance of the PSD model in helping to achieve persuasiveness in ecommerce websites and applications. These recommendations can be considered as important design rules and guidelines for e-commerce websites as well: (1) Designers should consider applying persuasive design guidelines such as the PSD model, which serve to influence users' behavior by getting them closer to a target behavior and play a major role in achieving effective interaction in e-commerce applications (2) Reducing a user's cognitive load is vital to preventing the user from feeling overwhelmed and unsure of his/her next step. A simple user interface that utilizes the reduction principle can clearly show where the user needs to go in order to accomplish tasks and reach targets (3) Designers should focus on emphasizing credibility support principles as they are the keys that directly and indirectly influence users' purchase decisionmaking by focusing on building credibility and trust (4) Designers should implement primary task support principles in an interactive way, which helps to enhance users' self-efficiency during their shopping experience. For example, ASOS supports users through their shopping experience by decreasing the required effort and getting them closer to the target behavior and by allowing searches using an image. This helps users find related products more quickly, in a convincing way (5) Designers should focus on implementing dialogue support principles that help to enhance the persuasiveness of the applications and lead to achieving effective interaction between buyers and sellers (6) It is recommended that NOON and ASOS take into serious consideration the application of dialogue support principles. In ASOS and NOON, the user needs to have ways to contact the seller in real-time, such as applying a chatbot technique (7) ASOS and NOON should invest in the advantages of applying social support principles, which SHEIN has highly utilized. This plays a role in achieving system persuasiveness by influencing a user's behavior based on the behavior of other users (8) The applications should focus more effectively on the importance of the review section for users, which allows users to rate and write reviews about products that will help other users learn from their experiences. This feature is considered one of the vital persuasive design principles (9) SHEIN and NOON should employ the simulation principle, which has a high probability of improving persuasion in the e-commerce domain (10) SHEIN should focus more attention on the recognition principle in a more effective and convincing way A few limitations of this study must be identified. First, this study investigated the influence from an expert point of view, without the consideration of the users. Surely, investigating these principles from the users' perspective may reveal additional enlightening results. Second, further examination of the importance of these guidelines from the point of view of users or experts can enrich the data analysis; such findings may help developers and business owners to prioritize which categories should be implemented or taken into consideration first. Third, examining actual user behavior during the use of the applications may offer more accurate results regarding how such principles affect their behaviors. Despite the limitations described above, it is argued that the results of this study contribute considerably to understanding the persuasive architecture of e-commerce applications. The results are considered to offer a foundation for future research investigating applications as user persuasion tools. Thus, it is expected that the present study will open a new direction for research on persuasive design in e-commerce from different perspectives.

Conclusion
Persuasive design plays a role in increasing revenue and the ability to keep and encourage customer loyalty. This paper reported on the evaluation of the persuasion of selected e-commerce applications in Saudi Arabia in order to guide designers in building successful applications or improving current ones. The PSD model was used in conjunction with expert evaluations to examine the persuasion of selected applications. This study found a wide use of primary task support, then credibility support and dialogue support and weakness in the implementation. The social support principles were the least utilized principles within the selected applications. Also, the results found limited utilization of the authority and verifiability principles, which may encourage researchers to further investigate a variety of ways to achieve these principles in the e-commerce domain.

Ethics
This article is original and contains unpublished material. The corresponding author confirms that all of the other authors have read and approved the manuscript and no ethical issues involved.