A Technique for Quality Evaluation of E-Learning from Developers Perspective

: Problem statement: E-learning is gaining more acceptance as days pass because it provides learning opportunity any time and in any place. Different people have different preferences in terms of learning style such as reading text, listing audio or video, speaking and communication. To support these different learning preferences, there is need for multiple e-learning delivery methods and teaching techniques. Furthermore, there are many stakeholders of e-learning systems such as system developers, administrators, instructors, instructional designers, multimedia designers, online facilitators, independent evaluators. Whose views are important indicators for a complete e-learning system evaluation, but the most important views of e-learning quality are user view, developer’s view and manger’s view. Approach: The main aim of this study is to propose a new technique to evaluate e-learning website quality from developer’s view. To achieve our objective an extensive study on related resources was conducted. Our technique adopts the weights of quality characteristics which are obtained by carefully selected questionnaires’ from professional developers. We also present the evaluation process using AHP technique and the result of trial evaluation for validation of our technique Result: We proposed four quality characteristics named Service Content, System Functionality, Information Technology and System Reliability. We further, proposed 11 sub-characteristics with its attributes by following the structure of standard IOS/IEC 912. Conclusion: Our results show that the proposed technique could be useful and effective for ensuring that high quality systems are developed.


INTRODUCTION
The growth of e-learning systems has increased greatly in recent years, due to the significant advantages of e-learning such as convenience, portability, flexibility and global learning community. Therefore, On-line learning is becoming essential for many realworld tasks, as such there is economic pressure on educational institutions for more flexible learning and cost saving options. Yet, there has been considerable criticism of the quality of the systems currently being used. According to (Baruque et al., 2007) many organizations are still experimenting with e-learning, using different approaches, applying different technologies and models for the delivery of e-learning contents. Hence, how to ensure a quality of e-learning website is still not clear. E-learning web site quality is a complex concept and its measurement is expected to be multidimensional in nature.
E-learning incorporates organizational, administrative, instructional and technological risks. This makes the evaluation process complex. The evaluation complexity is caused by the large amount of intervening characteristics and attributes and by the complex logic relationships among attributes and characteristics. Although a lot of study has been done regarding the management, planning and implementation. There are still big differences in the produced material and a lack of an effective e-learning evaluation quality model (Baruque et al., 2007;Ozkan and Koseler, 2009;Caramihai and Severin, 2009;Chua and Dyson, 2004;Yunus and Salim, 2008). Furthermore, there are many stakeholders of e-learning systems such as system developers, technicians, administrators, instructors, instructional designers, multimedia designers, online facilitators, independent evaluators. Whose views are also important indicators for a complete e-learning systems evaluation (Ozkan and Koseler, 2009) but the most important views of elearning website quality are; user view, developer's view and manger's view.
At least to the best of our knowledge no prior study exits specifically on evaluation method of e-learning from developers view point. In this regard this study intends to contribute.
Related study: There are many terms used to refer for online education. Such as education, Internet-based education, web-based education and education via computer mediated communication. However, the term e-learning is often used as a more generic term and as synonym for online education. According to (Ozkan and Koseler, 2009;Jabr and Omari, 2010) electronic learning refers to the use of electronic devices for learning, including the delivery of content via electronic media such as Internet, audio or video, satellite broadcast, interactive TV, CD-ROM and so on. There are a great number of different kinds of quality requirements (Kitchenham and Pfleeger, 1996;Cote, Suryn and Elli Georgiadou, 2003) The different types of quality requirements require quite different types of analysis methods. A quality model can be considered to be a framework for unifying and quantifying different viewpoints as mentioned in (Kitchenham and Pfleeger, 1996).

Types of E-learning:
No single e-learning method is best for every learning need. You will most likely need to use several e-learning technologies as well as traditional learning methods. E-Learning comes in many variations and often a combination of the following: • Asynchronous E-Learning: Learning where people are not online at the same time and interaction does not occur without a time delay, allowing people to participate on their schedules such as: • Self-study (Self-study with subject matter expert) • Discussion Groups • Computer-based (CD-ROM) (Video/audio tape) • Synchronous e-Learning: Learning where people are online at the same time such as: • Virtual classroom • Audio and video conferencing • Chat • application sharing • Blended learning-combination of online and faceto-face: We conducted extensive survey on quality evaluation methods proposed in the literature, (McCall, 1979;Pruengkarn and Srivihok, 2005;Baruque et al., 2007;Ozkan and Koseler, 2009;Wang et al., 2007;Chua and Dyson, 2004;Yunus and Salim, 2008). Also extensive review was conducted on e-learning system (Jabr and Omari, 2010;El-Sofany et al., 2009;Ghaleb et al., 2006;Harbouche and Djoudi, 2007;Hirzallah, 2007). According to IEEE Standard Glossary of software engineering terminology, software quality is defined as "the degree to which a system component, or process meets specific requirements or meets customer or user needs or expectations". In 1991, the International Organization for Standardization introduced a standard named ISO/IEC 9126; software product evaluation-Quality characteristics and Guidelines for their use. The ISO/IEC 9126 model was constricted in response to the search for universal quality model. It is based on Mc Calls model (McCall, 1979). ISO/IEC 9126 is a generic Quality Model, which can be applied to any software products and is very difficult to apply to specific domains such e-learning website, due to the special feature of elearning website (Chua and Dyson, 2004;Britain 1999). The new quality standard ISO/IEC 19796-1 was published in October 2005. It provides Reference Framework for the Description of Quality approaches (RFDQ). In the future, ISO will publish ISO /IEC 19796-2 as standard to certify organizations, services and products. And will provide orientation for all stakeholders. ISO /IEC 19796-3 reference methods and metrics for quality management and assurance. ISO /IEC 19796-4 guidelines for the adaptation implementation and usage of this multi-part standard and will contain a rich set of best practice examples.
In (Baruque et al., 2007) the authors concerned with applying the governance principles in e-learning, based on risk oriented approach to control standard as function and benchmarking reference. The authors of (Ozkan and Koseler, 2009) proposed a multidimensional approach to Learning Management System (LMS) evaluation via the following six dimensions: Information quality Service quality intention to use user satisfaction net benefit (Shee and Wang, 2008) Learner interface Learning community System content Personalization (Wang et al. 2007) System quality Information quality Service quality System use User satisfaction (Ozkan and Koseler,2009) system quality information quality Service quality instructor attitudes learner perspective Supportive issues Wang and Wang (2009) System quality Information quality Service quality User satisfaction Intention to use Net benefit Key: QF1= Quality Factor 1, QF2 = Quality Factor 2, QF3 = Quality Factor 3, QF4 = Quality Factor 4, QF5 = Quality Factor 5 (1)System quality, (2) service quality, (3) content quality (4) learner perspective (5) instructor attitudes and (6) supportive issues. The finding of this research showed that, each of the six factors had significant effect on learner's satisfaction. Many authors discussed evaluation of e-learning system from pedagogical prospective in (Balasundaram and Ramadoss, 2007;Pruengkarn and Srivihok, 2005;Yunus and Salim, 2008;Britain, 1999;Wang and Wang, 2009). Authors in (Ardito et al., 2006) proposed a good methodology, called eLSE (e-learning Systematic Evaluation) based on inspection technique. And in (Hammami, 2010) proposed a methodology for evaluating and comparing of websites quality.
In most of the related study the authors were mainly concerned with five dimensions of the quality of e-learning sites namely: System quality, information quality, service quality, attractiveness and their relationship. Table 1 presents a sample of such previous study.

MATERIAL AND METHODS
Research design: To achieve our objective an extensive study on related resources (Abdullah and Wei, 2008;Kian et al., 2003;Delone and McLean, 2003;Franch and Carvallo, 2003;Yunus and Salim, 2008) was conducted. The following methodology has been identified as shown in Fig. 1.

Case study:
Step 1: Design a questionnaire for collecting data from developer, who are developing e-learning websites Step 2: Assign points and assign the value for each attributes, which are obtain from questionnaires Step 3: Compute the attributes value, subcharacteristics value and the value of Quality Characteristics (QC) Step 4: Compute the Total Quality (TQ) of e-Learning website Particularization quality evaluation technique for eleaning website: A good cooperation between instructors and developers has the effect of increasing the quality of e-learning website, hence the perceived learners' satisfaction of e-learning website (Ozkan and Koseler, 2009). This is because the instructor knows the various pedagogical functions that the system had to support, which software developers may not know. The software developers generally view their design purely from technical point of view, normally based on the ISO model. But, the ISO model alone would be insufficient because it is a general software quality model and does not specify a particular teaching and learning activities needed for good learning (Chua and and Dyson, 2004) (Britain, 1999). This study attempts to identify e-learning quality requirements that can help in meeting customer needs. The quality characteristics and attributes of the proposed technique are presented in table 2. We have generally categorized e-learning quality requirements into three levels. The first level includes core services, while the second level includes those which facilitate core procedures and the third level includes factors support level two.    (Ozkan and Koseler, 2009;Chiu et al., 2005). Authors in (Ardito et al., 2006) defined e-learning platform as the interface environment which may offer a number of integrated tools and services for teaching, learning, communicating and managing learning material.
To be effective course management system should have a means of making necessary announcements on time. Additionally, it should consist of a module that contains structured exam. Calendar is another important attribute; it can be use for checking upcoming sessions, viewing available courses and to make appointment with teachers or students. Student tracking: to report the learner's performance within a course to a Learning System. In addition tracing allows for both "sequence" and "student" views of real-time progress. Finally, there should be a Teacher and User profile: to present personal information regarding user. We included this sub characteristic to ensure that our model conform to the earlier study of (Paechter et al., 2010) which presented it as part of five fields of instruction requirements for designing and management of Elearning course.
People learn in different ways and at different times. Different people have different preferences in terms of learning style such as reading text, listing audio, doing assignments and reviewing exams or speaking and communication. To support these different learning needs, there is need for multiple elearning delivery methods and teaching techniques. Attributes that can be grouped under teaching technique include; self Study method, online experts, feedback, recommended resource, archives and bookmarking technique to allow the learner stop the course at any time and restart it from the same point. Other issues when delivering courses can be grouped under service quality, which includes Print, video, audio, communication. (Arbaugh, 2002) stated that flexibility of the delivery of courses comes as a result of the medium being both place and time independent. System Functionality: System Functionality consists of browsing issues, usability and supportability. According to ( Larisa, 2002) study shows that 66% of webmasters do not add a search engine to their site. Reasons for not doing so included time, complexity and cost. Therefore, we ignored search tools from our requirements. Nowadays, there are many known properties that make website simple and easy to use. In a learning site, it would be good to create a browsing system that has a high link relevancy, means of scrolling and good structure for sub-sites and labels (current, next and previous) position that points visitors directly to each important section. A good navigation system should answer three questions: Where am I? Where have I been? Where can I go? However, visitors appreciate search capability on larger sites, e-commerce sites and any site that deals with several different products or topics.
Different methods can be used for evaluating the usability of web system (Abdullah and Wei, 2008;Kian et al., 2003;Chiu et al., 2005;Ardito et al., 2006) When usability issues are considered at the design phase, greater returns are possible to both user and developer. Attributes that can be grouped under Usability sub-characteristic site map should be included; this reflects navigation and main content areas. Content maps: Detailed maps that show what exists on each page and how contents on the page are related. Instructors must perform a variety of tasks in the process of teaching, such as give feedback of accomplishments, help, features, e-mail directory, Phone-fax directory and web-site last update. Which assist them to engage in learning activities (Paechter et al., 2010).
Another important sub-characteristic we recommend contains supporting or auxiliary components, these together are termed supportability. Such components like digital libraries are nowadays critical to both students and teachers. This is because they reduce the cost of buying books, journals and other essential day-to-day needs of teaching and learning. Virtual labs should also be included. Since most often visitors to e-learning site come from different part of the world, there is need for the site to provide at least the essential parts in multiple languages to improve its accessibility.
Information Technology: Information Technology characteristic consists of communication technique, security technique and interface issue. The quality of communication technique depends on employing various means of communication, such as electronic mail, online threaded discussion boards, synchronous chat, desktop video/audio conferencing and virtual classroom. We also recommend a facilitator to assists directs and stimulates the learning during an online course. This is because interaction with peer students and the exchange of information between students and instructor supports knowledge construction, motivation and the establishment of a social relationship (Paechter et al., 2010;Shee and Wang, 2008). Thus those who study have the possibility of asking questions and receiving answers. The effects of information technology may decrease or increase the overall quality of e-learning site, depending on how new advances in communication methods and techniques utilized (Wang and Wang, 2009). E-learning courses need to be accessible to different end users, so there is need to present study materials in different forms and format like CD ROM, DVD, or HTML (Fitzroy, 2007). To make sure that, information remains confidential e-learning system must provide a good mechanism of ensuring privacy and without affecting accessibility.
Certainly, to retain users' patronage, the site has to be attractive and should be consistent in presentation. This consistency is called style uniformity. It makes users to feel at ease wherever they might be in the site; hence it is expected to improve their satisfaction. Consequent upon this is to have cohesiveness to group Main control objects.
System Reliability: Reliability is one of the most important characteristics of assessment of any system. A broken link or a misspelled word may seem a trivial mistake, but it can greatly undermine the credibility of a website. If there is need for people visiting a site to be assured of the quality of the information in the site, different components of the site must be compatible. Therefore, there is need to use an authoring tool that will facilitate.
Certainly, to provide a reliable site, there is need to ensure that the performance at least as it is perceived by user is acceptable. Ensuring that performance remains at acceptable levels is function of many variables. Some of the important considerations we isolate and recommend are: Bandwidth must be sufficient in relation to the size and the number of visitors (Chaabene et al., 2007). It will also help, in ensuring that user perception of the site remains positive, if pages are not cluttered with too much unnecessary graphics and multimedia. Therefore, features such as complementarily between sound and image, choice of media with respect to content are valuable.

Weight decision technique for quality: A case study:
To decide weights of quality characteristic, we selected sample of professionals working on e-learning project in Center of Information and Communication (www.idec.upm.edu.my), university Putra Malaysia. Center of Information and Communication began on July 1, 2006. It was established through restructuring of Information Technology, Institute of Multimedia and Software.
Using a questionnaire that we designed, the respondents were requested to give their preferences on these characteristics and sub-characteristics proposed in our technique. The respondents have varied experience ranging from over 10 years (50%), between 6 to 10 years (38%)and (13%) have 2 to 5 years experience. Their careers related with e-learning website are distributed as follows: 75% developers, 13% are mangers and 13% in others careers.
The Analytical Hierarchy Process (AHP) is one of the extensively used multi-criteria decision making techniques. The AHP is aimed at integrated different measures into a single overall score for ranking decision alternatives. Responses received from these professionals are analyzed through AHP approach. In order to, decide weights of quality characters and later to compare between two virtual website (website A and website B). Moreover, Expert Choice is used as tool to support AHP and MS-Excel is used to process general data such as experience and careers qualification of respondents.
The weight values for each main characteristic in the range from 0-1. The sum of all weight value is 1. When we synthesized all the elements using Expert Choice, we obtained the results shown in (Fig. 2). (Fig.2) shows that in e-learning domain, reliability is the most important characteristics, while information system characteristic is the lowest important. However, as characteristics and involving important subcharacteristics, this lowest value cannot be ignored and has to be considered while evaluating the overall quality of e-learning system.
Further, weights of sub-characteristics are decided by AHP analysis. We compared in pair-wise subcharacteristics with respect to the characteristic. It may be noted from (Fig. 3) that the weight value for Teaching Technique sub-characteristics is the highest. This means the design technique of attributes such as self study, archives and recommended resource are valuable in e-learning system. The weight values obtained through the process mentioned above could help a developer to give high important stress to those quality characteristics and subcharacteristics. It could also help to assign the most experienced developer to the most important aspect. Consequently, it will control the development time and financial trade-off.  In order to prove the applicability of the proposed methodology, two virtual e-learning "site A" and "site B" were employed as illustrative alternatives. The judgments of the two virtual sites with respect to each criterion were rated on a 1-to -10 scales. In order to analyze the impact of change in the input data or parameters of proposed technique, sensitivity analysis was conducted using Expert Choice software. Figure 4. shows overall performance of these two website with respect to all characteristics. Use the "left y-axis" to read each characteristics priority. Use the "right y-axis" to read website priorities. For example, performance of sit A with respect service content, system functionality, information technology and reliability is 0.270.25, 0.58 and 0.40% respectively. Overall performance of site A can then be acquired by calculating average performance of its characteristics, which equal 0.38%. The results reflect that site B appears to be performing better than site A. But if we consider information technology characteristic as criteria site A appears better.
Each sensitivity analysis can be performed from the goal or from selected characteristic. For instance, Fig. 5 shows performance sensitivity graph with respect information system technique characteristic.

CONCLUSION
There are many stakeholders view of e-learning systems, but the most important view is developer's views. Therefore, it is critical to evaluate e-learning from the perspective of developer and to provide recommendations where necessary. In this study contributes, we proposed four quality characteristics named Service Content, System Functionality, Information Technology and System Reliability. We then proposed 11 sub-characteristics with its attributes by following the structure of standard IOS/IEC 9126. Our results show that the proposed technique could be useful and effective for ensuring that high quality systems are developed. Furthermore, this technique would enhance the relationship between instructors and developers. However, we need further study for general application to real projects and is our firm believe that the result of the study will contribute to the effectiveness of e-learning; directly increase the satisfaction of students and instructors.