© 2010 Science Publications Evaluating E-Learning Systems Using E-Traceability Systems

Problem statement: Electronic learning (e-learning) System is an ideal example of these modern learning options. It provides reliable, convenient and easily accessible environment of learning. Since the e-learning is now growing in King Saud University, this new learning approach has to be evaluated and monitored to measure the efficiency and ultimately improves its productivity. The monitoring of the e-learning applications deserves special attention and we need effective methodologies and appropriate guidelines to perform this task. Approach: The development and implementation of an effective and appropriate evaluation approach based on different types of information and analysis of reports offered by an E-Traceability System (ETS) in order to improve and enhance the e-learning system and its performance to make sure it performs its mission as desired. Results: The E-traceability system has been developed, implemented and extensively tested and the evaluation can be done with automatic and easy way. Conclusion: The outcomes of the tests are relevant information delivered by the system concerning any user's actions in a standard reports format represented as graph diagram. These reports are related to the following main evaluation questions are being performed and they pertain to measure the popularity of the e-learning system, engagement of the participants and interactions between them and it can be used in support services and decision making.


INTRODUCTION
E-learning represents one of the online education environments that use modern information and communication technologies to provide a very powerful tool for the development of the new society and to keep up with changes in the global economy that now occur in Internet time. However, the benefits of such systems can only be ensured if it is able to actually transfer the knowledge to the learner and this ability must be influenced by its level of functionality, usability and quality of learning by facilitating access to resources and services as well as remote exchanges and collaboration. For the above reasons, the monitoring and evaluating of e-learning systems deserves special attention and we need appropriate evaluation methodology to gather breadth and depth information about the impact or effectiveness of this e-learning event (Zaharias et al., 2002). Evaluating the usability of e-learning applications is not a trivial task (Zaharias and Poylymenakou, 2009). Using approach for the monitoring, we can measure the popularity of the elearning system, engagement of the participants and interactions between them and it can be used in support services and decision making. Therefore, it is important to investigate in the measurements because they might be used to improve the offering, determine if the elearning objectives were achieved, or determine if the offering has been of value to the organization (Zaharias et al., 2002) and help in improving the quality of e-learning and in informing and shaping future development in policy and practice. The most elearning monitoring and evaluating system based on questionnaires which have been widely used for both evaluating affect and usability of interactive systems (Zaharias and Poylymenakou, 2009). However, these methods require a high level of collaboration (Fetaji and Fetaji, 2007) and they use various indicators which are not standardized (Rubio, 2003;Barbara, 2002;Malataras and Pallikarakis, 2007;Yunus and Salim, 2008;Cardona, 2009). Therefore, to make e-learning more effective and efficient some useful techniques are used to implement and design an evaluation approach which is truly based on some factors which are evaluated thoroughly like content related factors, communication related factors (Ketabchi et al., 2008). Each approach considers indicators according to their context and in some cases in the same study, units of measurement are different, hinders their analysis (Ga-Jin, 2007).
Criticisms of these approaches are many. They are sophisticated in their design but they have lack guidance of 'objectivity' and their interpretation and analyses are unpredictable. For example, Posea et al. (2006) presented an evaluating method based on two different types of measurements, logs and social networks. The method is based on actual usage of platform by analyzing the users' behaviors (time, frequency aspects and logs). This analysis aimed to improve the platform and fix the drawback. One function of this method is the average time per session that related to number of logins that reflect the popularity of the platform by computing number of site access, but this depend on the platforms itself, because some platform load all course materials once in the beginning of the course, so the students login once and download the materials, while the other load the course materials incrementally, so the students need to access the periodically. Klassen and Smith (2004) presented a method to evaluate instructor online. They design a evaluation site and the student should be login to this site in last two week of the semester and answering for specific question according the department of each course, then the system calculate the result of the evaluation. Shoikova et al. (2004) described the evaluation approach basically on questionnaire concerning the e-learning architecture, the accessibility of course materials and flexibility of delivery for the users and does the architecture reach the usability criteria and satisfy users' needs in carrying out their specific tasks. In (Ab Hamid et al., 2006) the electronic learning system is evaluated for three main components: the usability evaluation to cover the Graphical User Interface, easy to use, helpfulness and alertness; the fulfillment evaluation to examine the features incorporated in terms of reusability, interoperability, durability and accessibility; and the overall satisfaction. (Ganchev et al., 2007) described some important E-learning tools for evaluation of elearning system. Mainly it was divided into main groups "General Criteria" and "SCORM Compliance Criteria". First group mainly concerned about general criteria for a software product for example cost of ownership and technical Support, Hardware and Software requirement, product documentation and source and support, while the second group has specific criteria for tools with existing learning object like learning content metadata, Learning content packaging and sequencing. Specifically ten e-learning tools are applied for Evaluation. So after a stabilized evaluation of e-learning tools "Reloader editor" tool shows best results because of its flexible capabilities. In the case of University of Tehran (Ketabchi et al., 2008) as study was conducted to assess the creation and offering of e-courses and evaluate the impact of the students' access by applying some e-courses. Initial framework for evaluation was presented in twenty four points but after thorough review of Experts seventeen indicators to measure them. These indicators are mainly categorized into 4 groups: Content-related factors, Communicativerelated factors, Evaluation-related factors and LMSrelated factors. After Evaluation result Content-related factors are of user satisfaction and Communicativerelated factors and Evaluation-related factors are fair while LMS-related factors are very little. (Sahin, 2007) used data collected using the Distance Education Learning Environments Survey (DELES) to explore the relationship between student satisfaction and some predictor variables. In this study, descriptive statistics and correlation analysis were used. These results show that a higher level of each DELES scale indicates a higher level of student satisfaction from distance education. But, the variables of this study are limited to the ones described in the DELES. (Ardito et al., 2006) proposed an empirical validation methodology, called eLSE (e-Learning Systematic Evaluation), which mingle a precise inspection technique having less experience in evaluating e-learning system to perform accurate evaluation. It is based on the use of evaluation patterns, called Abstract Tasks (ATs). On the basis of these activities AT formulate pattern template which are (i) AT Classification Code and Title univocally identify the AT and its purpose; (ii) Focus of Action lists the applications objects to be evaluated; (iii) Intent clarifies the specific goal of the AT; (iv) Activity Description describes in detail the activities to be performed during the AT application; (v) Output describes the output of the fragment of the inspection the AT refers to. The AT and Heuristic inspection technique has been evaluated on he basis of Content, Graphical Design Technical Problem, Feedback, Navigation, but still this technique has problem of reliability and usability. Daniel and Wang (2008) presented a methodology based on learner satisfaction and its applications, where the investigation carried out a survey of college students and the data thus obtained was then analyzed by analytic hierarchy process in order to derive an integrated preference structure of learners as a ground for evaluation. We found that learners regarded the learner interface as being the most important dimension of decision criteria.
The monitoring of e-learning applications deserves special attention and we need effective methodologies and appropriate guidelines to perform this task (Zaharias et al., 2002). According to the above, the purpose of this research is to develop an evaluation approach based on an E-Traceability System (ETS) in order to observe users' learning activities in real-time by monitoring the major possible number of behavioral aspects and personal traits. The International Organization for Standardization (ISO 8402:2105) defines traceability as being the "ability to trace the history, application or location of an entity by means of recorded identification". Even though this definition looks quite wide, we can specify the ETS to provide us relevant information for both evaluating affect and usability of interactive systems. It allows us to improve the traditional way of evaluation and defines tools which are flexible in use but consistent in results that save the time and doing it automatically. Using the ETS, we can evaluate the full course cycle from course design to course validation and we can measure the popularity of the e-learning system, engagement of the participants and interactions between them and it can be used in support services and decision making.

MATERIALS AND METHODS
Our work aims at contributing to the enhancement of online education system by defining an E-Traceability System (ETS) based on a quantitative method for the evaluation of the level of functionality, usability and acceptability of e-Learning systems. The ETS is based on the breadth information (frequency of access to the platform by users) and depth information (kind of activity for each access). It gives relevant information concerning the standards of design and development and program delivery and provides a relatively easy approach for both evaluating affect and usability of interactive systems. Using the ETS, we can evaluate the full course cycle from course design to course validation and included authors, tutors and students. This is involves the concept of interactivity: how students interact with learning materials, with the teacher and with peer learners. The advantage of interpreting these data in real time lies in gaining prompt information about the user's state. However, a full understanding of e-learning situation requires measurements across two main domains which can determine the quality of e-learning in university: Design and development and program delivery which they are directly related to the nature of the online activities users are engaged in and hence also to the level of engagement they have in the activities.
The ETS summarizes the amount of information to be recorded concerning the question about popularity and about the nature of the online activities for each course and highlights the different information which can be collected to see to what extent the teachers and students have been engaged in the various types of activities and how far we can go backward/forward to retrieve relevant information delivered by the platform concerning any user's actions. The ETS has been designed to provide reports in a standard format and an easy-to-use instrument to carry out a retrospective evaluation of an e-learning system. This retrospective analysis allows the administrator of the system to detect weaknesses and strengths of their learning program with regard to its organizational, pedagogic and technological implications. These reports are related to the following main evaluation questions are being performed and they pertain to: The interface issues, pedagogical issues, information architecture issues, accessibility/delivery issues, multimedia issues and the quality and reusability of learning designs. The key function of the ETS is the generation of report activities. For this we developed functions to display reports and statistics as interactive chart representation like pie chart, bar chart and area chart. As an example, ETS generates on real-time: (i) a visiting report to show the popularity of the elearning system; (ii) reports about teachers' and students' activities include access material, access homework, access live chat, access forum and access virtual classroom; (iii) reports to compare the usability of different courses; … We will give more details in testing and validation section.
These reports enrich our understanding of the elearning system and enable us to make significant decisions.
But in some cases, using the reports' interpretation our application may be usable but not in the pedagogical sense and vice-versa, for this we are invited to add e-questionnaire design based on generic quality criteria (Zaharias and Poylymenakou, 2009;Daniel and Wang, 2008) to facilitate the analysis of reports and generate a final interpretation. Subsequently, we designed an electronic tool that allows the creation of on-line questionnaires includes observation of students about descriptions of learning objectives and content, accessibility of linked resources, inclusion of required tools, appropriateness of assessments and improvement of their knowledge and skills and provides rich feedback and statistical evidence for evaluators. This approach aims to correlate both reports generated by the ETS and the answers to the comprehension e-questionnaire. Next we will give more details about ETS but not the equestionnaire tool.

RESULTS
The E-Traceability System is based on the breadth information (frequency of access to the platform by users) and depth information (kind of activity for each access). The activities are the things that done by users when they logging on to the system. For easy design, the ETS was decomposed into smaller subsystems. Each subsystem performs a specific function. The decomposition is shown in Fig. 1 that contains four layers, Interface Layer which authenticate and determine the user type that trying to login to the system, Record Activities Layer witch collect teachers and students activities and record these activities in database, Report Layer witch its main objective is to generate activities reports and statistics and data layer to store all users' activities in the database.
Our e-learning system contains an Interface Layer with an E-learning Portal Entity (Hammami and Mathkour, 2008) that allows users to access to elearning resource according to users' kind and its permission.
When E-learning Portal Entity authenticates the user and determines its kind, it sends its information to the ETS Interface Layer. The ETS interface layer has determine user type entity (Fig. 2), this entity divided into three entities, Teacher Entity, Student Entity and Administrator Entity. Each Entity collects the information and sends it to record activities layer to register this user in visiting database and tracing his activities. The Administrator Entity allow to the administrator to generate the reports and statistics by using the Report Manager Entity. Fig. 1: Architecture of E-traceability system ETS record activities layer consists of visiting entity and activity listener entity (Fig. 3). Visiting Entity create a visitor object with information that coming from ETS Interface layer (Detriment User Type Entity) and add this visitor to visitor database. Activity Listener Entity creates an activity object with information that coming from ETS interface layer (detriment user type entity) and adds this activity to activity database.
ETS report layer (Fig. 4) has a Report Manager Entity. Report Manager Entity is goal of system users tracing, it considered as recourse of system activities report; by using report manager entity functions, system administrator can extract and generate reports about the system activities by executing queries on Visiting and Activity databases.

DISCUSSION
The ETS has been developed using Netbeans 6.7 IDE (Integrated Development Environment) platform as a development environment. Its project is focused on creating an extensible development platform, runtimes and application frameworks for building, deploying and managing software across the entire software lifecycle. The developed language, Java programming language, provides a flexible and robust environment for applications based on object oriented design. The JSP (Java Server Pages) technology is used to develop the web pages of our application and in database part we used MySQL Server. In order to prove the applicability of the proposed methodology, a pilot test started in December 2008 with teachers from Computer Science Departments of KSU and with students B.Sc Degree on E-Learning from KSU. During the test we use the ETS to illustrate the traceability of students and teacher activities based on two separate courses of study. The ETS provide reports containing information on three common areas of interest concerning e-learning system use: its popularity, nature of use and the engagement of the participants. Figure 5 and 6 show reports about the popularity of the e-learning system. These reports represented as chart graph and percentage numbers. We see in Figure 5 that students' day visiting percentage were 57.14% and teachers were 42.86% and in Figure 6 the students' monthly visiting percentage were 61.54% and teachers were 38.46%.
For each course the ETS generates students' activities reports. Figure 7 and 8 show reports about course students' daily activities and monthly activities. Students' activities include, access material, access homework, access live chat, access forum and access virtual classroom. The teachers' activities reports ( Fig. 9 and 10) are used to ensure learning is actually taking place as intended. For example, teacher needs to be selfassessment, than these reports help us to verify the learning conversations between instructor and learner. Therefore, comparing course students' activities reports and teachers' activities reports we can confirm or not that the teacher did detailed feedback for a student depending on their actions. Our e-learning system provides a mix of Arabic and English e-contents. The ETS helped us to evaluate the impact of the system on the performance of students who are using Arabic and English e-contents separately as compared to that of mix e-contents. The ETS allows us to compare students' activities between more courses (Fig. 11). We notice that the popularity of the course depends on the language of instruction.

CONCLUSION
The proposed tool allows observing users' learning activities in real-time by monitoring the major possible number of behavioral aspects and personal traits. The tool has been extensively tested. The outcomes of+ the tests are relevant information delivered by the system concerning any user's actions in a standard reports format (for this we developed functions to display reports and statistics as interactive chart representation like pie chart, bar chart and area chart). These reports are related to the following main evaluation questions are being performed and they pertain to measure the popularity of the e-learning system, engagement of the participants and interactions between them and it can be used in support services and decision making.