Input and Output Evaluation of Undergraduate Qualification Improvement Program through Dual Mode System in West Nusa Tenggara, Indonesia

Corresponding Author: Abdul Fattah, Ushuluddin and Religious Studies, State Islamic University of Mataram, Nusa Tenggara Barat, Mataram 83116 Indonesia Email: abdulfattahuinmataram@gmail.com Abstract: The focus of this study examined the effectiveness of the undergraduate qualification improvement program, for the teachers of Islamic elementary school and the teachers of Islamic Education through Dual Mode System at Mataram State Islamic University. The specified targets are viewed as input and output components. The research included an analysis of documents, interviews with Mataram State Islamic University lecturers and students, Dual Mode System organizers program and Communities. The program evaluation was found that; The input components have been running well (91.61%), the product aspect showed remarkable achievement (100%) and the outcome shows successful in creating 'different values' for its alumni. Based on the analysis, this study recommends there needs to be an increase in quantity and quality aspects.


Introduction
In Indonesia, teachers hold positions as professionals in primary, secondary and early childhood education levels on formal lines raised by legislation (Law No. 14 the Year, 2005a: Article 2 Paragraph 1, 2010). The recognition of the position of teachers as professionals as mentioned above is evidenced by the certification of educators (Law No. 14 the Year 2005b: Article 2 Paragraph 2, 2010) followed by "professional awards incentives" (Sarimaya, 2008, p. 113). This provision applies to all teachers, including teachers of Madrasah Ibtidaiyah (Islamic Elementary School-MI) and Pendidikan Agama Islam (teachers of Islamic Education -PAI) in all.
Madrasahs in the last decade of the 20th century is an alternative education institution (Nurkholida, 2017); (Subhan, 2012). Some state teachers are seconded in madrasah (Ibtidaiyah, Tsanawiyah and Aliyah) under the auspices of the Ministry of Religious Affairs due to the limited number of teachers in Madrasah who hold Bachelor degree. Therefore, the Ministry of Religious Affairs tried to upgrade the teachers' education qualification through a scholarship program for those who meet the requirements to pursue further education at the universities a place determined by the Ministry. Teachers within the Ministry of Religious Affairs who  (Fattah, 2014) with the number of state Islamic Elementary School (MIN) In the academic year of 1.686 (7.55%) and the Private Elementary School (MIS) as many as 20.782 (92.5%). The number of State Junior Secondary Schools (MTsN) of 1.437 (9.7%) and the Private Junior Secondary School (MTsS) as many as 13.320 (90.3%). Direktorat Jenderal Pendidikan Islam Kementerian Agama RI (Pendis.Kemenag, 2013), the number of State Madrasah Aliyah are 758 (11.8%), while the number of Private Madrasah Aliyah (MAS) is 5.657 (88.2%). Teachers MI and PAI ranked first regarding quantity in the 2008-2009 academic year (Nata, 2001).
The Ministry of Religion has innovated this program in a way that it would not interfere with the execution of day-to-day tasks of each teacher. The Directorate General of Islamic Studies of the Ministry of Religious Affairs of the Republic of Indonesia began academic year 2009/2010 to organize an undergraduate academic qualification program for MI teachers and PAI teachers in schools using Dual Mode System (DMS) approach. The program faced a numbe of obstacles that hinder successful implementation of the program.
To investigate the success and failure of this program, an evaluative study particularly evaluating the program of upgrading teachers' qualification through DMS, organized by West Nusa Tenggara Province in Islamic State University (UIN) Mataram, is worth implementing. This study employed an Input, Product and Outcome (IPO) model.

Input and Output Evaluation of Undergraduate Qualification
In the book Essentials for Educational Evaluation by Wandt and Brown (1957) stated that "Evaluation is defined as an action or a process to determine the value of something" (para.17). By that opinion, the evaluation of education can be defined as an action or a process to determine the value of everything in the world of education or anything that has to do with the world of education. At the beginning of the development of evaluation science, Schriven defines "Evaluation as judging the worth or merit of something (Fitzpatrick et al., 2004, p.5). To put it in other words, evaluation is an activity of judgment or giving a decision on the feasibility or benefits of something. Chen (2005) defines program evaluation as "The application of evaluation approaches, techniques and knowledge to systematics assessment and improvement the planning, implementation and effectiveness of programs" (p.3). Chen's definition explains that program evaluation is an activity of applying various approaches, techniques and evaluation knowledge to systematically assess and strive to improve the quality of planning, implementation and effectiveness of a program. In tune with Chen, Sudjana (2006) "Proposes that program evaluation is a systematic activity to collect, process, analyze and present data as input for decision making" (p.21). Stufflebeam and Shinkfield (2007) suggested that "Evaluation is the systematic assessment of an object's merit, worth, probity, feasibility, safety, significance and equity" (p.13). This definition of Stufflebeam reveals that evaluation is a systematic assessment of an object of benefit, value, honesty, worthiness, safety, meaning and or justice. Spaulding (2008), advocates "Evaluation programs to determine their worth and to make recommendations for programmatic refinement and success" (p.5).
Based on several definitions of program evaluation above, it can be concluded that program evaluation is a systematic activity of gathering information, analyzing and assigning values based on the criteria or standards established for decision making.
The evaluation of the input according to Tayibnapis (2008) is "An evaluation that can help to manage decisions, determine existing sources, what alternatives are taken, what plans and strategies to achieve needs and what procedures required to achieve them" (p.14). Input evaluation provides specific data and considerations for staff assessment, time, budgetary needs, education and administration strategies and others (Fernandes, 1984).
Product evaluation according to Stufflebeam and Shinkfield (2007), should assess the desired goals and desired outcomes both positive and negative. Outcome assessment is a consequence of the intangible outcomes of a program (Smith, 1996).
The research questions of this study are formulated: 1. To evaluate the success of the program from the input aspect by the following questions which are investigated: Research 2. To evaluate the success of the program from the process aspect, the following questions are investigated: Research Question 2: a. Are the learning outcomes of DMS Program students in accordance with the established goals (product)?
3. To see the success of the program from the process aspect, the following questions are investigated: Research The improvement program for qualification of undergraduate teachers of Islamic elementary school (MI) and Islamic Education (PAI) in schools is an education program specifically designed for in-service teachers within the Ministry of Religious Affairs. This program is implemented by Islamic Higher Education (PTAI), in which its teaching and learning process employs dual mode system (DMS) approach through the integration of conventional learning system (face-to-face on campus) and self-instruction system.

Objectives and Competencies of Dual Mode System Program Graduates
The implementation of the Bachelor Degree Academic Qualification Program for MI and PAI teachers at schools through DMS aims to: 1. Produce graduates who are academically qualified in Bachelor of Islamic Education for MI and PAI teachers at school. 2. Provide educational services of qualification improvement for MI and PAI teachers schools that graduated PGA (Islamic teacher education program, a similar level as senior high school graduates) and a two-year Diploma according to the guidance of law.
The graduate competency of the undergraduate academic qualification program for MI and PAI teachers in schools using DMS approach directs its graduates to have pedagogical competence, personality competence, professional competence and social competence.

Dual Mode System Program Curriculum
The implementation of this program is based on the mandate of Law Number 14 the Year 2005 the curriculum structure of undergraduate degree for MI and PAI teachers in schools using the DMS approach consist of basic courses, main courses and other subjects, with the total number of credits between 144-150 credits. The semester credit units and the length of the program that must be pursued by the students adjusted to the educational background of prospective students by referring to the Decree of the Minister of National Education no. 234/U/2000, as illustrated in the following Table 1.

Dual Mode System Teaching-Learning Process
Dual Mode System program teaching and learning process were conducted using dual-modes approach through a combination of face-to-face learning system and self-instruction system.
First, face-to-face learning is held to strengthen student's mastery of the material presented in self-study material through a series of direct meetings between students and lecturers on a scheduled basis. The study materials studied in face-to-face learning activities include: (1). The concepts in self-learning materials that are difficult and still not understood by the students after studying independently; (2). Application and problem solving raised from the material contained in self-study materials; (3). Input for the completion of tasks that must be done by students in the capacity of individuals and groups and (4). Input for the implementation of practicum that must be done by students either individually or in groups. These face-to-face learning activities include: (a) Lecture and Questionnaire, (b) Class or group discussion, (c) Guidance on practical activities and practicum and (d) Guidance on completion of tasks.
Second, independent learning is a learning that is conducted by using self-learning materials called modules. At the beginning of the lecture, the lecturer of the course lecturer explained how to learn by using the module. The module also describes explanations on how to study it in more detail according to the characteristics of each course. In the independent learning process, students can study modules, either individually and or in study groups.

Evaluation Model of Qualification for Islamic Elementary School Teachers and Islamic Education Teachers at Mataram State Islamic University
To comprehensively evaluate the MI and PAI teachers Qualification Enhancement Program through DMS in UIN Mataram comprehensively, its Input component, product and impact/influence. This study only selected three components from Context, Input, Process, Product and Outcome (CIPPO) model (Arikunto and Jabar, 2008) that is input and output (product and outcome).
The input and output evaluation model was selected in this study to evaluate the MI and PAI teachers qualification enhancement program through DMS at UIN Mataram comprehensively. The outcomes in this study is limited to the impact of this program on the graduate students of DMS, namely; (1). The ability of DMS alumni teachers in preparing the Lesson Plans based on the instruments provided by the Ministry of Religious Affairs, (2). The ability of DMS alumni teachers in teaching based on instruments Ministry of Religious Affairs of the Republic of Indonesia and (3). Maximize the level of their empowerment at the institution/madrasah/school on duty.
The input and output model is also selected because it includes summative evaluation, which is done after a program is completed (ex-post).

Approach, Method and Design of the Research Model
In relation to the research focus, the approach used in this research is a qualitative approach, using the criteria is the operational guidance/guidelines of DMS Program implementation stipulated by the Directorate of Islamic Higher Education Ministry of Religious Affairs (Diktis). The guidelines are in accordance with the Government Regulation No.19 of 2005 on National Education Standards consisting of content standards, process standards, graduate competency standards, educator standards and education personnel, equipment and infrastructure standards, management standards, financing standards and educational assessment standards. It then attempts to measure where the achievement of goals has been achieved.
This study is an evaluation of the program by using case study method, where the research focused on one selected phenomenon only and sought to be understood in depth, regardless of other phenomena. Such a phenomenon according to Sukmadinata (2010) can be a program, a policy implementation, or a concept. In this study, the program evaluated is the Bachelor Degree qualification improvement program through DMS. The research design model used an IPO, with the research designs listed in the following Fig. 1.

Instrument Validity and Reliability
The data collected in this study used three types of instruments: document studies, interview guides and questionnaires/questionnaires divided into two stages of evaluation: input and output (product and outcome). Before the research instrument is used, validation of the contents and constructs is performed. Non-test instruments used to measure attitudes have met the construct validity. To test the validity of the construct, the expert (panelist) judgment about the instruments that have been prepared (Sugiyono, 2009) can be relied on (Sukardi, 2012).
The validation of this instrument was completed by five experts and analyzed using validation index formula V (Aiken, 1980) and interrater reliability analysis (Djaali and Mulyono, 2008). After revision based on experts' recommendation, the instrument (questionnaire) was then tested for its reliability test, conducted by Internal Consistency, i.e. the test instrument (questionnaire) is only done once (Sugiyono, 2009). Then the validity of data obtained was analyzed by using Pearson correlation formula, Product Moment. Further, the reliability of instrument grains (questionnaires) test results that are declared valid was then analyzed with Alpha Cronbach formula using SPSS application and obtained value rcal questionnaire manager 0.987; rcal lecturer questionnaire 0.966 and rcal student questionnaire of 0.954. The value of r arithmetic is greater than t-table. According to Sukardi (2012), the value of correlation coefficient between 0.80 -1.00, means high correlation coefficient and certainly can be directly used for research.
Based on the results of the trial data (questionnaire) in this study: 1. The manager questionnaire totaling 47 items is declared valid because the value of r-count> r-

Data Collection and Data Analysis Procedures
The data used in this evaluation study were both primary, secondary data. The primary data are data directly collected from the Teachers Educators or Lecturers, students, University Leaders, DMS Managers, Ministry of Religious Affairs (central/district/city) and Madrasah or School Head/Vice head school, who became respondents. Secondary data were gathered from existing data in the form of policies, guidance/operational guidance or standardization of DMS program implementation in Mataram Islamic State University.
Qualitative data collection techniques employed were interviewed and guidelines, open-ended questionnaires and document analysis. While the quantitative data collection using closed-ended questionnaire and guidance. In accordance with qualitative research characteristics, data collection is considered complete, discontinued, if the data collected is sufficient and unchanged or to the point of saturation (Hosker, 2008).
Data analysis techniques used in this evaluation research are descriptive statistics and data analysis qualitative. Qualitative data analysis using Miles and Huberman (1994) consists of three flow of activities or processes that occur simultaneously, namely: data reduction, data presentation and withdrawal of conclusions/verification.

Input Evaluation
The input evaluation of the DMS program covering the components of students, lecturers, administrators and administrative staff, curriculum, syllabus/SAP, lecturing module, budget, availability of facilities and infrastructure and support of community participation, indicated that overall the input components have worked well and met the characteristics of the DMS program input standard in UIN Mataram.
In the student component, for instance, the implementation of the recruitment of undergraduate DMS students in general smoothly, but the administrative selection element is less thorough. Another issue that should be notified here is related to student mixing. Referring to the ideal concept, the Islamic Elementary Teachers class is for students who serve as MI teachers and Islamic Education Teachers classes are for those who are assigned as PAI teachers in public schools. In practice, however, student placements point to another fact where the PAI class (a total of 136 people) was filled with 75 students with the MI teacher status. Similarly, in the MI class (total of 374 people) there were still 45 students who are PAI teachers.
In lecturer component, DMS Manager of Tarbiyah (education) the Faculty of Science and Teacher Training of UIN Mataram has tried to apply appropriate working mechanisms in recruitment, selection and placement of lecturers. Although it must be admitted that the implementation is not perfect and is unable to accommodate all the interest of lecturers who want to teach in DMS due to academic qualification and competence considerations and it may be due to their experiences which are not sufficiently 'qualified' to be accommodated as a lecturer in DMS. Based on the documents analysis of the Undergraduate Program of UIN Mataram, Lecturers can be grouped as permanent and non-permanent lecturers. As for permanent Lecturers, there are 18 lecturers with doctoral academic qualification, 113 lecturers with master degree qualification. As for non-permanent lecturers, there was only one lecturer with doctoral qualification, 15 lecturers with master degree qualification and 5 with undergraduate qualification. The details of the qualifications of DMS Teachers/Lecturers can be illustrated in Table 2. About administrators and administrative personnel, it should consider the coherence of the nomenclature of the scientific posture between departments (programs) opened by the head of the program so that "the right man's credo on the right job" is implemented. Also, the head of the program must be able to completely mobilize all available resources to simultaneously move the wheels of the program more effectively and efficiently.

Table 2: Amount and level of education for DMS program at UIN Mataram Education Qualification ------------------------------------------------------------------------------------------------------------
In the structure of curriculum content, it is revealed that the posture of the student curriculum of DMS of UIN Mataram is a specialized model curriculum, which idealizes the specific output competencies supported by instrumental /secondary skills to support its primary ability as an output of Islamic labeled educational institutions. The curriculum structure has met the standards in the preparation of the college curriculum and can meet the pedagogical ability of students as prospective teachers. In other words, the students of DMS Program in the Faculty of Education and Teacher Training as a prospective professional teacher have been equipped with several subjects that match the subject composition of the Islamic elementary School level for MI teachers education and equivalent for Islamic Education Teachers.
The program so it is expected after they finish lectures at undergraduate level they would not feel foreign again with the subjects they will teach.
In the meantime, the number of teachers who have to follow the qualification improvement program and the diversity of educational background and the experience of the prospective participants, it is possible to do the conversion of credits or the Recognition of the Prior Learning (RPL), in the form of educational qualifications, teaching experience, professional training and job performance. This is done as a form of appreciation for teachers which can become "credit earning" in the completion of the teacher quality improvement program. To carry out the conversion of lectures, the DMS management of UIN Mataram developed a task force that understands the anatomy of the curriculum of the Faculty of Education and Teacher Training.
Normatively, the conversion completed by the Manager is competent but still requires refinement. In the future, if there is a similar program, managers must accommodate the experience and activities of students (in positions: teachers) that can be recognized equal to the courses to be followed.
In relation to the preparation of syllabus and lesson plan (Course Schedule), based on the results of the questionnaire analysis distributed to lecturers/teaching staffs of DMS and documentation verification, it indicated that the majority (97.82%) of the teaching staff of DMS in UIN Mataram preconditioned the lecture process by compiling syllabus and lesson plans. The result of analysis also showed a very high rate of attendance of lecturers (91%) and students (93%).
The above percentage suggests that there is a small part of the lecturers who did not prepare the syllabus with the assumption that the module is 'ready-use" which provide detailed 'what' and 'how' learning process should be constructed. Moreover, the DMS context syllabus was not a complicated job because the preparation of the syllabus simply required adjustment for the module distributed by the Managers, the Directorate of Islamic Education of the Republic of Indonesia. For that reason, in the preparation of syllabi, lecturers of DMS need to review, add and or reduce the syllabi according to the context and the needs of students.
In the module component, the DMS lecture modules are mostly composed by the Directorate General of Islamic Education Teams and modules composed by UIN Mataram lecturers, technically and qualitatively eligible and representative to support the ideal lecture process both on the module substance aspect.
Module contents demonstrate the interrelationship between parts, the scope of readability aspect. The module layout is good, the use of Indonesian language, the demands of sentence formulation, the integrity of the description and the use of the terms in the module is sufficient although it must be admitted that quantitatively the modules physical appearance duplicated by the Management are criticized by the DMS students. For instance, they criticized the physical appearance, non-consecutive page placement such as some missing pages, blurred or less obvious colors, improper binding and in-attractive appearance. But the problem of the display module that is un-interesting, perhaps less substantive as the Manager has facilitated the module to support qualified, meaningful and meaningful lectures for DMS students as learning subjects.
In the aspect of the budget, data documentation of DMS financial report reinforced by UIN Mataram DMS that every one semester each student is budgeted, equal to, IDR. 3.700.000, -(three million seven hundred thousand rupiah) allocated for (1) student registration fee of IDR. 100,000, -(one hundred thousand rupiah), (2) indirect cost of IDR. 2.300.000, -(two million three hundred thousand rupiah) consisting of tuition fee IDR. 400,000, -(four hundred thousand rupiah), the cost of BPP IDR. 1.600.000, -(one million six hundred thousand rupiah), the cost of laboratory IDR. 200.000, -(two hundred thousand rupiahs), monitoring and evaluation cost IDR. 100,000, -(one hundred thousand rupiah) and (3) direct cost of IDR. 600.000, -(six hundred thousand rupiah).
Lack of funding for DMS implementation, as reported by DMS Core Management was due to absorption of BPP budget consisting of lecturer transport, rent and maintenance costs of facilities and infrastructure and operating costs in different areas of operation.
In this context, the Central Ministry of Religious Affairs as the Budget Holder must be keen in making budget schemes based on territorial mapping by taking into account the breadth and the difficulties of locus and not just based on calculations of 'distance' standardization.
Even though the learning facilities in DMS Center are quiet, these facilities, unfortunately, are rarely used maximally. Based on observations, it is found that the existing laboratories were rarely used for the teaching and learning process. No learning practice indicates the existence of the educational interaction between lecturers and students in the laboratory. The list of DMS student visits to UIN Mataram library is also very low. Some DMS courses require the utilization of other learning resources as a method of professionalism enhancement of prospective teachers.
The enrichment of these various learning resources becomes crucial for constructing a professional and a talented teacher candidate. Also, on the aspects of facilities and infrastructure that must also be considered is the consistent aspects of lecture loci that is easily accessible while considering the availability of learning process instruments.
In Based on the results of the questionnaires analysis distributed to the respondents of DMS program manager, the role of the community in supporting the implementation of DMS program revealed that 6% stated very good, 46% stated good, 35% of respondents stated average, 9% said poor and 4% states very poor, as presented in Fig. 2.
Based on the results of document studies, interviews and questionnaire analysis, it was revealed that the community had a big role in supporting this DMS program. Support and partnerships built were in the form of facilities for the lecture with the rental system, both with universities, madrassas/schools and related agencies.   Besides, the graduates or alumni users took part in socialization, student selection and developing students' professionalism skills through Integrated Participatory Work Field Work Practice and providing a place for research location (thesis).

Product Evaluation
The results of the evaluation of the program on the aspects of the product covering the components of DMS student achievement showed remarkable achievements. Several indications characterize it; first, the graduation rate of courses. Concerning the documentation, data shows that the student passing DMS in each course almost 100% in each semester. This is marked by the absence of DMS students repeating courses in each semester. Second, the achievement of the praxis dimension which is indicated by knowledge gained by students in the lecture room they could be implemented in the real learning environment. Based on documented data, teaching practicum combined with participatory workshop activities called integrated PPL-KKP, it is revealed that 100% of students graduated with satisfactory grades. Third, graduation achievement and student's GPA.
What requires further improvement is the quality of the student's thesis. Therefore, in the future, the Management Party is suggested to consider that the involvement of students in social and religious activities can be accommodated as Participatory Working Lecture or students' internship. Teaching practicum as a program to practice how to teach or implement the theory in real class was treated as a program to improve teaching skills, which can be packaged in the form of Madrasah or School-Based Lesson Study. Besides, Thesis writing as a course to practice conducting research and making scientific work is directed to be a means of1` improving the learning process in the form of Classroom Action Research.

Outcome Evaluation
The results of the evaluation of the outcome aspect, based on the analysis of documentation and interview with the students, institutions and community recognition, showed that from that the DMS program succeeded in creating different values' for its alumni in West Nusa Tenggara Province.
Formally, the DMS program success is characterized by the increase of the academic ability of DMS student output in the form of preparing lesson plans, teaching competence, preparation of other learning instruments and achievement of DMS output as much as 175 people certified as professional teachers in UIN Mataram in the academic year 2013-2014. Despite this, it is worth bearing in mind that the majority of DMS alumni teachers who follow the in-service teacher education program and pass the certification, the majority of them have medium skills, especially in the preparation of lesson plan and professional teaching practices (as per the instrument of Ministry of Religious Affairs of the Ministry of Religious Affairs). This study revealed that out of 175 DMS alumni who follow the in-service teacher education program, only 16 people (9,14%) have the ability to create lessons and have good teaching competence. The majority of them (131 people or 74.86%) are in the medium category. Furthermore, 27 people (15.43%) belong to enough category and one person (0,57%) in less category. This is unfortunate because the government has spent huge funds to succeed this DMS program.
In addition, career development of DMS alumni in UIN Mataram showed a significant leap with the achievement in the form of promotion for instance as supervisor, head of a madrasah, deputy head of madrasah, homeroom, managing educational funds and prioritized for training, such as curriculum training (Fig. 3). For the institution, where DMS alumni serve, it becomes its credit point for accreditation of an educational institution in which the teaching staff is part of the appraisal item.
This means the acceptability of teachers, DMS alumni, at their formal duty rise. The status of scholars in some societies is also a determinant factor for the possibility of the election in various educational activities as well as the community. This DMS program has contributed to the acceleration and progression of its alumni.

Conclusion
In general, it is concluded that the implementation of the undergraduate qualification improvement program through DMS in UIN Mataram has been well implemented and in accordance with the guidance and rules set by the Ministry of Religious Affairs, the Republic of Indonesia and Government Regulation No. 19 Year 2005 on National Education Standards, both in input and output (product and outcome) dimensions. Some aspects need to be improved and enhanced, in quantity aspect, more specifically in quality aspect.
There are several aspects that need to be improved and enhanced, especially the outcome aspect, namely the ability of madrassah teachers (MI) and PAI teachers, guiding good practice in exact lessons or social lessons and in making the Implementation lesson plan and teaching competencies based on teacher in-service education program instruments established by the Ministry of Religious Affairs.