Evaluating the implementation and effectiveness of GIS-based application in secondary school geography lessons

The purpose of the study was to investigate the barriers preventing the use of Geographic Information Systems (GIS) in secondary school geography lessons and to determine its effectiveness on students’ success. A workshop focusing on ways to implement GIS-based application in the classroom for 14 teachers from nine high schools was conducted in 2006. The teachers were given GIS software, digital data for an application, and the necessary written documents describing the application. Due to various obstacles, only two teachers at two schools out of 14 implemented the application successfully. The study revealed that the use of GIS increased the students’ success on geography lessons by 38% at the first school and by 51% at the second one. The success rate of the students in this study substantiates the need for GIS to be better incorporated into the Geography curriculum in the secondary school level in developing countries.


INTRODUCTION
Geographic Information Systems (GIS) has been listed among the 25 most important developments which have affected the life of humanity in the 20th century [1] . GIS has revolutionized the methods and dimensions of spatial analysis resulting in a dramatic change in the direction of Geography and become a major component in other disciplines with a spatial component (i.e., biology, political science, urban planning, geology, sociology etc.). It has also led to significant reforms in geographic education [2] . The use of GIS in geographic education first began at the university level. The number of the programs, which were offering GIS lessons at the US and Canadian universities in the beginning of 1980s were approximately 10. By the end of the 1990s, this number exceeded 2000 [3] . GIS has now become an integral part of geography departments in most universities in the USA, Canada and some European countries, with some offering specific degrees in GIS or Geographic Information Science at the undergraduate, masters and doctoral level.
At the same time that the use of GIS at institutions of higher education was becoming widespread, it was also having significant impact on secondary education institutions in the USA and Europe [4] . At the beginning of 1990s, GIS was increasingly seen as an essential element which could introduce secondary school students to the use of technology, and spur them to be motivated in careers in science and engineering [5] . The interest in this area has increased at the secondary school level after many other studies revealed that GIS is an educational tool rather than a technology and contributes to creating an inquiry based learning environment [6,7,8,9,10,11] . Due to these pedagogical advantages, GIS has found a place in secondary school curriculums in diverse programs including Science, Chemistry, Biology, Mathematics, Environmental and Social Sciences along with Geography in the USA, Canada, and Europe [10,12,13,14,15] .
GIS is a set of integrated software programs designed to store, retrieve, manipulate, analyze and display geographical data-information concerning people, places and the environment [16] . Since it enables versatile analysis connected to geographic references of data, GIS has many advantages especially to geographic education and among the other disciplines. GIS has been a major contribution to aid students to develop spatial thinking skills [14,17,18,19] . Wanner and Kerski [7] explained that GIS has simplified the processes of analyzing and presenting geographic information and has accelerated geographical inquiry in a class environment. According to Bednarz and Van der

SCI-PUBLICATIONS Author Manuscript
Schee [15] , geography teachers are using GIS due to the three major reasons: 1) GIS supports geography teaching and learning, 2) GIS is a tool for investigating geographical problems at different scales, 3) GIS is a necessary tool for the business world in the 21st century.
Although the potential benefits of GIS for students and teachers have been widely documented in the literature, its integration into secondary school education still remains a challenge [19] . Various studies have addressed this issue by proposing proper methods that should be used to gain the expected benefit from GIS [7,10,14,20] . Other studies have even suggested that choosing a wrong method may negatively affect the students' comprehension [14,21] . According to Walsh [22] , the danger lies in allowing the GIS to force the methodology and the analysis without a full understanding of the basic problem being addressed, thereby changing the role of GIS from a tool to a magic box. In order to develop the right method, it is necessary to maintain the relation and boundaries between GIS and geographic education. Sui [23] offers an important clue about these issues. He states that there are two different sides of GIS education: "Teaching with GIS" and "Teaching about GIS". The aim in "Teaching with GIS" is to allow students to learn about geography and gain geographic skills with the help of GIS as a tool. The aim of "Teaching about GIS" is mainly to teach GIS technologies. The overall opinion among the educators about the type of the use of GIS in geographic education focuses on "Teaching with GIS" [9,12,14] . Nonetheless, Johansson [19] has mentioned that it is necessary to revert to the 'Teaching about GIS' to some extent before using GIS in classroom.
Despite the potential of GIS in secondary school geographic education, its use is not widespread throughout the world today. GIS was used by less than 2% of high schools in 2003 in the USA, which was one of the first countries to use GIS in education [12] . The level of the use of GIS at secondary schools is approximately the same in the UK [24] . According to Lloyd [17] , the adoption of the use of GIS in secondary education are related to three major obstacles: 1) technical factors such as the availability of hardware, software and data, 2) lack of teacher training and curriculum materials, 3) systemic issues that encourage or discourage innovation in education. Some additional obstacles were mentioned in other studies as the lack of time for teachers to learn GIS and use it for their class activities [12] , the unwillingness of teachers to learn and use new technologies [14] , and insufficient place of GIS at curricula [19] . The difficulty in using GIS softwares has also been mentioned as an obstacle. According to Bednarz and Van der Schee [15] , GIS software has high technical demands, is a challenge to master, was not designed for a teaching/learning function and does not offer obvious opportunities for teaching/learning to many educators.
The number of methods to use GIS at secondary school geography lessons is increasing with new research and developments in GIS technology and its pedagogical use. Electronic mapping and Internet-based mapping are a few of many new techniques which are being tested pedagogically in classrooms today [25,26] . In the adaptation of GIS to geography lessons, two methods are being widely utilized: project-based learning and applied learning. In project-based learning, geography lessons are supported by projects which are assisted with GIS [7,27] . One of the latest studies in which project-based learning was used successfully in geography lessons was the Geographical Information Systems Applications for Schools (GISAS) project completed in 2006 [19] . The GISAS project was a threeyear education and research project funded by MINERVA Action of the European Commission. The project included members from 7 European countries (Belgium, France, Greece, Hungary, Italy, Latvia and Sweden) and focused on water quality analysis. The students made observations and analysis on water resources around their schools, mapped their findings in a GIS environment and then shared them with other project members within a web-based learning environment [19] .
The use of GIS in applied learning usually consists of students implementing a GIS-based application in geography lessons. The number of studies on development of GIS-based application has increased recently [12,15,21] . The book, Mapping Our World: GIS Lessons for Educators [28] published by ESRI, is among the best examples of using GIS to develop applications for geography lessons. This book contains lessons in the theory and practice of GIS along with student handouts, assessments, and a one-year site license of ArcView 3.x for middle and high school geography students. The benefits of GIS-based applications for increasing student's interest in computer technology and geography lessons have been widely acknowledged in several studies [8,9,14,15] . However its overall effects on students' skill development and learning at geography lessons have not been documented sufficiently. This uncertainty brings about the first research question of this study: At what scale can a GIS-based application affect students' success on geography lessons?
The studies concerning obstacles preventing the effective use of GIS at the secondary school geography lessons were mainly conducted in developed countries like the USA, UK and Canada where GIS has been used at the secondary educational level for more than a decade. Also with the recent inclusion of GIS into the European curriculum, some countries like France, Sweden and Finland have found difficulties incorporating GIS in their geography classrooms [19] . However there is not enough research about the barriers preventing the use of GIS at secondary school geography lessons in developing countries.
Turkey is an example of a developing country which has initiated a new secondary school geography curriculum in 2005 with a strong emphasis on Information Communication Technologies (ICT) [29] . With this new curriculum, GIS became an important part of geography education officially for the first time in Turkey. Previously, GIS was only discussed at a very rudimentary level in some textbooks. In the new curriculum, it was specifically included as a tool for activity development. The curriculum suggests that "teachers should introduce GIS into the classroom depending on the availability of adequate hardware and software at their particular schools" [29] . The use of GIS was recommended 20 times in the curriculum, but no definition of GIS or the means of implementation methods into geography lessons was supplied [30] .
The advent of GIS into the geography curriculum has increased teachers' interest for this technology, but it also raised many concerns among educators and pedagogues in the country. One of the key questions is: How can GIS be incorporated into the curriculum and to what extent in a developing country like Turkey where there has been no experience taking into account that other countries (i.e., UK, USA), which have a much longer experience using GIS in secondary schools, are still experiencing difficulties incorporating it into the classroom? Some other pressing questions which are being raised by educators is: How will GIS be mastered and used by geography teachers who have no or limited awareness of the technologies and concepts of GIS and is there adequate hardware, available software, infrastructure and available space to accommodate the use of GIS in developing countries where GIS will be used for the first time? These concerns constitute the second research question of the study.

MATERIALS AND METHODS
A noteworthy effort is being made in all countries to improve the quality of education. However, any effort to better the level of education is often hampered by the lack of resources. The excessive number of students in classes, insufficient infrastructure in terms of ICT at school and classrooms, lack of interest and ability on teachers to use new technologies and teaching methods are among the most important factors restricting education especially in developing countries. These obstacles prevent lessons from being studentcentered, problem-based and application oriented. In this environment, it was not easy to use a new technological system like GIS and test its effectiveness on students' achievement in geography lessons which have been mainly based on textbooks and administered as teacher centered.
In the beginning, it was assumed that there were three main obstacles preventing the implementation of the study. These were that the teachers: 1) did not know how to use GIS software, 2) were unsure how GIS would be incorporated into their geography lessons, and 3) did not have the necessary infrastructure such as hardware and software available in the classroom. These obstacles were removed by using three strategies: 1) training a group of teachers in the use of GIS software, 2) preparing a GIS-based application and delivering it to the same group of teachers, and 3) selecting schools that had adequate computer laboratories to complete the application.
The study was completed in three phases in 2006 in Turkey. In the first phase, a training workshop was held for a group of teachers about GIS, software and the application. In the second phase, teachers were monitored to see how successfully they had implemented the application at their schools. In the third phase, the effect of the GIS application on students' success was evaluated by using the findings of teachers who had implemented the application in their geography lessons successfully.

Development of the GIS-Based Application:
In previous studies concerning the integration of GIS into geography lessons, the focus was mainly on project based applications [6,19,31,32] . In these GIS-based projects, students chose to investigate problems in their local environments, collect their own data with different tools and analyze them by using GIS. However, in this study a different method was used in that a GIS-based application was prepared with the necessary digital data and written documentation and tested in a classroom setting.
The GIS-based application was prepared for the ninth grade geography curriculum. It was designed to be implemented at computer laboratories of selected schools during 40 minutes. Using the standards indicated in the new geography curriculum, it was decided to focus on one of the theme of the new geography curriculum "plates, earthquakes, and volcanoes". In this particular application, students were expected to understand the relationship between plates, earthquakes and volcanoes by using GIS after the same topic was covered in the classroom with traditional methods. The application also sought to determine the cities and regions which are under earthquake risk by using different GIS layers in the provided database (i.e., faults, epicenters, location and population of cities in the world and Turkey in particular).
Arc View 3.2 was used as the GIS software in the application because it was easier to obtain and use compared to others available in Turkey. Different data sources were used to create the GIS database for the application. Fault lines, plate boundaries, epicenters of the earthquakes which took place in 2000, and location of cities in the world were taken from a previous study made by ESRI [28] . A digital data set for Turkey was produced which included the location and characteristics of large earthquakes causing considerable loss of life and damage in the past, earthquakes occurring in 2005 with magnitude 2 and greater, major fault lines, locations and populations of cities, and earthquake risk zones.
Unlike project based GIS applications, collecting data and digitizing were not part of the application. Instead, the application was based on observations of data given in different layers, answering questions by identifying different relationships and conducting basic spatial analysis operations. The steps of the application, the questions and how to create different spatial analysis to answer these questions were described in detail in a hand-out. Two important considerations were taken into account when the exercise was prepared: the first being that neither the teacher nor the students had previously used the software; and the second that the language of the software was English. To eliminate the latter barrier, a Turkish user guide of the software was developed.
An exam was prepared in the study to evaluate the effects of the GIS-based application on the students' comprehension of the subject matter. In addition, a self assessment form was prepared to allow students to evaluate the application and how useful GIS had been in understanding the material at the end of the application. Seven questions were asked in the exam in two different sections. The first section evaluated the students' map skills and consisted of three questions. The second section measured the students' knowledge in the given subject and consisted of four questions.

Implementation of the GIS-Based Application:
The presence of hardware, software and data are among the most important prerequisites to incorporate GIS in geography curriculums [33,34] . To increase the chance of the implementation of the GIS-based application, only private schools were selected because they had adequate computer labs capable of running the selected GIS software, were better at using ICT and their class size was generally smaller than public schools.
A test group for the application was selected consisting of 14 teachers who were teaching ninth grade geography lesson in nine different high schools in a workshop format. In the beginning of the workshop, a survey was conducted among teachers about their knowledge, skills, and experience in GIS, their general experiences in geography education and physical conditions at their schools. During the workshop, teachers were given basic knowledge about GIS, potential ways it can be used for geography lessons, and basic operating instructions about ArcView 3.2-the package to be used in the application. After the completion of this basic introduction to GIS, the application was explained to the teachers in detail. At the end of the workshop, a trial version of the GIS software was given to teachers along with the digital data and other documentation necessary for the implementation of the application.
To measure the effects of the use of GIS on student's success, an experimental design was chosen. In each school, the teachers were requested to select two groups of students at the same level, one as the control and other as the experimental group. The lesson was to be given to the control group with traditional methods. The experimental group was to be given the lesson augmented with the use of GIS. The effect of GIS on student's success was to be evaluated by assessing the results from the two groups of students. Teachers were also asked to have the experimental group complete a self-assessment form at the end of the application.
Teachers were instructed to reduce non-individual responses by ensuring that every student would complete the application in front of a separate computer. To provide additional quality control, teachers were asked to implement the application at computer laboratories at their schools. To finish the implementation in a timely fashion, teachers were also asked to teach basic functions of the GIS software to the experimental group during an appropriate time after school or by using computer lessons. After completion of these tasks, the assessments of students' success, students' self-assessments, the results of teacher survey, and the whole process from the beginning to the end of the study were evaluated all together to answer the research questions of the study.

Sufficiency of teachers and physical environment:
The survey taken before the workshop revealed important findings about the qualifications of the teachers and the computer facilities at their schools. Initial questions in the survey were about teaching experience of the teachers. The average time of teaching was nine years. 11 teachers indicated that they had been teaching geography for more than five years. These results indicate that teachers were quite experienced with geographic education. Their average class size was 21. These figures are quite low considering that the average class size in Turkey is 34 [35] . The average weekly teaching load of the teachers was 18 hours.
The questions asked in the second part of the survey were aimed to reveal the teachers' level of knowledge, skill, and education. Three teachers indicated that they knew about computer to the extent that they could search for information on the Internet, could communicate through e-mails, and could use Microsoft Excel, Microsoft Word and PowerPoint programs effectively. 11 teachers said that their computer skills were limited to the use of the Microsoft Word program and the using the Internet for research. The survey further included a question to reveal the teachers' level of English. Only one teacher indicated a reading proficiency of English. Four teachers indicated a partial reading proficiency. Nine teachers stated that they had little or no knowledge of English. When asked if they had attended any in-service education program organized by the Ministry of National Education over the last year, only one teacher said yes. 12 of teachers who attended the workshop have undergraduate degree and two of them have graduate degrees.
The survey alluded to some physical characteristics of the classrooms that could hinder teaching geography. Only one teacher stated that there was a classroom which was exclusively designed for geography lessons. Only six teachers stated that there were computers and projectors in their classrooms and only four included an Internet connection and printer.
The characteristics about the use of technology in the classroom are also noteworthy. As seen in Table 1, most of the teachers infrequently used a technological aid such as PowerPoint or overhead projectors and are using the textbook as their primary source for instruction. Only one teacher uses a computer for every lesson with eight never using the computer in their lessons at all. With this low usage of supplemental technology, it is not surprising that nine teacher never use the Internet in their geography lessons. Only two teachers stated they used the Internet once per semester, and three teachers said they used it once a month.
The last section of the survey was indicative about the teachers' knowledge and expertise in GIS. As seen in Table 2, the teachers in the survey have limited knowledge about GIS. Twenty-nine percent of the teachers stated that they have not previously even heard of the term 'GIS'. All of the teachers indicated that they have not used any GIS package. Therefore, it is not surprising that they had not used GIS in their geography lessons. Although all the teachers stated that GIS should be used in geography lessons, only 21% indicated that they knew how it could be included in their lesson plans.

The implementation of GIS-based application and its effectiveness in geography lessons:
The GIS-based application was successfully implemented by only two teachers out of 14. The remainder of the teachers could not complete the experiment due to a variety of reasons ranging from the physical conditions at their school to the limited amount of time to accomplish the tasks required. Only one teacher could not implement the application due to the lack of a computer laboratory. The biggest obstacle expressed was finding enough time to complete the tasks required for the experiment. The majority of the teachers indicated that there was a lack of time to learn GIS software and the application better before implementing it at their schools. Many of the teachers could not find adequate time to prepare their laboratories for the exercise. Lack of time to organize the students and teach them how to use the GIS software before the usual class time was another obstacle. The other reasons indicated by the teachers were: the computer laboratories at schools were busy; the application was long and could not be finished in time; and the students were not able to understand the software fully.
Only two teachers were able to complete the experiment. However of the two, only one followed the provided instructions (designated as Teacher A). The other teacher (Teacher B) first taught the lesson to a ninth grade class with traditional methods, and then administered the examination. After implementing the GIS-based application in the same class, the same examination was administered again and presented some different results which are only partially comparable to those of Teacher A due to the aberration from the prescribed methodology.
Teacher A's control group included 15 students while the class size for the experimental group was 14. The average score for the control group was 30 points out of 70 (43%) while the experimental group's score was 48.3 points (69%). These results show that the experimental group achieved 38% higher scores than the control group. Further analysis of the different section of the examination gave some details concerning the actual improvement in specific areas.
When only the map skill oriented questions are evaluated, the control group received 4.9 points out of 30 (16%), while the score of the experimental group was 18 points (60%). However, when the evaluation is made over the questions asked to see the change in the students' knowledge level, the control group scored 25.1 points out of 40 (63%), while the experimental group's score was 30.3 points (76%). The experimental group performed 72.7% better in the map skills section and 17% better in the knowledge section than the control group. When evaluating the individual questions, the experimental group performed better on all the map-skill oriented questions. In the knowledgebased section, the control group scored better on only one question. The results of the questions by section are seen in Figure 1.
The results of the examination conducted on a single class by teacher B before and after the implementation are similar to the previous evaluation. The class size was 15. The success of the class

Questions yes no
Have you ever heard of the term GIS? 71 29 Have you ever attended an education program about GIS? 0 100 Do you know how to use a GIS software? 0 100 Do you know how to use GIS for geographic education? 21 79 Have you ever used GIS in your geography lessons? 0 100 Is there a computer laboratory at your school for GIS use? 93 7 Do you think GIS should be used at geography lessons? 100 0 increased from 23.3 (33%) to 47.9 points (68%) over 70 after the implementation of GIS-based application. This realizes a 51% improvement in average score of the class. The students' score averaged 2.5 over the three map skill oriented questions before the GIS-based application but increased by 89% and became 22.6 points after the application. The success of the students was not as dramatic when examining the knowledge based questions as their average score increased 18% from 20.8 to 25.3 points after the application. When the answers of the questions are evaluated individually, it is seen that students got better results from all the questions in the examination after the GIS-based application ( Figure 2). The self-assessment forms which were filled out by the two classes which implemented the application gave important clues about the students' opinion of GIS and its use in geography lessons. When the results of these forms are evaluated together, it is seen that more than 80% of the students found the GIS-based application useful because it increased their interest in geography lessons, it helped them understand the lesson better, it showed them how to use geographic information in daily life and it motivated them to learn more and to use new technologies. 79% of the students indicated that GIS helped them improve their inquiry skills. 46% of the students thought that their computer skills had improved through the use of the GIS-based application and 50% of them stayed neutral in the related question ( Table 3). The self-assessment form included some open ended questions. Evaluation of those questions revealed that 82% of the students did not encounter any problem during the implementation. The remainder of the students thought that the application was difficult to understand and implement, and the software was difficult to install. When they were asked what they would suggest to make this application better, the students recommended that there should be more time for such applications, computers should be of higher quality, and the class hours should be increased for geography lessons. All students except one agreed that GIS and other related technologies are essential tools to be used in geography lessons.

CONCLUSION
GIS has many pedagogic advantages for students and teachers at secondary schools, especially in geographic education. Recent studies which demonstrate concrete benefits of GIS in education have increased the number of countries which desire to make it a part of their secondary education. GIS has been used for many years at the secondary education level in the USA, Canada, UK. Many other countries like France, Sweden, Finland, and Turkey are currently making a step in this direction with the recent development of a new geography curriculum which directly recommends that GIS be used as a teaching tool. However, as can be seen in this study, while the guidelines have increased the teachers' curiosity about GIS and its inclusion into their lesson plans, there are some serious limitations before it can be an integral part of geographic education in those countries.
In this study, the problems with using GIS in the classroom is quite apparent in that only two teachers out of 14 implemented the GIS-based application at their schools although each was given the same The reasons behind the failure to implement the application can be summarized as follows: Teachers had difficulties in understanding the GIS software. Although they had received training in the workshop, the overall lack of computer skills hampered the successful completion of the experiment. Teachers who taught more than 18 hours a week could not find enough time to practice using the GIS software and the application after the workshop at their home or at schools. Teachers had difficulties in preparing the computer laboratories and students.
These findings give important clues about the overall obstacles facing the use of GIS in secondary school geography in countries where there is lack of resources in education. When the teachers' answers to the survey questions and their performance during the whole study are taken into consideration, it is clear that the obstacles in developing countries are different from those found in the USA and the UK although there are many similarities. The most striking obstacle in Turkey is that most Geography teachers are not aware of GIS even at a very rudimentary level. It is not surprising that they cannot understand how they can benefit from using GIS in the classroom. The lack of sample course plans which include the use of GIS is an additional barrier. Inadequate computer facilities, software and data, along with the teachers' knowledge of ICT are also considerable barriers to successfully incorporating GIS into geographical education in Turkey. Part of this problem is that teachers have graduated and are presently graduating from universities with education programs specializing in Geography without any knowledge and skill concerning GIS. Compounding this is the deficiency of in-service educational opportunities and resources for teachers.
For the successful use of GIS in the classroom, there must be adequate technology and supporting pedagogic infrastructure. If schools do not have adequate computer laboratories, GIS software or data and teachers do not have enough GIS skills and knowledge, then it is obvious that GIS cannot be successfully integrated into the Geography curriculum. If the potential of the use of GIS in Turkey at the secondary school level and in other developing countries is considered from this context, real progress will be made only by the provision of adequate technology in schools and inclusion of GIS in teacher training -either at the university level or by in-service training. However, this should not bar interested teachers and school administrators seeking their own solutions to integrating GIS into the classroom.
The study revealed important results about the effects of GIS-based application on students' success at geography lessons. In the two schools where the GISbased application was implemented successfully, the experimental group became 38% more successful than control group at the first school and 51% at the second school where the experiment was conducted on a single class. The increase at the second school can be attributed to the students being more familiar with the questions. However, in both instances there was remarkable change in overall scores. Not surprisingly, the use of GIS increased the students' success greatly on the map skill-oriented questions at both schools.
It is obvious in these results that GIS helped students to develop spatial thinking skills by exposing them to spatial analysis. Geography lessons became more visual and students were able to conduct spatial  analysis through the use of GIS. The effects of GIS were not only on the achievement of the students, but the use of GIS also positively affected the attitude of the students concerning geography lessons. Because of the GIS-based application, students became more interested in new technologies and methods used in the discipline of geography to address current problems. The study also helped students' computer skills in a positive way. The GIS-based application was a good opportunity for the teachers to understand to what extent they knew and used ICT in geography lessons. This study shows that GIS-based applications can be used at schools successfully to support geographic education. The failures of 12 teachers to implement the GIS-based application at their schools resulted not from inability of GIS but from the obstacles facing the use of GIS. Therefore, new strategies should be developed to eliminate these obstacles to realize benefits from GIS at a secondary education level in Turkey and other countries where GIS was incorporated into the curriculum recently. The use of standard based Geography curriculums has been laid out by the Geographic Education community in the USA and the UK and could generally be considered as a guide for other countries in both the developed and the developing world. However, the successful inclusion of GIS into a secondary level education requires adequate infrastructure and software, in-service training and instructional material to support the standards, and appropriate courses in the educational curriculums in higher educational institutions. These are daunting tasks given the resources of most developing nations. However, they are not impossible ones with adequate local leadership, appropriate allocation of resources, additional international funding and consultation and adequate teacher instruction in GIS at higher education institutions.