The Impact of User Knowledge on Web Search Satisfaction

: Problem statement: Searching on the web is a tedious process as it requires knowledge and skills on what and how to search. What to search is basically, the core of the searching activity as it represents the need of the searcher. How to search is related to the knowledge on how the facilities available on the web can be utilized in order to achieve the needs. Search satisfaction is the level of measurement that describes the achievement of the searcher towards his/her information needs. Approach: In this study, two categories of knowledge: Topic understanding and web search system understanding have been identified to contribute to the search satisfaction. These knowledge was measured based on objective and subjective knowledge. The statistical analysis is employed study the impact of these knowledge on search satisfaction. Results: The Linear Regression analysis confirms that both topic and search system understanding are equally important since they are significantly associated with search satisfaction. Conclusions/Recommendations: This finding proves that user knowledge is an important aspect in searching activities. This finding can benefit the search system provider when updating their search system facilities especially query processing, matching and manipulation engine.


INTRODUCTION
Satisfaction is a subjective state of satisfaction. It is a state where people feel pleased with their achievement due to some effort. A comprehensive review on wide context of satisfaction definition by (Giese and Cote, 2002) summarizes that satisfaction is "some type of affective, cognitive and/or conative response, based on an evaluation of product related standards, product consumption experiences and or purchase related attributes and express before choice, after choice, after consumption, after extended experience, or just about any other time a researcher may query consumers about the product or related attributes".
In Information System (IS) context, user satisfaction is a general measurement of belief of how well a system meets user's requirements and expectations (Shirani et al., 1994). It have been an indicator to evaluate the satisfaction level of the user towards the IS system (Griffiths et al., 2007) and as an indicator for system success (Wang and Liao, 2007). It is also related to the user experience when using the IS system. This study is concerned with one of the components of IS system that is the satisfaction of the search result of the search system, which is called search satisfaction. Search result is a list that contains title, short description of the document/article/web pages and Uniform Resource Locator (URL) stored in the search system database. These results served as information to the searcher that contains related document/article/web pages that matches their query. Further, searcher will evaluate each of these results in order to determine one that meets his need.

Determinants of user search satisfaction:
Search satisfaction is a subset of user satisfaction in which the measurement is focused mainly on the experience faced by the searcher during the search session. It is a measure of how well the searcher is satisfied with the results returned by the search system. It is an indicator to determine searcher achievement of his information need (Zoe and DiMartino, 2000) supported by their search strategy (Johnson, 1997) which leads to the correct inference (Newell et al., 2004). Search satisfaction is also influenced by the search performance (correctness or accuracy, time) and the searcher's attitudes (confidence and satisfaction) towards the search activity.
A typical judgment of search satisfaction depends on the ranking made by the search system such that the number of clear topically relevant references in the top twenty items retrieved (Sihvonen and Vakkari, 2004). Most importantly, the information (search result) is considered relevant when it matches the query entered during the search session (Rieh, 2002). This judgment is typical as most of the search systems provide this facility as a part of their services.
Initially, these factors are pre-determinant for query selection and manipulation. As in the search system, query plays an important role to ensure the search satisfaction, understanding and knowing how to formulate the query will benefit best the user. This reformulation is motivated by the fact that the initial query returns a result that rarely meets the user's need that means to modify the original user's query by adding significant terms to give back a more releant results (Abdelkrim et al., 2010). Previous research has also proved that query-based search system is more popular compared to other search systems (Liaw and Huang, 2006;Ali, 2005). Therefore, in this study user knowledge is selected as a main factor for search satisfaction. This is due to the fact that knowledge is fundamentally essential in formulating a query during the search session. After the search session, user will typically update his or her knowledge about the query manipulation and how to use the search system. The "update" process will create a new knowledge to the user. This new knowledge is referred to as experience. In this study, experience is not considered as primary factor as it is gained after the searching process. At the initial stage of searching, knowledge is basically the initial factor (Nesset, 2005).

User knowledge and search satisfaction:
Knowledge is a belief that is true and justified and a characteristic of a person that influences the person's behavioral potential (Hunt, 2003). Knowledge can be defined as knowledge of objects (factual knowledge), knowledge of events (experiential knowledge), knowledge of performance (process knowledge) and meta-knowledge (Connell, 1995). Meanwhile, Roxanne et al., (2010) state that knowledge is from information, information is the raw product and knowledge is the finished result of the product. User knowledge can be classified into objective knowledge and subjective knowledge (Mattila and Wirtz, 2002;Knight, 2002;Hunt, 2003). Objective knowledge is actual knowledge about what is actually stored in memory (Mattila and Wirtz, 2002) and subjective knowledge is actual information about how much users perceive they know (Mattila and Wirtz, 2002).
In IS research, user knowledge plays an important role in determining the success of the system. In Enterprise Resource Planning (ERP) for example, user knowledge and involvement has been found to be one of the ten-item measurement to access the user satisfaction of the ERP system (Wu and Wang, 2006). This finding justifies that, user is the closest entity with ERP and the one's who participates in the ERP implementation in business. Thus, user knowledge is a significant measurement of user satisfaction. In search system, user is the one who with the information needs, formulates the query, constructs the query and operates the search system. Thus, user knowledge either on the topic or on the search system would significantly affect the search satisfaction (Hirsh, 1995;Holscher and Strube, 2000).

Variables and measurement:
In this study, user knowledge is divided into two; knowledge on the domain (topic understanding) and knowledge on how to use the search system (or web search system understanding). Utilizing both types of knowledge will contribute to search satisfaction. Both knowledge can be measured using objective measurements as stated by (Mattila and Wirtz, 2002). These measurement aims to evaluate the initial knowledge of the searcher.
Domain knowledge: Knowledge on the domain is a depth understanding of the domain including the search topic. It can be described as knowledge of facts, concepts and their relationships in a specific domain (Sutcliffe and Ennis, 1998). Research has shown that the more familiar user to the topic, the more efficient their searching (Kelly and Cool, 2002). High domain knowledge enables users to search effectively and provides a richer set of concepts and terms for query formulation (Sutcliffe and Ennis, 1998), thus initiating a successful search (Navarro-Prieto Scaife et al., 1999). Conceptual and semantic knowledge related to the query is required to articulate a good query (Large et al., 2001;Sridhar, 2004). User domain knowledge can support more efficient search by helping users to separate relevant information from irrelevant responses, facilitating learning of search principles and formulating accurate queries (Hong et al., 2002).

Web search system understanding:
Knowledge of the search system is the knowledge on how to use the search system including the search strategies supported by the search system. Researchers believe that the usefulness of the information and search productivity (search results) depends on the searcher's ability to understand the system (Borgman, 1987;Hildreth, 1997;Fidel et al., 1999) and use the technology effectively.
The feedback received from the study such as Pollock and Hockley (1997) and Topi and Lucas (2005) reveal that knowledge of the search technology is a very important factor for the search success. As highlighted by (Fidel et al., 1999) the searching behavior of the users illustrates that they would have greatly benefited from easy and immediate access to knowledge tools and those that support navigation.
Advanced facility such as assisted tool which are typically available in commercial search engine have been found to have a significant effect on the performance, satisfaction and confidence (Topi and Lucas, 2005). In its absence, a Boolean operator was also found to be an effective means for improving user performance. Knowing how to use Boolean to formulate query is a great advantage for web users (Chau et al., 2007). Therefore, it is no doubt that Boolean has been recorded as the most frequently used facility to support searching (Ali, 2005). Muylle et al. (2004) study on user satisfaction on website has distinguished user satisfaction into four dimensions which are layout, information, connection and language customization. Out of these dimension, information dimension which represent the content of the website is found to be relevant with search satisfaction. This dimension is inline with earlier study by (Zmud, 1978). Zmud (1978) viewed information as the value of IS. Thus the presentation of the information is critical to decision maker perception and subsequent usage.

Search satisfaction:
The dimensionality of information suggested in (Muylle et al., 2004) includes the information relevancy, information accuracy, information comprehensibility and information comprehensiveness. Information relevancy is the degree to which the information perceives to meet the user needs. While, information accuracy is the preciseness of the information content. These dimensionalities are inline with the other literature that address accuracy (Topiand Lucas, 2005;Gohmann et al., 2005;Cheung et al., 2008) and relevancy (Sihvonen and Vakkari, 2004;Beg Fig. 1: Research framework and Ahmad, 2007;Beg, 2005;Rieh, 2002;Bilal, 2002;Birgelen, 2008) as important consideration for user satisfaction and online decision.
Information comprehensibility indicates the extent to which the user understands the information. It is a situation where user can interpret the information, judge and act accordingly. Such an example, in road safety (Siebenhandl et al., 2007), information comprehensibility of the signage and pictorial messages is very important in order to avoid an accident. Good information comprehensibility will alert the drivers to act accordingly when seeing a signage even though in unfamiliar environment or in an abundance of information. Information comprehensiveness measures the completeness of the information. According to (Cheung et al., 2008), "the more comprehensive the messages are, the higher the perceived information usefulness of the message".
Research framework: Figure 1 shows the framework for this study. The framework consists of two major parts namely; user knowledge and search satisfaction. Search satisfaction is identified as the dependent variable, which is the subject of study. Search satisfaction can be viewed as the level of end user satisfaction on information after search session based on the task given. User knowledge is the independent variable. User knowledge is made up of two components; topic understanding and web search system understanding.
Based on the research framework, two hypotheses have been formulated:

MATERIALS AND METHODS
In this study five instruments have been constructed to collect data from the respondent. The instruments are knowledge test question, questionnaire, search task, search interface and search log.
Knowledge test question is use to obtain information that are related to the respondent's knowledge on the topic understanding and the web search system understanding. Knowledge test question is in a form of multiple-choice question, true-false and subjective type of question.
Once completed the knowledge test, the respondents were given the search task. Search task contains the problem scenario and instruction of what respondent have to look for. Theoretically, the search task will influence the information need (Broder, 2002). In this study, the search task chosen is related to Malaysian's homestay. Homestay is considered as general knowledge as homestay is a world wide concept.
The search activity was performed through the search interface. Search interface is developed to capture respondents' queries and store into the search log. This interface will send the queries to Google search engine for processing and returning the results. Upon completing the search session, respondents were given the questionnaire to access their satisfaction towards the results of the search engine. This questionnaire is divided into two sections: Section I and Section II. In Section I, the respondents are required to evaluate the information that they have obtained from the online searching activity.
The questions were designed to access the respondents' satisfaction on the information return by the search system. Section II is used to collect the respondents' demographic information. The search log that contains the queries, the date and time of the search, session number/ID and user's computer IP was analysed for frequencies of attempt and types of queries.
The respondent for this study is undergraduate students. Undergraduate student is consider significant to this study as this group of students are actively use Internet to find information that can support their learning (Jefferies and Hussain, 1998;Dalgleish and Hall, 2000;Zoe and DiMartino, 2000;Selwyn, 2008;Fayed et al., 2006;Walraven et al., 2009) such as completing the assignment (Whitmire, 2004) and research theses (Junni, 2007;Sanni et al., 2009).

RESULTS AND DISCUSSION
In this study, Linear Regression was performed to examine the direct relationship between the independent variable user knowledge (topic understanding and search system understanding) and dependent variable search satisfaction. The result is presented in Table 1. The result shows that the coefficient determinant (R 2 ) is 0.16 indicating that the independent variables (user knowledge) explained 16%  The relationship between user knowledge and success of the searching can be described as the utilizing of the knowledge on the search task in order to achieve the best result. This has been shown by a number of related studies such as (Hirsh, 1995); (Gursoy and McCleary, 2004) and (Wu and Wang, 2006). The knowledge will influence the user behavior (Hunt, 2003) to formulate the best query that fit the search task. The user will then use their knowledge to evaluate the results of the search system and then decide whether they satisfied or not. If they do, the search will end, otherwise, it will be repeated until the user found what they want.
In this study user knowledge has been divided into two dimensions: Topic and system understanding. Topic understanding represents the knowledge that is related to the topic of the interest. In this study the topic of the interest is homestay, materialized as the search task. The familiarity with the topic enables users to search effectively with a richer set of concepts that are useful in query formulation (Sutcliffe and Ennis, 1998). This will contribute to the efficient (Kelly and Cool, 2002) and successful searching (Navarro-Prieto Scaife et al., 1999). The findings of this study has shown that there was a positively and significant relationship between topic understanding and search satisfaction (β = 0.29, p<0.01). This is inline with previous study such as (Fidel et al., 1999).
The system understanding represents the knowledge regarding the search system. The knowledge includes the information on what is the search system and the functionalities provided. Knowledge on the facilities provided by the system such as the assisted tool and Boolean operator has been shown to have a significant effect on the search performance, satisfaction and confidence (Chau et al., 2007). The finding of this study has shown that system understanding has a positive and significant relationship with the search satisfaction (β = 0.28, p<0.01).
Findings show that the low relationship between topic understanding and search system understanding. These scenarios happen because of respondent's own evaluation on the search results. This study found that, though, some respondents possess high knowledge but are not satisfied with the information obtained. On the other hand, some other respondents that possess low knowledge satisfied with the information obtained. This finding is inline with (Makinster et al., 2002).
However, this study confirms that both dimensions of user knowledge namely topic and search system understanding are equally important since they are statistically significant associated with search satisfaction. It can be concluded that topic and search system understanding can enhance the search satisfaction in the context of information need.

CONCLUSION
User knowledge typically topic and system understanding has been found to be the most important factor in web search activity. The results of the study has proved that the user knowledge have an impact on search satisfaction. The relationship between user knowledge and search satisfaction can be described as how users utilized their knowledge to get best results that fulfill their information need.