An Approach for Finding Semantic Relatedness Score Between Two Sentences Based on their Senses

Corresponding Author: Nazreena Rahman Department of Computer Science and Engineering, Kaziranga University, India Email: nazreena.rehman@gmail.com Abstract: Finding semantic relatedness score between two sentences is useful in many research areas. Existing relatedness methods do not consider its sense while computing semantic relatedness score between two sentences. In this study, a Word Sense Disambiguation (WSD) method is proposed which is used in finding the sense-oriented sentence semantic relatedness measure. The WSD method is used to find the correct sense of a word present in a sentence. The proposed method uses both the WordNet lexical dictionary and the Wikipedia corpus. The sense-oriented sentence semantic relatedness measure combines edge-based score between words depending the context of the sentence; sense based score which finds sentences having similar senses; as well as word order score. We have evaluated the proposed WSD method on publicly available English WSD corpora. We have compared our proposed sense-oriented sentence semantic relatedness measure on standard datasets. Experimental analysis illustrates the significance of proposed method over many baseline and current systems like Lesk, UKB, IMS, Babelfy.


Introduction
In the era of Natural Language Processing (NLP), recent applications require an efficient process for calculating semantic relatedness score between two sentences, phrases or words. Semantic relatedness is a mea-sure of conceptual distance between two words based on their meaning. Relatedness measure helps in many applications like information retrieval, text mining, web page retrieval and dialog systems. Text mining tasks include text clustering, text summarization, text categorization, sentiment analysis. Semantic relatedness helps in finding unknown knowledge from textual databases (Atkinson-Abutridy et al., 2004). For the biomedical applications, semantic relatedness calculation is used as an important tool in gene expression, gene clustering and disease gene prioritization (Pesquita et al., 2009;Lord et al., 2003;Pedersen et al., 2007). Semantic relatedness score calculation between query and input text sentences is important in query-based text summarization. Semantic relatedness calculation is an emerging research topic which is widely used in many recent application fields (Hasan et al., 2020a;2020b;Park et al., 2019;Zhu et al., 2019;Sadr et al., 2019).
Existing relatedness methods do not consider its sense while computing semantic relatedness score between two sentences. In this study, we introduce a sense-oriented semantic relatedness measure which incorporates sense of the sentence. Sense adds correct meaning to the sentence itself. Proposed method finds semantic relatedness score between two sentences based on specific sense present in WordNet.
The remainder of the paper is as follows: Section 2 presents the literature review on different techniques used in semantic similarity and relatedness measure. Section 3 states the motivation. Section 4 provides the brief introduction of the proposed technique. Section 5 describes the importance of sense for finding semantic relatedness measure. Section 6 presents the proposed unsupervised method for word sense disambiguation. Section 7 describes how to find the sense-oriented sentence semantic relatedness measure between two sentences in detail. Section 8 illustrates one example in detail. Section 9 includes the experimental analysis and discussion. Finally, this paper concludes with section 10 with some feasible future scopes.

Related Work
Different recent works are done in the area of natural language processing by providing different solutions for calculating semantic similarity between two text sentences. In this section, different related works will be discussed to explore the advantages and disadvantages of existing methods. Related works can be classified into following categories:  Semantic similarity based on vector space model  Semantic similarity based on ontology  Web search engine based measure

Semantic Similarity Based on Vector Space Model
The vector space model is an algebraic model. It has two steps, first step is to convert the text documents into vector of words and second step is to convert it to a numerical form to apply text mining techniques. It is widely used in information retrieval system (Meadow et al., 1992). In information retrieval, there is a compiled word list having n words. This word list includes all the meaningful words present in natural language. Each document is represented as a vector in n-dimensional space. Query is associated with each document and also is represented as a vector. To compute these values, tf-idf weighting scheme is widely used (Hiemstra, 2000). Necessary documents are retrieved based on similarity between document vector and query vector (Pawar and Mago, 2018). Similarity will be high if both vectors have same words. The advantages of this method are:  It can extract keywords from the documents  It matches with documents, hence does not depend on size of the document Although this retrieval mechanism is quite simple, but it has some drawbacks:  It does not consider order of the words present in a sentence  It does not consider semantic relatedness between two sentences  A same word might have different meaning for different sentences. Therefore, sense should also be taken into account

Semantic Similarity Based on Ontology
A lexical dictionary is used here to compute the semantic similarity between word pairs present in each sentence. We have to combine the word level similarity to get sentence level similarity. Sentence similarity is computed by aggregating semantic similarity values of all word pairs (Okazaki et al., 2003). The word 'relatedness' or 'similarity' is defined as the number of shared common features of meaning. Relations be-tween two words can be defined on the basis of thesaurus methods or distributional methods. Thesaurus based method normally use on-line thesaurus; example: WordNet, MeSH. Distributional methods depends on the distribution of two words in a corpus. There are mainly four different thesaurus (Gupta and Yadav, 2014)

Structure or Edge Based Measures
In case of structure or edge based semantic similarity measure, it finds the path length in ontology hierarchy structure. It computes the length of the path linking the words and on the position of the words in the ontology. Similarity between two words increases if they have more links among the concepts (Rada et al., 1989;Richardson et al., 1994):  Shortest path measures: Shortest path measure finds the closeness of two words in hierarchical semantic nets (Rada et al., 1989). Less similar words have greater distance. with less shared common features of meaning. Let W1 and W2 are two words, the equation to find shortest path is: SP (W1, W2) = Number of edges in the shortest path  Wu and Palmer Measure: This similarity measure finds the most specific common ancestor of W1 and W2. The most specific common ancestor is known as Least Common Sub-summer (LCS). Figure 1 shows that for the word Minicab and Gypsy_cab, the LCS is cab Wu and Palmer measure depends on depth of the two words in the taxonomy hierarchy. It also considers the depth of the least common subsummer (Wu and Palmer, 1994). This measure can be calculated: Here, shortest path(W1, W2) defines the shortest path length between two words W1 and W2 in the taxonomy hierarchy and D is the maximum depth of the taxonomy

Information Content Measures
Information content based measures depend on amount of common information shared between two words. If two words share more information in common, then the information content increases. In Information Content (IC), it finds the probability of the word W (P(W)). IC of W is calculated by counting the frequency of that word in the corpus divided by the total number of words present in the corpus: We will discuss a few information content based semantic similarity measures in this following section:  Resnik Similarity Measure: Resnik similarity (Resnik, 1995) directly depends on the commonly shared in-formation. Shared information is carried out by finding information content of least common subsume of two words W1 and W2 in the taxonomy hierarchy: Here, LCS stands for least common sub-sum.  Jiang and Conrath Similarity Measure: (Jiang and Conrath, 1997) calculates the similarity measure by considering the information content of two words along with information content of their least common subsume: This Jiang and ConDistance measure gives dissimilarity between two words. As the dis-similar value increases, it shows low relatedness as well as low similarity between two words.  Lin Similarity Measure: In case of Lin similarity (Li et al., 2003), it captures the similarity between two words as the ratio of amount of information shared between two words to the information possessed by each word:

Feature-Based Measures
Based on the description of the feature of words, feature-based similarity measures are introduced. Feature of a word contains the important information about the word. Each word is described in terms of set of words (definitions or glosses in WordNet) that contains the properties. Feature based similarity methods try to over-come the disadvantages of path based similarity method in the fact that taxonomical links present in a ontology do not necessarily give uniform distances. Here, each term is described by set of words which indicates properties or features. Similarity gets increased if two words share more common features. They can find similarity between two words if they are present in two different ontologies (Slimani, 2013):  Tversky's similarity Measure: Based on the description of the two words W1 and W2, (Tversky, 1977) has put forward a feature based similarity measure. It basically calculates the common words between the description of the W1 and W2, words present in W1 but not in W2 and words present in W2 but not in W1:  Each different Semantic Relation type (SR) is considered (such as is-a and part-of inWordNet) and take the highest value among them. Ssynsets(W1, W2) and Sglosses(W1, W2) both can be calculated as: where, w1 and w2 are the set of synsets and glosses for the words W1 and W2.

Hybrid Measures
In hybrid measure, it combines the characteristics of different similarity measures as it is described above.
This combination gets higher accuracy in many similarity measures:  Knappe Similarity Measure: This measure (Knappe et al., 2003) based on the fact that there are multiple paths to link two words in the taxonomy hierarchy. The proposed measure is given as: where, value of p is in between 0 and 1. AnsW1 and AnsW2 both defines the ancestor nodes of the words W1 and W2 respectively. The reachable nodes shared by both W1 and W2 is AnsW1  AnsW2.  Zhou Similarity Measure: Both path based and information content based measures are integrated by (Zhou et al., 2008) with the help of the following equations: Here, LCS(W1, W2) stands for least common sub-sum of W1 and W2. If the value of k = 1, then the Zhou Similarity Measure will be a path based measure if k = 0, then it will be an information content based measure. deepmax is the depth of the taxonomy

Web Search Engine Based Measure
This methodology computes relatedness based on web search engine results. It finds the frequently occurring words together (Bollegala et al., 2007). All the above mentioned applications are domain specific and use different methods to get the results.

Motivation
From the literature survey, it is seen that many different techniques have been applied in semantic similarity/relatedness measure till now, but sense of a sentence is not incorporated while finding similarity or related-ness score between two text sentences. It is also seen that, no semantic relatedness or similarity measure is there in which score can be found on sentence level. A word should take that sense which is suitable with context of the sentence. Existing semantic measures depend on individual words. Therefore, it is important to find exact sense of both words for which the senses are appropriate for the sentences. We have added sense as a measure while finding semantic relatedness measure. We have also added word order similarity between two sentences by finding its longest common substring. Finally, semantic relatedness measure, sense relatedness measure and word order measure are added in an equation to get proper sense-oriented semantic relatedness score between two sentences.

Proposed Method for Finding Sense-Oriented Sentence Semantic Relatedness Measure
Sense-oriented sentence semantic relatedness score between two sentences is calculated by combining the semantic relatedness score, sense relatedness score and word order similarity score. The process for finding semantic relatedness score, sense relatedness score and word order similarity score between two sentences is shown in the following Fig. 2 to 4. These following steps are described briefly for finding semantic relatedness between two sentences: 1. Pre-processing: Initially, pre-processing is done on text sentences by applying various techniques proposed by linguists. Steps for sentence preprocessing is already described in section 6 2. Word sense disambiguation on content word: To get proper semantic relatedness score between two sentences, appropriate sense of each word of the two sentences is disambiguated based on the context of the sentence. This word sense disambiguation technique is based on collocation score. The algorithm for extraction of proper sense to disambiguate a word is already described in section 6 3. Semantic relatedness score between two sentences: To get semantic relatedness score between two sentences, a lexical dictionary 'WordNet' is used. The detailed description of finding semantic relatedness score between two sentences is described in section 7.1 4. Sense based relatedness score between two sentences: To find the relatedness between two sentences based on their sense, sense based relatedness score is calculated in section 7.2 5. Word order score between two sentences: Order of words present in sentences also makes an impact while finding relatedness score between two sentences. The detailed description of word order similarity measure between two sentences is in section 7.3 6. Find sense-oriented semantic relatedness score between two sentences: To find out the senseoriented sentence semantic relatedness score, all the three measures are added in a equation described in section 7.4

Importance of Sense for Finding Semantic Relatedness Measure
Query based text summarizer extracts semantically query related sentences from input text sentences. In most cases, to find the semantic relatedness score between two words using WordNet, existing measures find the score with all the senses and give maximum score. In WordNet, a word has many senses. For different types of part of speech of a content word, we get different senses. Senses are the gloss or the definition. For a content word, if it has more than one sense then different number senses have different glosses. A content word may contain different senses for a same part of speech. Table 1 shows about different gloss of word interest present in WordNet. Each synset of interest contains its parts-of-speech and sense number. Gloss implies a dictionary-style definition.    One of the main issue in NLP is that presence of lexical ambiguity. Lexical ambiguity is a writing error that occurs when a sentence contains a word that holds more than one meaning. For example: We take two sentences: (1) Ram went to the bank to deposit money and (2) Ram went to the bank of river Brahmaputra. Here, the word bank has two different meaning; one is related with financial institute and other one is related with sloping land. Now, we find the semantic relatedness score between the content words of two sentences. In this example: We take two words that are bank and river where: bank word comes from the first sentence and river word comes from the second sentence. Both bank and river words are noun here. While finding the semantic similarity score, we have to give the word, then it's part of speech and sense number. When we do not give any particular sense as an input, WordNet takes automatically that sense for which it gets the highest semantic similarity score. UsingWordNet lexical dictionary, we get the following semantic relatedness scores for different measures listed in Table 2. Table 2 shows that by default almost all semantic relatedness measures take the first sense of bank as it gives maximums score with river. Table 3 provides the different senses present for the word bank. For the word river only one noun sense is present in the WordNet described in Table 4. Table 5 shows the trace definition present for bank#n#1, bank#n#2, bank#n#3 and river#n#1. Trace definition shows how the word is present in WordNet taxonomy. From these tables, it is quite clear that though the word bank is actually related with the financial institution, here, by default all semantic relatedness measures take an incorrect sense of bank. Therefore, finding sense of a word is much essential to get accurate relatedness score between two words as well as between two sentences and will help in eliminating lexical ambiguity problem. A sense of concern with and curiosity about someone or something Synset('sake.n.01') A reason for wanting something done Synset('interest.n.03') The power of attracting or holding one's attention (because it is unusual or exciting etc.) Synset('interest.n.04') A fixed charge for borrowing money; usually a percentage of the amount borrowed Synset('interest.n.05') (law) a right or legal share of something; a financial involvement with something Synset('interest.n.06') (usually plural) a social group whose members control some field of activity and who have common aims Synset('pastime.n.01') A diversion that occupies one's time thoughts (usually pleasantly) Synset ('interest.v.01') Excite the curiosity of; engage the interest of Synset('concern.v.02') Be on the mind of Synset('matter to.v.01') Be of importance or consequence Table 2: Relatedness/similarity score between 'bank' and 'river' Different semantic relatedness/similarity measure Relatedness score The relatedness of bank#n#1 and river#n#1 using vector pairs Li et al. (2009) 0.0353 The relatedness of bank#n#3 and river#n#1 using vector Li et al. (2009) 0.1958 The relatedness of bank#n#1 and river#n#1 using hso Hirst and St-Onge (1998) 0.0000 The relatedness of bank#n#1 and river#n#1 using Adapted lesk Banerjee and Pedersen (2002) 16.0000 The relatedness of bank#n#1 and river#n#1 using res Resnik (1995) 0.6144 The relatedness of bank#n#1 and river#n#1 using lch Leacock and Chodorow (1998) 1.4917 The relatedness of bank#n#1 and river#n#1 using lin Li et al. (2003) 0.0782 The relatedness of bank#n#1 and river#n#1 using jcn Jiang and Conrath (1997) 0.0691 The relatedness of bank#n#1 and river#n#1 using wup Wu and Palmer (1994) 0.4286 The relatedness of bank#n#1 and river#n#1 using path Rada et al. (1989) 0.1111 Sloping land 2 A financial institution that accepts deposits and channels the money into lending activities 3 A long ridge or pile 4 An arrangement of similar objects in a row or in tiers 5 A supply or stock held in reserve for future use 6 The funds held by a gambling house or the dealer in some gambling games 7 A slope in the turn of a road or track 8 A container (usually with a slot in the top) for keeping money at home 9 A building in which the business of banking transacted 10 A flight maneuver; aircraft tips laterally about its longitudinal axis (especially in turning) A large natural stream of water (larger than a creek) *Root*#n#1 entity#n#1 physical entity#n#1 object#n#1 geological formation#n#1 slope#n#1 bank#n#1 bank#n#2 *Root*#n#1 entity#n#1 abstraction#n#6 group#n#1 social group#n#1 organization#n#1 institution#n#1 financial institution#n#1 depository financial institution#n#1 bank#n#3 *Root*#n#1 entity#n#1 physical entity#n#1 object#n#1 geological formation#n#1 natural elevation#n#1 ridge#n#1 bank#n#3 river#n#1 *Root*#n#1 entity#n#1 physical entity#n#1 thing#n#12 body of water#n#1 stream#n#1 river#n#1

Proposed Unsupervised Method for Word Sense Disambiguation
The overall process for finding the sense of a target word is shown in the following Fig. 5.
Following steps are described briefly for finding sense of a word present in a text sentence using collocation score:  Pre-processing: Initially, pre-processing is done to remove the unwanted words from the text sentence.
Here, the unwanted words mean stop words and person, location or organization's name. This makes the text sentence a lighter one. Following techniques are applied to do the pre-processing of text document:  Part of Speech Tagging: To classify the words on the basis of part of speech category, part of speech tagging (Bird et al., 2009) is done. part of speech tagging classifies the content words. Tags include noun, adjective, verb and adverb  Named Entity Tagging: To distinguish different Names person, location or organization names, we do the named entity tagging (Bird et al., 2009). We will not consider person's name to find semantic relatedness as it is not present in Lexical resources  Stop Word Removal: It is better to filter words like out a, an, the, in etc. which do not give any semantic meaning to the sentence. This is known as stop word removal in text mining applications.
Here, we use stop word list stores in Natural Language Tool Kit (NLTK) in python  Stemming: Finally, stemming is done to the content words. Stemming brings the word to its root or base form. For example to convert a word from plural to singular root form (girls to girl) or removing ing from a verb (singing to sing). Number of algorithms are available to do the stemming in natural language processing  To get the correct sense of a word, we take collocation feature. Following section shows how collocation feature can be applied for finding word sense

Finding Collocation Score between Two Words
Collocation refers to the word or phrase that is often used with other word or phrase. With collocation, we can find what words occur near other words. The computational technique that finds commonly collocated words or phrases in a document or corpus is known as collocation extraction. Collocation score between two words is calculated by finding the number of occurrence of those words together in a corpus. Here, Wikipedia corpus (Denoyer and Gallinari, 2006) is used. The co-occurrence between two terms is calculated by finding its bi-gram frequency. Collocation gives the associativity between two words. For example: Car and bike are two concepts or words that are semantically similar. They have some common features like wheels or have common function like transport. In contrast, car and petrol both are associated as they occur frequently in language and space. This can be said as functional relationship. Association and similarity both are neither mutually exclusive nor independent. Car and patrol both are related two both relations to some degree (McRae et al., 2012;Plaut, 1995). To find the bi-gram collocation score for each sense of word w1 (McKeown and Radev, 2000), we find frequency of occurrence of words present in the sense definition for w1 with other words present in the sentences and take the maximum one. For example, we take one sentence Mary treated John for his injuries. To find the exact sense of the word treat, we first find out the all the senses present for treat in WordNet. Senses are the glosses or the definitions. The method finds the collocation score of each word present in the gloss with the word present in the sentence. Here, one gloss for treat is interact in a certain way. The content words present in the sentence is injuries and in the definitions are interact, certain, way. After finding collocation score of interact, certain, way with injuries, the method takes the highest score. In this way method will calculate for each sense and finally we take that sense for which the collocation score is maximum. The proposed WSD method will work for all the content words present in WordNet except the person's name. We will not consider the person's name, but of course, we will include an organization or location's name using standard Named Entity Tagger (Perkins, 2014). The proposed method considers mainly the content words as they carry the salient information. Content word includes noun, main verb, adjective and adverb. First, the proposed WSD method finds all the senses of the target words present in the WordNet. Senses are the glosses. For each sense of a target word, we have removed the stop words. We also remove the stop words from the sentence in where target word is present. To find the collocation score between two words (one word is from the gloss of w and other word is from the sentence w'), we use the following Equation 1. Here, Wikipedia corpus is used (Denoyer and Gallinari, 2006): log 2 x sizeCorpus w w span collocation score w w While finding the collocation extraction score, we are giving the flexibility that if the two words are not together in Wikipedia, we increase the window size upto 3. We consider the span size as 3 because it works best for our proposed WSD method. We will search for bi-gram frequency where the words may be separated by three other words in the text of Wikipedia. To find the sense of a word present in a sentence, initially, we get a set of probable senses of the word. Now for each sense, we calculate the collocation score between each content word of every gloss of a sense with all other content words present in the same sentence. Same process will be followed for every sense and finally we take that sense for which the collocation score is maximum. The Collocation Score (CS) of a sense (gloss) for a Target Word (TW) present in the sentence is: After finding the collocation score of the set of all senses of TW, we consider that sense of TW for which the collocation score is maximum.

Finding Exact Sense of a Word Present in a Sentence
The proposed WSD method is implemented to find the sense of a word which will further helps in calculating semantic relatedness score between query and input text sentences. Following Algorithm 1 gives the systematic steps to find the sense of a word: Algorithm 1: Steps to Find Exact Sense of a Word present in the Sentence Data: Target Word (T) and Other Words Present in the Sentence (OT) Result: Sense Number and Gloss of that Sense of T for Which the Collocation Score is Maximum 1 Do the part of speech of T 2 if T is a person's name then 3 Go to step 24 4 end 5 else 6 Find out the senses of T using WordNet where 7 if T has more than one sense then 8 for each sense (s) of T do 9 Do the pre-processing of s 10 Do the pre-processing of OT 11 Find out the collocation score between s and OT by using Equation 2  12 end 13 Extract the sense number and the gloss of T for which we get the maximum collocation score 14 if T has more than one sense having same maximum collocation score then 15 Extract Relatedness implies larger set of potential relationship between words. Antonyms are highly related words but they are less similar. Human and women are not similar but highly related than human and car while girl and women have similarity. Thus Similarity is a sub case of Relatedness. We prefer to use semantic relatedness score to find query relevant sentences. It is quite clear that semantic measures needs actual sense while finding its score between two words. A word's sense depends on other words present in the same sentence. In case of semantic relatedness calculation, default sense does not give the assurance of accuracy of score for specific sentences. Hence, sense detection is important to increase accuracy of measures. To find sense-oriented semantic relatedness score we combine sentence semantic relatedness score, sense relatedness score and word order relatedness score. In the following section, sense-oriented semantic relatedness score calculation is shown sequentially.

Finding Semantic Relatedness Score between Two Sentences
To find the relatedness between two sentences, semantic relatedness score is calculated between two words present in both sentences. Semantic relatedness mea-sure gives the relatedness score between two sentences on the basis of the meaning of the sentences. This relatedness measure is based on word to word relatedness.
We consider only the content words while calculating the semantic relatedness score between two sentences. Initially, pre-processing step removes the stop words. Stemming is also done to get the root form of a word. Before finding relatedness score between two words, it is important to find exact sense of that word. Full description of the word sense disambiguation method is described in Algorithm 1.
This relatedness score uses Path Weight measure (Pedersen et al., 2004;Hirst and St-Onge, 1998). It is a semantic relatedness measure between two words based on path which is described in WordNet lexical dictionary. It is found from the literature survey that this measure includes more relations and can find relatedness score between different part of speech (Patwardhan et al., 2003). It classifies the relations in WordNet as up-ward, downward or horizontal directions. Higher score implies the shorter path length and less changes of directions. While finding the semantic relatedness score between two words, the proposed method will give the accurate sense number and its part of speech along with the respective words. The equation to find Semantic Relatedness Score (SEM_R_S) between two words w1 and w2 is: Here: sense_no_1 = Sense number of W1 p_o_s_1 = Part of speech of W1 sense_no_2 = Sense number of W2 p_o_s_2 = Part of speech of W2 Here, c and k are the constants and values are c = 8 and k = 1. The maximum semantic relatedness score between two word is 16 which signifies that two content words are identical. The minimum score is 0 which signifies there is no semantic relatedness between them. Following equation ?? is used to find the Semantic Relatedness Score (SEM_R_S) between two sentences s1 and s2: The overall process to find semantic relatedness score between two sentences is shown in the Algorithm 2.
Algorithm 2: Steps to find semantic relatedness score between two sentences Data: Two Sentences (s1, s2) Result: Semantic Relatedness Score between s1 and s2 (SEM_R_S(s1, s2)) 1 Do the pre-processing of s1, s2 by using the steps mentioned in section 6 2 for each content word w1 in s1 and w2 in s2 do 3 Find out the sense number of w1 and w2 by using the Algorithm 1 4 end 5 for each content word w1 and w2 from s1 and s2 do 6 Find out the SEM_R_S ((w1, sense_no_1, p_o_s_1), (w2, sense_no_2, p_o_s_2)) by using the Equation 3 7 end 8 Find the SEM_R_S(s1, s2) between s1 and s2 by using the Equation 4 Finding Sense Relatedness Score between Two Sentences We have already discussed that finding relatedness measure between two sentences on the basis of its sense is quite important. Proposed WSD method finds the sense of each content word present in both text sentences by using the mentioned method described in Algorithm 1. After doing the pre-processing of two sentences as mentioned in section 6, we have disambiguated the sense of each content word if it has more than one sense. We get the gloss for each content word. We will do the stop word removal and stemming on the content words to get the root form of the content words present in the gloss. Here, we find the sense relatedness score between two sentences by finding the common content words present in the gloss of a sense of a content word in the first sentence with the words present in the second sentence. The method uses the Equations 5 to 7 to find the Sense Relatedness Score (S_R_S) between two sentences s1 and s2: The overall process to find the sense relatedness score between two sentences is shown in the Algorithm 3.

Algorithm 3: Steps to Find Sense Relatedness Score between Two Sentences
Data: Two Sentences (s1; s2) Result: Sense Relatedness Score between s1 and s2 (S_R_S(s1, s2)) 1 Do the pre-processing of s1 and s2 by using the steps mentioned in section 6 2 Find out the gloss of the sense of the content words by using the Algorithm 1 3 Do the pre-processing of the gloss of the sense of the content words by using the steps mentioned in section 6 4 Find the S_R_S(s1, s2) between s1 and s2 by using the Equation 7 Finding Word Order Similarity Score between Two Sentences On the basis of same sequence of words present in the two sentences, word order similarity provides how much two sentences are similar. Finding longest common substring between two sentences adds more impact on similarity measure. Word order similarity method depends on the order of words present in both sentences. It helps in signifying the relatedness between two sentences though they share same words. Example: (a) Ram killed Shyam and (b) Shyam killed Ram. These two sentences use same content words but we can see that sentence a and sentence b have opposite meaning. Longest common substring can easily distinguish that meaning of sentence a is not similar to the meaning of sentence b. We can take another example: Sentence 1 is Narendra Modi's visit to China and sentence 2 is Ram Nath Kovind's visit to China. Both sentences are different though they carry maximum common words. Hence finding longest common substring can identify the differences present in both sentences. It can also find similarity between two sentences if numerical data present in both sentences in a same order. Ontology based semantic relatedness measure can not find this type of similarity as lexical dictionary does not contain numerical data or some proper nouns. Example: Sentence 1: In 2006, Ram came to Assam to meet his friend Rahim and sentence 2: In 2006, Ram came to Guwahati to meet Rahim. Here longest common substring is in 2006, Ram came to. Hence, it also helps in finding similarity for numerical and proper noun words. Here, we will not do any pre-processing task. The method uses the following equation ?? to find word order similarity between two sentences s1 and s2: Finding Sense-Oriented Sentence Semantic Relatedness Score between Two Sentences Semantic relatedness measure gives how much two sentences are related to each other. Sense relatedness measure defines how much two sentences are related on the basis of its sense definition and word order similarity gives information relations between sentences. All these semantic and same sequence of words information play an important role while finding sense-oriented sentence semantic relatedness score between two sentences. Hence, the sense-oriented sentence semantic relatedness measure between the two sentences is the combination of all these three measures: Semantic relatedness, sense relatedness and word order similarity.
The sense-oriented sentence semantic relatedness score between sentences s1 and s2 is given in Equation 9: Here  +  +  = 1 and ,  and are weighting parameters. They specify relative contributions to the sense-oriented sentence semantic relatedness measure of semantic, sense and word order measures. As semantic information caries more relevant information, therefore more weightage is given to semantic relatedness measure (Wiemer-Hastings, 2000). Considering the view that the word order information plays a subordinate role in finding relatedness between sentences, hence the weightage given to word order information is minimum. The overall process to find sense-oriented sentence semantic relatedness score between two sentences is shown in the Algorithm 4.

Algorithm 4:
Steps to find sense-oriented sentence semantic relatedness score Data: Two Sentences (s1; s2) Result: Sense-Oriented Sentence Semantic Relatedness Score (Sense_Sem_Rel) of s1, s2 1 Find semantic relatedness score between s1 and s2 by using the Algorithm 2 2 Find sense relatedness score between s1 and s2 by using the Algorithm 3 3 Find word order similarity score between s1 and s2 by using the Equation 8 4 Find sense-oriented sentence semantic relatedness score between s1 and s2 by using Equation 9 Implementation of the Proposed Sense-Oriented Sentence Semantic Relatedness Score Finding Method To illustrate the implementation of the sense-oriented sentence semantic relatedness measure between two sentences, we have elaborated the proposed method with three sentences S1, S2 and S3:  Mary treated John for his injuries  John treated Mary to a nice dinner  Mary gave medicine to John for his treatment Initially, we do the part of speech tagging and Entity named tagging of these sentences S1, S2 and S3. Following Table 6 shows the tagging of all three sentences along with the Table 7 for tags and its description for part of speech tagging and named entity tagging.
We only use the content words (excluding the person's name) for finding semantic relatedness score between two sentences. The content words for first sentence: 'treated', 'injuries'; second sentence: 'treated', 'nice', 'dinner' and third sentence: 'gave', 'medicine', 'treatment'. Here, the proposed method shows how exact sense of the tagged words can be achieved for S1 sentence. Table 8 shows different senses present for word 'treated' in WordNet. Table 9 gives the content words present in different senses of 'treated'. For the word 'injuries', senses are present in Table 10 and content words are shown in Table 11. Table 12 and 13 shows different collocation score of each sense of a content word 'treated' and 'injuries' with other content words present in the sentence. Table 14 gives the correct the senses for all the content words listed below. Correct senses fit with the meaning of the sentence. Here, POS means part of speech. Table 15 gives different semantic relatedness scores between content words present in the above sentences. Using Algorithm 2, we get the semantic relatedness score among the sentences. Now, to find sense relatedness score, we use Algorithm 3. We find word order information between two sentences using equation 8. All the scores are listed in Table 16. Finally, senseoriented sentence semantic relatedness score between sentences are calculated using Algorithm 4 for  = 0,5,  = 0.3 and  = 0.2 which are empirically determined. Table 17 shows the final scores between sentences. From the above example, it is quite clear that though S1 is having more number of common words with S2 (Marry; treated; John), but S1 is more semantically related with S3. Our proposed sense-oriented sentence semantic relatedness measure can distinguish it.         Give (as medicine) Medicine n 02 (medicine) something that treats or prevents or alleviates the symptoms of disease Treatment n 01 Care provided to improve a situation (especially medical procedures or applications that are intended to relieve illness or injury) Table 15: Semantic relatedness score between two content words present in S1 and S2, S2 and S3 and S3 and S1 sentences Word pair Semantic relatedness score Treat-regale  0  Treat-nice  0  Treat-dinner  0  Injury-regale  0  Injury-nice  0  Injury-dinner  0  Treat-give  6  Treat-medicine  0  Treat-treatment  0  Injury-give  0  Injury-medicine  0  Injury-treatment  0  Table 16: Semantic relatedness, sense relatedness and word order information score between two sentences Sentence pair Semantic relatedness score Sense relatedness score Word order information score S1-S2 0 0 0.14 S1-S3 0.375 0.33 0.5 S2-S3 0 0 0.12 Table 17: Sense-oriented sentence semantic relatedness score between two sentences Sentence pair Sense-oriented sentence semantic relatedness score S1-S2 0.028 S1-S3 0.387 S2-S3 0.024

Dataset Description
A standard dataset of 65 noun pairs originally measured by (Rubenstein and Goodenough, 1965) is used widely to evaluate semantic similarity between two words. This dataset has used in many investigations and is established as a stable dataset for finding semantic similarity or relatedness measure (Sánchez et al., 2012). As stated by (Bollegala et al., 2009), to evaluate the accuracy of semantic similarity or relatedness measures is quite tough as it is a subjective human judgment. Rubenstein and Goodenough defined the first experiment as a group of 51 students having English native language. They accessed the similarity of 65 word pairs from ordinary English nouns and scaled it as 0 if pairs have low relatedness and 4 if they have highest relatedness. Miller and Charles (1991) did the same experiment in 1991 with 38 undergraduate students by taking the subset of 30 noun pairs. Resnik (1995) recreated the same experiment. Here, 10 computer-science graduate students and post-graduate students did this experiment. Finally, (Pirró, 2009) The Equation 10 states for Pearson's coefficient; here, xi corresponds human judgment of i-th element and yi stands for computed value of similarity score for i-th element and n gives the number of sentence pairs. Spearman's Coefficient 11 is computed by comparing the sentence similarity measure with human judgment. Here, di gives the difference between xi and yi:   To evaluate our proposed semantic relatedness measure between sentences, we use Pilot Short Text Semantic similarity benchmark dataset by (O'Shea et al., 2008). According to the (Li et al., 2006) explanation, to establish a measure for semantic similarity a survey is conducted by a panel of 32 participants, who are native English speakers. These participants marked the similarity measure for the sentences. Li et al. used 30 word couples of (Rubenstein and Goodenough, 1965) and replaced these word couples by their definitions from the Collins Cobuild dictionary (John, 1987).
Another popular dataset MirosoftSoft Parsphrase Corpus (MPSC) is used widely to evaluate whether two sentences are paraphrases or not (Dolan et al., 2004). To evaluate the proposed method, this dataset is used to determine how many paraphrases pairs present in the corpus are correctly identified. At present, MPSC is the largest publicly available paraphrase annotated corpus and is used extensively in evaluation. The matrices used for MPSC corpuse are: Experiments and Results using Li et al. Dataset Table 20 gives the mean human sentence similarity values with Proposed semantic relatedness method using Li et al. dataset. Our sentence similarity score is compared with mean human similarity. A few portion of the results are presented here. Here, for the 56th R&G number, the semantic relatedness value is highest as both sentences contain common words. From the table, it is seen that our proposed sentence relatedness score is much similar with mean human similarity.
Our proposed method's sentence relatedness achieves good Pearson correlation of 0.8987 and Spearman's Coefficient of 0.9091 with mean human similarity. Following Table 21 compares different Pearson's coefficient (r) and Spearman's Coefficient (r) values with (Li et al., 2006;Islam and Inkpen, 2008;Oliva et al., 2011;Taieb et al., 2015) algorithms using Li et al. dataset. The results got from proposed semantic relatedness measure are competitive and exceeds other existing measures.   Fruit or a fruit is something which grows on a A furnace is a container or enclosed space in 0.0475 0.0521 tree or bush and which contains seeds or a which a very hot fire is made, for example to stone covered by a substance that you can eat. melt metal, burn rubbish or produce steam. 6 An automobile is a car. In legends and fairy stories, a wizard is a 0.0200 0.0345 man who has magic powers. 10 An asylum is a psychiatric hospital A monk is a member of a male religious 0.0375 0.0431 community that is usually separated from the outside world. 16 An asylum is a psychiatric hospital. A cemetery is a place where dead peoples 0.375 0.245 bodies or their ashes are buried. 17 The coast is an area of land that is next to A forest is a large area where trees grow 0.0475 0.0568 the sea.
close together 23 A mound of something is a large rounded The shores or shore of a sea, lake, or wide 0.0350 0.0321 pile of it.
river is the land along the edge of it. 34 A car is a motor vehicle with room for a When you make a journey, you travel from 0.0725 0.0845 small number of passengers.
one place to another. 56 The coast is an area of land that is next to The shores or shore of a sea, lake, or wide 0.5875 0.5974 the sea. river is the land along the edge of it.  (Pedersen et al., 2005) which is implemented in Pywsd present in NLTK library in Python (Tan, 2014). But this method does not always give the accurate sense based-on the sentence's context. The proposed method also uses more number of relations present in WordNet.

Experiments and Results using MPSC Dataset
Identification of a sentence paraphrase pair depends on the interpretation of proposed sense-oriented sentence semantic relatedness measure (S1, S2). If semantic relatedness value between (S1, S2) >= [0, 1], then this sentence pair is considered as a paraphrase. Following Fig. 8 shows quite satisfactory results obtained from the MPSC data. We compare our F-measure value with STS (Islam and Inkpen, 2008)) and FM3S (Taieb et al., 2015) measure for different  values. For each paraphrase pair in the dataset, we consider threshold value  as 0.6 because we get highest accuracy for  = 0.6 as 76.83%. We calculate the semantic relatedness score using Equation 9 and consider it as a paraphrase, if threshold exceeds 0.6.
There is a lack of published research works for providing results on MPSC dataset. Hence, the proposed method is only compared with STS (Islam and Inkpen, 2008) and FM3S (Taieb et al., 2015) measures. Following Fig. 6 Figures 9 and 10 show the precision and recall values of the proposed sense-oriented sentence semantic relatedness calculation method. In case of precision value, it reaches maximum with  = 0.9 gives a value of 0.7738. It concludes that pairs of sentence considered as highly related by our proposed sense-oriented sentence semantic relatedness calculation method are treated as paraphrases in MPSC dataset. It is also seen that value of precision is nearly persistent. Actually, proposed method gives high value of relatedness which helps in getting good recall value for threshold  = 1. In comparison with STS and FM3S methods, for  = 1; recall value obtained by the proposed method is 0.4883, whereas for STS, it is 0.0054 and for FM3S, it is 0.4557.    (Corley and Mihalcea, 2005) 71.5 72.3 92.5 81.2 Combined(U) (Mihalcea et al., 2006) 70.3 69.6 97.7 81.3 Baselines Threshold-1 (Mihalcea et al., 2006) 33.8 100.0 0.44 0.87 Vector-based (Mihalcea et al., 2006) 65.4 71.6 79.5 75.3 Random (Mihalcea et al., 2006) 51 We also compare our method with other ontology based algorithms for calculating the semantic relatedness score between two text sentences. The Table 22 shows baseline methods and several other methods from (Corley and Mihalcea, 2005) and (Mihalcea et al., 2006) on test data. Here, results are also evaluated in terms of accuracy. Accuracy is defined as number of pairs correctly identified as paraphrases by total number of pairs in dataset. We obtain higher F-measure and precision at the cost of decreasing value of recall.

Conclusion and Future Work
Finding accurate semantic relatedness score plays a crucial role which helps in research areas of artificial intelligence. This paper presents a sense-oriented sentence semantic relatedness calculation method for finding the relatedness score between two sentences. First, it finds the sense of each word based-on sentence's context. This appropriate sense helps in finding accurate semantic relatedness score between two words. In fact, while finding relatedness between two sentences, already established relatedness measure does not consider the sense of content words. Hence, we add the particular sense of a word for finding relatedness score. It helps in getting accurate semantic related score. Thus, our semantic relatedness measure does not depend only on lexical knowledge based common human knowledge model, but also able to find sense wise semantic relatedness score. Secondly, we incorporate sense wise relatedness measure by finding number of similar words present in the sentence to the respective sense of each word. It definitely helps in calculating sense-oriented sentence semantic relatedness score between two sentences. Finally, incorporating word order similarity score enhances the accuracy of sense-oriented sentence semantic relatedness score. Different order of words gives different meaning of a sentence. Word order information between two sentences makes an impact to the sequence of same words present in the two sentences. Longest common substring is used here to find word order information between two sentences. Our method can find semantic relatedness score between those sentences in which words are not available in WordNet database. Our method can identify negations.
Our methodology is tested on two widely used benchmark datasets Li et al. and MPSC successfully. From the paraphrases detection task on MPSC dataset, proposed method performs well for higher threshold value of [0.6, 1]. We compare our results with human rated mean similarity values. The results achieved from these experiments conclude that proposed method shows much higher accuracy with improved correlation value of 0.8987 than many existing ontology based methods.
This sentence relatedness measure technique is applicable for short, medium or long type of sentences. This relatedness measure can be used in text clustering, text summarization, plagiarism detection and so on. The key point of this work is the use of sense while finding semantic relatedness between query and input text sentence in query-based text summarization. This senseoriented sentence semantic relatedness measure helps in extracting more semantically related sentences.
It also helps in solving lexical ambiguity problem. Lexical ambiguity means a word contains more than one meaning. As our proposed semantic relatedness score is based on the sense definition of the each content word present in the sentence, therefore it can also solve the problem of lexical ambiguity. Text summarization and text clustering both are widely used NLP applications. In future, we can use this sense-oriented semantic relatedness measure for creating text summarization and text clustering.