Tree Rotations for Dependency Trees: Converting the Head- Directionality of Noun Phrases

Corresponding Author: Ika Alfina Faculty of Computer Science, Universitas Indonesia, Depok, Indonesia Email: ika.alfina@cs.ui.ac.id Abstract: To overcome the lack of NLP resources for the low-resource languages, we can utilize tools that are already available for other highresource languages and then modify the output to conform to the target language. In this study, we proposed an approach to convert an Indonesian constituency treebank to a dependency treebank by utilizing an English NLP tool (Stanford CoreNLP) to create the initial dependency treebank. Some annotations in this initial treebank did not conform to Indonesian grammar, especially noun phrases’ head-directionality. Noun phrases in English usually have head-final direction, while in Indonesian is the opposite, head-initial. We proposed a variant of tree rotations algorithm named headSwap for dependency trees. We used this algorithm to convert the head-directionality for noun phrases that were initially labeled as a compound. Moreover, we also proposed a set of rules to rename the dependency relation labels to conform to the recent guidelines. To evaluate our proposed method, we created a gold standard of 2,846 tokens that were annotated manually. Experiment results showed that our proposed method improved the Unlabeled Attachment Score (UAS) with a margin of 32.5% from 61.6 to 94.1% and the Labeled Attachment Score (LAS) with a margin of 41% from 44.1 to 85.1%. Finally, we created a new Indonesian dependency treebank that converted automatically using our proposed method that consists of 25,416 tokens. The dependency parser model built using this treebank has UAS of 75.90% and LAS of 70.38%.


Introduction
Syntactic parsing is "a task of recognizing an input string and assigning a structure to it" (Jurafsky and Martin, 2008). In general, the approaches in syntactic parsing are divided into two types: Phrase structure and typed-dependency structure. Phrase structure focuses on identifying phrases and their recursive structure, while the type-dependency structure focuses on relations between words. Phrase structure parsing is also known as constituency parsing.
The dependency parsing has gained more popularity because of its applicability to a wide range of NLP tasks such as machine translation (Čmejrek et al., 2004;Galley and Manning, 2009;Jiang et al., 2016;Gao et al., 2017), information extraction (Niklaus et al., 2018;Gashteovski et al., 2019), question answering (Meng et al., 2017;Cao et al., 2018) and so on. These works have motivated the conversion of the available constituency treebanks to the dependency treebanks.
Several works had built constituent-to-dependency converter for English treebank (Johansson and Nugues, 2007;Choi and Palmer, 2010;De Marneffe et al., 2006;Schuster and Manning, 2016). These converters accept treebanks in the Penn Treebank format as the input. The Penn Treebank (PTB) is a constituency treebank in English (Marcus et al., 1993). The PTB annotation guidelines are considered as a de-facto standard in building constituency treebank.
For non-English treebank, many converters of constituency-to-dependency have been built, such as for Arabic (Žabokrtský and Smrz, 2003), Spanish (Gelbukh et al., 2005), French (Candito et al., 2010) and Sanskrit (Goyal and Kulkarni, 2014). In general, these works used a rule-based approach based on the target language's morphology and syntactic in converting constituency to dependency annotation.
Indonesian, a language of the Austronesian language family, is a low-resource language for Natural Language Processing (NLP) studies. Not only are dataset limited, tools for processing datasets are also rarely available.
As far as we know, the only constituency treebank available was developed by the Universitas Indonesia (UI) as the continuation of the development of their POS-tagger corpus (Dinakaramani et al., 2014). This treebank was later converted to the Penn Treebank format by (Arwidarasti et al., 2019).
As for the dependency treebank, there are two treebanks publicly available, both provided by Universal Dependencies (UD): Indonesian-GSD (McDonald et al., 2013) and Indonesian-PUD (Zeman et al., 2017). However, according to , these two dependency treebanks do not fully conform to Indonesian grammar, especially the tokenization and POS tagging annotation. The Indonesian-PUD recently had been revised by . This situation motivated us to convert the only Indonesian constituency treebank (Dinakaramani et al., 2014;Arwidarasti et al., 2019) to a dependency treebank.
In this study, we present a different approach to convert constituency to dependency annotation. Unlike previous works that create the tool from scratch for the target language, we prefer to utilize the already available tools for the high-resource language like English and conducting some adjustments so that the final output will conform to the target language, in this case to Indonesian grammar.
We proposed a method to revise the output of an English NLP tool named Stanford Universal Dependencies (SUD) converter (Schuster and Manning, 2016) so that the resulting treebank conforms to Indonesian grammar. SUD converter was initially built for treebank in English. It was reported that the accuracy of this tool is more than 90% for an English treebank. However, when we use this tool for treebank in Indonesian, we found out that accuracy is very low of around 60%.
After conducting error analysis, we observed that one of the low accuracy causes is the difference in headdirectionality in some noun phrases between English-Indonesian. According to (Hawkins, 1990), there are two kinds of head-directionality: Head-initial or head-final. For head-initial, the second word describes the first word and for head-final, the opposite. While English usually uses head-final direction for noun phrases, Indonesian noun phrases usually use head-initial direction with some exceptions (Alwi et al., 2010). Figure 1 shows an example of noun phrases in English and its corresponding noun phrases in Indonesian. For the English noun phrase, the position of store as the head is after the book as the dependent, while for Indonesian noun phrases, the position of toko (store) as the head is before buku (book). To revise the dependency tree to have noun phrases with the correct head-directionality, we proposed a variant of the tree rotations algorithms for dependency trees. Our tree rotations algorithm will change the shape of the tree where some tokens get promoted to be the head and other being demoted to be the dependent of the new head. We named our proposed tree rotations algorithms for dependency trees as the headSwap algorithm.
We also use this algorithm to implement a rule to convert the head-directionality of noun phrases that were initially labeled as a compound. Upon applying this rule, we achieved an improvement of around 32% for UAS (Unlabeled Attachment Score). This result shows the effectiveness of our proposed method.
The contributions of our work are three-fold: 1. We propose a variant of tree rotations algorithms named headSwap that works on the dependency trees to swap the head between two nodes 2. We present a case in which the headSwap algorithm can be applied: Revising the head-directionality of noun phrases that initially labeled as compound for Indonesian treebank 3. We produced a new dependency treebank for Indonesian that had been made public 1 We believe our proposed headSwap algorithm can also be applied not only for noun phrases but also for other phrases such as verb phrases, prepositional phrases, etc. Since the head-directionality differences do not only happen between English and Indonesian, the headSwap algorithm can also be applied for dependency trees of other languages.
The rest of the paper is organized as follows: Section 2 describes the related work, section 3 presents differences between Indonesian and English noun phrases; section 4 describes our proposed method; section 5 discusses the experiments and results and finally, section 6 presents the conclusions and future work.

Related Work
In this section, we discuss dependency trees, Universal Dependencies (UD) and Stanford UD converter.

Dependency Trees
Dependency parsing is an approach to represent the syntactic structure of sentences in natural language using dependency grammar (Jurafsky and Martin, 2008). For dependency grammar, a sentence's syntactic structure is described in terms of the words and associated set of directed binary grammatical relations among the words. The arguments to this binary relation consist of a head and a dependent. Also, a label that describes the kinds of grammatical relation between the dependent and its head can be added.
Dependency graphs and dependency trees are used to represent the sentences for dependency parsing. In (Kübler et al., 2009), a dependency graph/tree is defined as follows:  A sentence is a sequence of tokens denoted by S = w0w1…wn where w0 is an artificial ROOT token.  Let R = {r1,…, rm} be the dependency relation type set  A dependency graph G = (V, A) is a labeled directed graph consists of nodes V and arch A, such that for sentence S = w0w1…wn the following holds: if (wi, r, wj)A then (wi, r0, wj)  A for all r  r  Any dependency graphs that is a directed tree originating out of node w0 and has a spanning node set V = VS are called dependency trees  A dependency tree G = (V, A) satisfies the singlehead property Figure 2 shows a dependency graph for a sentence of "He worked for the BBC for a decade." and Fig. 3 shows the corresponding dependency tree.

Universal Dependencies
Universal Dependencies (UD) is currently the defacto standard in annotating the dependency treebank. Before, some treebanks have their own annotation guidelines that made it difficult for cross-lingual parsing. A consistent and universal annotation guideline is needed for multilingual syntactic analysis.

Fig. 2:
A dependency graph for a sentence of "He worked for the BBC for a decade." (Zeman et al., 2017)

Fig. 3:
A dependency tree for a sentence of "He worked for the BBC for a decade." (Zeman et al., 2017) Manning, 2008). Stanford dependencies scheme was designed to represent English grammatical relations between words in a sentence. In 2014, the Stanford dependencies were adopted to create universal dependencies that can be applied to other languages to support cross-linguistically parsing, named Universal Stanford Dependencies (USD) (De Marneffe et al., 2014).
Finally, several initiatives agreed to create a new standard named Universal Dependencies by introducing annotation guidelines and a set of treebanks called Universal Dependencies v1 (UD v1) (Nivre et al., 2016). The recent version of the annotation guidelines is the UD v2 (Nivre et al., 2020), which has a tagset of 17 POS tags and 37 universal dependency relations with additional subtypes to adjust to the specific features of certain language.

Stanford Universal Dependencies Converter
Among the English NLP tools that can be used for low-resource languages like Indonesian is the Stanford UD converter (Schuster and Manning, 2016). This tool was built to convert an English constituency treebank in the Penn Treebank (PTB) (Marcus et al., 1993) format to a dependency treebank in the CoNLL-U format.
This converter is a revision to an initial converter (De Marneffe et al., 2006) that converts PTB-like treebank to dependency treebank in Stanford Dependencies (De Marneffe and Manning, 2008) representation. The SUD converter was reported to have UAS of 96.1% and LAS of 92.6% when evaluated on an English UD Treebank.
This UD converter tool is included in Stanford Core NLP . Since the constituency-to-dependency converter of the latest release of the Stanford CoreNLP only has output conforms to the UD v1 guidelines, Schuster had provided a tool 2 to convert it to the UD v2 format. We refer to the Stanford UD converter tool and its extension to UD v2 as the SUD+ converter.

Syntactic Annotation of Noun Phrases
In this section, we discuss the syntactic annotation of noun phrases. First, we present the differences between noun phrases in English and Indonesian, then we explain how UD v2 annotates noun phrases and finally, we describe how the SUD+ converter represents the noun phrases. Table 1 shows six types of Indonesian noun phrases with their corresponding head-directionality and the comparison with English. Note that on that table and the rest of this study, we use the Part-Of-Speech (POS) tagset of the Penn Treebank (PTB) (Marcus et al., 1993) to explain syntax or rules.
Noun phrases in line #1-#5 on Table 1 already discussed in . We added the sixth syntax that is a special case of the 5th type (NN/NNP + NN/NNP), which involved nouns used to describe another noun's position. In (Alwi et al., 2010), the locative noun in Indonesian is discussed. Examples of such nouns are atas "above", dalam "inside" and antara "between". The locative nouns are used in prepositional phrase, with the syntax of di/ke/dari + [locative NN] + [NN/NNP] (Alwi et al., 2010). Table 2 shows some examples of noun phrases with locative nouns in Indonesian and English corresponding phrases. In (Alwi et al., 2010), di atas meja is parsed into di and atas meja, not into di atas (on) and meja (table).
We can see from Table 1 that Indonesian noun phrases usually have head-initial direction, except for noun phrases with quantity determiner and locative noun.

Noun Phrases in the UD v2 Annotation Guidelines
The dependency relations (deprels) in UD annotation guidelines are divided into two groups: Universal dependency relations and language-specific relations called subtypes. UD v2 defines 37 universal deprels and many subtypes that can be defined for special construction in certain languages.   The time modifier of a noun 2019 annual report Note: Words with bold font in the 4th column are words to be annotated with the corresponding deprel label. Among 37 universal deprels of UD v2, seven deprels are used to represent noun phrases. Table 3 shows the seven universal deprels for noun phrases and four subtypes that are usually used in the English dependency treebank. Note that the universal deprel clf that is used to label the classifier of a noun is rarely used in English, but this syntactic construction does exist in Indonesian and other languages.

How the SUD+ Converter Annotates Noun Phrases
In this study, we used the SUD+ converter to convert an Indonesian constituency treebank to a dependency one. We analyzed how the SUD+ converter represents the noun phrases. Table 4 shows the statistics of deprels that SUD+ converter used to annotate noun phrases in the Kethu treebank. We present the frequency of occurrences of each deprel along with the proportion of each head-directionality.
For deprel amod that is intended to label adjectives that describe nouns, SUD+ converter had already annotated them correctly since 98.40% of headdirectionally are head-initial as expected by Indonesian grammar. For the remaining 1.60% of them are noun phrases affected by English terms such as makro ekonomi "macro economy" or some exceptions in Indonesian grammar such as for pertama kali "first time".
We conducted a further analysis for deprel compound that occurred 4331 times (around 15% of total 28,262 tokens). Table 5 shows the 11 noun phrases syntax where deprel compound is used for one of its tokens along with the example in Indonesian, the expected head-directionality and the expected deprel. Table 4 shows that 99.54% of head-directionality of deprel compound are head-final, but the head-directionality of 9 noun phrases syntax for Indonesian are head-initial. Only the fourth syntax in Table 5 aligns with the SUD+ converter's output that was initially designed for the English treebank. Based on the Indonesian dependency treebank's annotation guidelines , almost all relations that initially tagged as a compound by SUD+ converter have the incorrect label.
For deprel det, we found that the SUD+ converter only labels quantitative determiners correctly since its syntax is the same as English. All demonstrative determiners are labeled as dep, a label used by UD v2 for unknown relation.
For deprel nmod, 84.37% of tokens are already tagged correctly since the expected head-directionality is headinitial in Indonesian grammar. We found out that most of the errors caused by SUD+ that uses nmod as the default value when there are options either to use nmod or obl.
For deprel nmod:poss, SUD+ converter has been correctly labeled the relation between the noun and possessive pronoun with 100% accuracy. However, it fails to recognize the possessive relationship between noun and noun. For deprel nmod:tmod and nummod, since both head-directionality are possible for them, we can not evaluate the correctness of SUD+ annotation using this data.
We can see that among the seven deprels used by the SUD+ converter to represent noun phrases, the deprel amod is the one that best fits Indonesian grammar, while the deprel compound is the least compliant. The deprel det and nmod:poss are also already 100% correct, but other cases that should be label det or nmod:poss are still tagged with deprel dep.
Based on this analysis, we decided to propose a method on how to revise the annotation for the deprel compound. Specifically, we propose the method to convert its the head-directionally from head-final to head-initial. NNP + NNP Bill Gates initial flat:name 8 CD + CD 5 juta "5 million" initial flat 9 FW + FW net buy initial flat:foreign 10 FW + NN rating lembaga "institutional rating" initial nmod 11 FW + NNP rating LBPN initial nmod Note: words with bold font in the 3rd column are words to be annotated with the corresponding expected deprel label

The Proposed Method
This section presents our proposed method of conducting tree rotations for dependency trees to convert the head-directionality of noun phrases.

Tree Rotations for Swapping the Head
Our proposed tree rotations algorithm's objective is to swap the head between two nodes in a dependency tree. If node A is initially the head of node B, we want to change the tree, so that node A becomes the dependent of node B. The tree rotations should preserve the requirement for a dependency tree (Kübler et al., 2009). We named this proposed tree rotations algorithm the headSwap algorithm.
To illustrate the headSwap algorithm, we will use a sentence as the example: "Pemkot Delhi berencana mendatangkan monyet dari negara bagian Rajasthan." (The Delhi city government plans to bring in monkeys from the state of Rajasthan.). Figure 4a shows the initial dependency graph given by the SUD+ converter to this sentence and Fig. 4b is the expected dependency graph.
There are three dependency relations with label compound in Fig. 4a: Between token Pemkot and Delhi where Delhi becomes the head, between token negara and bagian where bagian becomes the head and finally between token bagian dan Rajashtan. We want to swap the head-directionality of those three pairs of tokens, as shown in Fig. 4b, so that in the relation between Pemkot and Delhi, token Pemkot will become the new head. The same situation applies to the other two pairs. Note that we also need to swap the parent and the dependents of the respected tokens. The parent and the children of the old head become the parent of the new head.
In this study, we work on the dependency trees in the CoNLL-U format 3 . Among the ten fields in the CoNLL-U format, we will utilize only five fields: ID, FORM, 3 https://universaldependencies.org/format.html UPOS, HEAD and DEPREL. Furthermore, we designed a data structure for token data with five attributes: ID, FORM, UPOS, HEAD and DEPREL. We define four arguments for the headSwap procedure, as shown in the Algorithm 1, as follows: 1. tokenList, contains the tokens data in a dependency tree 2. oldHeadID, the ID of the old head 3. newHeadID, the ID of the new head 4. moveDepFlag, the boolean flag whether the dependent(s) of the old head need to be moved to the new head  This procedure swaps the head from the old head to the new one. If the flag is true, then all the old head's previous dependents will be moved to the new head.

Applying the HeadSwap Algorithm to Revise the Compound Noun Phrases
In this subsection, we explain in more detail how to convert the head-directionality for compound nounphrases in a dependency tree. The procedure is shown in Algorithm 2. To illustrate the proposed rule, we use the sentence in Fig. 4. This sentence consists of 10 tokens with ID of 1-10. First, we need to generate the list of tokens that become the head of the noun phrases contains the compound label. We named this list phraseList in the procedure. The phraseList is a list of A  B where A is the head and B is the dependent(s) of A. For our example, we have three pairs of tokens with the compound label: Pemkot-Delhi, negara-bagian and bagian-Rajashtan. For the first pair token #2 becomes the head of token #1, for the second pair token #8 is the head and for the third pair, the head is token #9. Hence, we have three phrases: 2  {1}, 8  {7} and 9  {8}.
Secondly, for each phrase in phraseList, we need to apply the headSwap algorithm to change the headdirectionality. However, since in Table 5, there are cases where the compound noun phrases already comply with Indonesian grammar, i.e., locative nouns case, we have to set the value of the skipFlag variable to true so that the phrase is not to be swapped. Otherwise, we applied the headSwap algorithm.
Finally, after applying headSwap, we need to update the phraseList. Table 6 shows how the phraseList was updated from the initial to final state. After each headSwap, we update the head and dependency information.

Revising the Dependency Relation Labels
Besides revising the head-directionality of noun phrases in the dependency tree, we also proposed a set of rules named rename to revise the dependency relation labels. These rules were designed by observing the recent version of an Indonesian-PUD treebank revised by . Table 7 shows the design of our 32 proposed rules to improve the accuracy of dependency relation labels.
The decision to rename was made based on the information of the current deprel label of a token (old deprel), the POS tag of the token (child POS) and the POS tag of the head of the token (parent POS). In some cases, the decision was based on more detailed information. For example, for label nmod:lmod that is used to represent the locative nouns, this rule needs to have a list of locative nouns in Indonesian.
The rule rename currently only revised the label for a token initially labeled as compound, compound:prt, dep, nmod, nmod:tmod, nsubj and obj. In future work, we can add more labels after conducting further analysis of the annotation guidelines.  For deprel compound, we created seven rules (#1-#7) that were aligned with the discussion in section 3, especially with Table 5 that discusses the expected deprel label for each syntax that was initially labeled as compound.
For deprel compound:prt, since in English, there is a unique construction of verb compound with syntax "Verb (VB) + Particle (RP)" such as give up, take down and so on, SUD+ converter labels each occurrence of this syntax to compound:prt. However, for Indonesian grammar, such syntax is not for verb compound, but for foregrounding particles such as lah, kah, tah and pun (Sneddon et al., 2010). Rule #8 was designed for this problem.
The deprel dep is used to label unknown relations. The SUD+ converter used this label if it does not familiar with the syntax on the processed treebank. So, it is important to rename this label completely to other labels. To rename this label, we proposed 17 rules (#9-#25) for various cases.
We also decided to revise the label of deprel nmod that in UD v2 is used to represent the nominal modifier of a noun. In UD v2 annotation guidelines, the nominal modifier of a noun is labeled as nmod, while the nominal modifier of a predicate of verb/adjective should be labeled as obl (oblique modifier). However, we found out that SUD+ incorrectly labels tokens that should be labeled as obl as also nmod. Rule #28 was designed to fix this problem.
Rules #26, #27, #29 and #31 are related to noun phrases that are used as a temporal modifier. There are two subtypes for temporal modifier: nmod:tmod and obl:tmod. The label nmod:tmod is used if the phrase modifies a noun and obl:tmod is used if the phrase is to modify a predicate of verb/adjective. Rules #26 and #27 are used to revise the label for a token that initially labeled as nmod, rule #29 is used for the incorrect label of nmod:tmod that should be obl:tmod, while rule #31 are for tokens that is initially labeled as obj (the object of a transitive verb) but actually should be labeled as obl:tmod.
Rules #30 and #32 are related to passive verbs. Using rule #30, we revise the token that initially tagged as nsubj to nsubj:pass if its head is a passive verb and in rule#32 we change the label from obj to obl if the head is a passive verbs since grammatically passive verbs could not have an object.

Experiments and Results
This section discusses the experiments in converting the head-directionality of noun phrases and automatically building an Indonesian dependency treebank.

Dataset
In this study, we use the Kethu treebank produced by (Arwidarasti et al., 2019) as the initial constituency treebank to be converted to a dependency treebank. This treebank uses the same format as the Penn Treebank, both the Part-Of-Speech (POS) tagset, the bracketing and the annotation guidelines, which makes Kethu suitable as the input for the SUD+ converter that designed for the Penn Treebank. The Kethu treebank consists of 1,030 sentences and 28,262 tokens, with an average sentence length of 27.4 tokens per sentence. The genre of the sentences is news, mainly about the economy and finance.
To evaluate the proposed method, we chose 105 of 1,030 sentences of the Kethu treebank as the sample. For every 50 sentences, five first sentences are chosen to make sure a representative sample from the original treebank was created. This dataset consists of 2,846 tokens. We named this subset of Kethu treebank as the Kethu-105 treebank.

Creating the Gold Standard
We used the SUD+ converter to convert the Kethu-105 treebank to a dependency treebank in UD v2 format. We regarded the dependency treebank from Kethu-105 as the baseline treebank that will be converted using our proposed method so that the noun phrases comply with Indonesian grammar. To evaluate our proposed method, we need to create a gold standard.
The gold standard was created by revising the baseline treebank manually. Two annotators with a background in computer science and Indonesian linguistics revised the dependencies.
The gold standard creation consists of three phases that conducted iteratively: (1) Learning the UD v2 annotation guidelines; (2) Learning the proposed adjustment of UD v2 to Indonesian grammar by ; and (3) Revising the baseline treebank manually.
Several meeting was held to compare and discuss the annotation results between the two annotators until all of the inter-annotator disagreements were resolved. The resulting gold standard was named Gold Kethu-105.

Evaluating the Proposed Method
We evaluated the accuracy of our proposed method using MaltEval (Nilsson and Nivre, 2008). The quality measurements used are Unlabeled Attachment Scores (UAS) and Labeled Attachment Score (LAS) (Kübler et al., 2009). Table 8 shows the experiment results for three scenarios. First, the evaluation for the output of SUD+ converter as the baseline. The second scenario is by applying SUD+ converter plus the rule compound and the last scenario by combining SUD+ converter, rule compound and rename altogether.
We evaluated the accuracy of our proposed method using MaltEval (Nilsson and Nivre, 2008). The quality measurements used are Unlabeled Attachment Scores (UAS) and Labeled Attachment Score (LAS) (Kübler et al., 2009). Table 8 shows the experiment results for three scenarios. The first scenario evaluates the output of the SUD+ converter as the baseline. The second scenario is applying the rule compound to the output of the SUD+ converter. The last scenario is combining the SUD+ converter, rule compound and rename altogether.
For the baseline, the UAS and LAS are only 61.6 and 44.1%, respectively, which is very low. Since the SUD converter was reported to have an accuracy of more than 90% for an English treebank, we can see that some adjustments are needed to use this tool for non-English treebanks, especially for Indonesian in our case.
The result for the SUD+ converter plus rule compound is very good since the UAS improves significantly from 61.6 to 94.1% with a margin of 32.5% and the LAS improves 11.5% from only 44.1 to 55.6%. This result shows the effectiveness of our approach in converting the head-directionality of noun phrases in the treebank.
Finally, the last scenario combines the SUD+ converter, rule compound and rename to produce a dependency treebank that has UAS of 94.1% and LAS of 85.1%. This result shows that the rule rename has successfully improved the LAS with a margin of 29.5% from 55.6 to 85.1%.
Furthermore, we investigated what rules that have revised the deprel labels with reasonable accuracy.  Furthermore, we investigated what rules that have revised the deprel labels with good accuracy. Table 9 shows the comparison of the distribution of deprel labels between the Gold Kethu-105, the baseline (output of SUD+ converter) and the final result after applying the rule rename. We can see the big differences between the Gold Kethu-105 and the SUD+ converter for labels of compound, det, flat, flat:foreign, flat:name and nmod.
We suggest that besides the differences in noun phrases syntax between English and Indonesian, the SUD+ converter itself has not implemented the rule for deprel flat, a universal dependency relation in UD v2 annotation guidelines. The SUD+ converter labels all dependency relations that should be flat or its subtypes to the compound.
We also can see that the rule rename have errors less than or equal to 10% for deprel det, flat:foreign, flat:name and nummod. For deprel flat, nmod:tmod and nmod:poss the errors are more than 23% which are need improvement.

Building Dependency Parser
Furthermore, we built an Indonesian dependency parser using the supervised method by using the remaining 925 sentences of Kethu treebank that were not used as the gold standard on the previous experiment as the training dataset. We named this part of the Kethu treebank as the Kethu-925 dataset. We converted this dataset to a dependency treebank using the best approach we got before: The combination of SUD+ converter, the compound rule with the headSwap method and the relabel rules. Table 10 shows the general comparison between the gold standard (Kethu-105) and the converted Kethu-925 dataset.
We can see that the average sentence length of both treebanks is almost the same, that we can suggest the level of difficulty for conducting parsing in both treebanks are more or less the same. The converted Kethu-195 treebank has more variations on the POS tagset and deprel labels. We found that there is no token with POS SYM that is usually used for nouns like %, or $ on the gold standard, while there are 17 occurrences of them in the Kethu-195 dataset. Kethu-925 also has tokens with POS WRB that are typically used for adverbial interrogative pronouns like how, when, where and why. This POS tag is not represented in the gold standard.
After creating the converted Kethu-925, we used UDPipe (Straka et al., 2016), a trainable pipeline for tokenization, tagging, lemmatization and dependency parsing of CoNLL-U files to build the Indonesia dependency parser model. The evaluation results for the resulting dependency are UAS of 75.90% and LAS of 70.38%. We consider this result quite good since another work by? that also built the Indonesian dependency parser had UAS of around 82% and LAS of 79% with the treebank that fully manually annotated with the size of 19,440 tokens and the average sentence length of only 19.4 tokens per sentence.
Moreover, we also already converted the POS tagset of the Gold-Kethu-105 and the Converted Kethu-925 from PTB to UD v2, along with additional information required by the UD validation tool like sentence id, the original sentence before the tokenization and the tag SpaceAfter = No that is used to indicate whether after a token there is a space or not. Both datasets are already made public 4 .
From these two experiments, we have shown our approach's effectiveness in converting an Indonesian constituency treebank to a dependency one using an already available tool for English treebank and our proposed headSwap rules to convert the headdirectionality of noun phrases in Indonesian sentences.

Conclusion and Future Work
We proposed an approach to revise automatically a dependency treebank of a low-resource language that was initially produced by an NLP tool of a high-resource language. In our case, we need to revise an Indonesian dependency treebank produced by converting a constituency treebank to dependency using the Stanford UD converter (SUD+) that is designed for English treebank. We proposed a variant of tree rotations algorithm for dependency trees named headSwap to swap the head between two nodes to fix the wrong annotations.
We applied the algorithm to revise the headdirectionality of noun phrases with the dependency relation label named compound, a label based on our observation that the SUD+ conversion has more than 90% incorrect head-directionality for Indonesian noun phrases. The rule to revise the head-directionally of the deprel compound was named rule compound. Moreover, we also propose a rule called rename to revise the dependency relation labels so that the treebank conforms to the UD v2 annotation guidelines and the recent guidelines used by an Indonesian dependency treebank.
To evaluate the proposed method, we conducted experiments on an Indonesian constituency treebank named Kethu. First, we created a gold standard of 105 sentences and 2,846 tokens by annotating them manually. The evaluation shows that the rule compound successfully improves the UAS by a big margin of 32.5% from 61.6 to 94.1% and the LAS with a margin of 11.5% from 44.1 to 55.6%. The rule rename finally improve the LAS to 85.1%. These results show the effectiveness of our proposed method in revising the output of the SUD+ converter for the Indonesian treebank.
Furthermore, we also built the Indonesian parser model using a new training dataset that was built automatically using our proposed method. The training dataset consists of 925 sentences and 25,416 tokens. We built the parser using UDPipe, a trainable pipeline for dependency parsing, to build the model. Experiments show that the resulting dependency parser model has UAS of 75.90% and LAS of 70.38%, a quite good result for a small treebank that automatically converted and has an average sentence length of 27.4.
We want to build a bigger Indonesian dependency treebank using a semi-supervised approach using the treebank produced by this study as the seed for future work.