Perils of Simplistic Classifications of Universities : A Preliminary Case Study of Economics Departments

Universities represent a highly complex organizational structure that is beset with serious informational problems. A simplistic allocation rule based on university rankings is sometimes applied to fund disciplines/departments in order to promote efficiency. This study shows the pitfalls of using university rankings to glean information on rankings of disciplines, or departments, across universities. We offer a new measure to quantify the mismatch between university rankings and department rankings, and stress the need for using the rankings of departments for allocating research funds. We marshal evidence from Europe, USA and also from the global rankings of universities and departments of economics to argue that rankings of universities do not statistically explain the rankings of economics departments. Therefore, in order to foster competition and promote efficiency, it is mandatory to develop classifications/rankings of departments, reward good performance and publicise discipline-based research and teaching performance. This study then examines the research performance of staff members of departments of economics in New South Wales of Australia that homes some of World’s Top 100Universities [1].


INTRODUCTION
A typical university in the western world represents a highly complex organizational structure. Large universities must respond to the expectations of a wide range of stakeholders -notable among them are actual and potential students, professional bodies, academic communities, public and private funding agencies and society at large. Students demand high quality education at low prices, academics wish to have reasonable wages and excellent working conditions, funding bodies expect reasonable rates of social return for their funds, professional bodies ask for rigorous standards in research and teaching and communities want clean, professional and globally-competitive universities. Balancing these expectations is one of the daunting tasks of the management in universities. It is reasonably simple to set goals, but implementing these goals poses a serious challenge that is further aggravated by information holes. Thus, a typical university invokes management control issues under conditions of limited information and conflicting objectives. The performance of a university depends on the general level of cooperation and social capital that various stakeholders offer to each other. It makes the success of a university very much dependent on the social incentives of stakeholders, which derives from the overall academic culture of a university. It is widely believed that the management of university must use some kind of yardstick competition: compensation tied to the performance of a university relative to its competitors. Towards this end, the Federal Government of Australia is in the process of initiating significant changes to improve the performance of Australian universities. This study highlights the potential dangers of adopting a simplistic policy reform based solely on classifying Australian universities. Proposals by the Federal Government of Australia to classify Australian universities into three categories in teaching as well as in research will trigger wide-ranging debates in Australia. There is no gainsaying to the fact that the Australian higher education sector will need a careful and intelligent shake-up in order to get rid of inefficiency for promoting an academic culture in teaching and research that will further consolidate the global positioning of Australian universities -as reported in [1]. The current proposal does have elements that can have serious and irrevocable adverse impacts on higher education in Australia. This can cause "strategic piggybacking," which is a common concern in businesses with similar organizational features of a university. However the crying need of the day is to reform the sector by offering a classification of disciplines, not universities, on the basis of their research and teaching. This classification will foster healthy competition that will eventually reduce inefficiency.
This study will focus only on the discipline of economics and experts from other areas should educate the public and decision-makers by sharing their

Series1
Linear (Series1) discipline-based experiences. The goal of this study is two fold: first, we examine the American system that offers classification of universities as well as disciplines and examine the overlap between these groups. We also provide a quick international comparison as in [2]. Secondly, we will turn our eyes to a case study to assess the discipline of economics in universities of New South Wales -based on the findings in [1]. Once we do that the reader will get the flavour of dangers latent in the current proposal and appreciate the need for discipline-based classification as a first step towards the reform process.
Lessons from the US system and a quick international comparison: The US university rankings for 2005 are collected from [3]. The rankings of economics departments are taken from [4]. We chose the following classification of universities and universities as per the above two ranking series:  One can attempt to quantify the mismatch by looking at the absolute value of the difference in rankings of a university and its economics department. Suppose the rankings of economics department in the USA are given by a vector E and its i th element is e i . Similarly, let us assume that the rankings of universities are given by the vector U whose j th element is u i . If a university (department) is not in the rankings of university (department) we attach a rank 31 to the university (department). This index therefore gives an underestimation of mismatches.

Definition 1:
A mismatch between these two rankings for a university and its economics department is defined as the absolute value of the difference in these two rankings for a university and its department. We define mismatch as S i for the i th university and its economic department as: S is the vector of mismatches in which S i is an element.

Definition 2:
The unweighted index of mismatch for n universities (economics departments) is defined as M: Definition 3: The weighted index of mismatch is defined as M*: Two important points we have to bear in mind: first, the way we constructed the index, it depends on the arbitrary point that we assign to a university (department) that does not belong to the current rankings. One way to rectify the problem is to consider the entire set of universities and their departments for which it is difficult to get reliable data. We have chosen the top 30 universities (economics departments). The second caveat is that one can use a wide range of weighting rules. We use the rule that assigns a higher weight to a bigger mismatch. The detail of the findings is given in the appendix available upon a request from authors. We offer a summary of mismatch in Fig. 1 the horizontal axis labels the rankings of economics departments (X) and the vertical axis (Y) labels the rankings of universities. In a perfect world in which academic merits of a university perfectly signal the merits/qualities of its department, the relationship between X and Y will be given by a 45 0 straight line passing through the origin with a positive slope. Figure 1 shows how poorly university rankings reflect discipline-based rankings (economics in our case) and vice-versa.

Fig. 1: Mismatches in US rankings
As reported in the appendix we find that M=8 and M*=112. One can interpret M (=8) as the average ranking mismatch per university (department), which should ideally be zero. Once we use weights we find that the mismatch index rises to 112 per university / department, which should ideally be zero. What is important to note from the above figure is that the

Series1
Linear (Series1) mismatch is low only for a handful of universities (points close to the 45 0 line). Most of these mismatches are very large. It is also interesting to note that the correlation coefficient between these rankings is very low (R 2 =0.19) that partly reflects the mismatch. The point is that if we use a classification in terms of universities, it will hide very valuable information at the discipline level. More seriously, if a university has a strong reputation due to the existence of a few strong disciplines, or for historical reasons, researchers and teachers from other disciplines will not face any competition and have no incentive to lift up their profiles. Any attempt to profile universities must be preceded by careful rankings of individual disciplines. Otherwise the proposed changes will only stifle the sector without addressing the critical issue of efficiency. It is important to make a quick international comparison. The detail is provided in the appendix that is available from the authors. Fig. 2 Table 1b.
It is evident from the above that these two rankings are most consistent (less mismatches) in the US vis-àvis Europe. The European rankings are more consistent (less mismatches) than the World rankings. The correlation between these twin rankings is very low reflecting the need for using both rankings in judging the performance of a university and its principal areas of strengths. We now cast a look at the economics departments in New South Wales and take some lessons. Economics departments in New South Wales (NSW): A case study: Students, academics, governments and businesses pledge huge resources and other commitments that turn the wheel of fortunes of NSW universities. These stakeholders all need to know which are the best universities in NSW. A recent study [1] provides a comprehensive ranking of world universities in which University of New South West (36), University of Sydney (40), Macquarie University (68) and University of Technology (113) have shined bright. How do their economics departments stack up? The answer to this question is directly based on a provocative study undertaken [2] to examine  University of Sydney 3.
University of Western Sydney, 4.
University of New England 6.
University of New Castle 7 University of Wollongong 8.
University Technology Sydney Source: Calculated from Table 8 in [2]. the research performance of economics departments in Australian universities. We have used the data on individual performances to provide a ranking of academic economists in New South Wales. In the current conjuncture -beset with funding cuts, aggressive competition for full-fee paying students and escalating student-staff ratio in the higher education sector -the Australian Public will need to know how well their taxes are being utilised by Australian universities. This is the first attempt to etch out a list of high-performing economists working in these departments of NSW universities. This attempt is based on the data provided [2] and it hence carries all the weaknesses and limitations of [2]. This research does not pretend to lend a valedictory shove to this subject and instead intends to open up an open discussion on the performance of academic economists working in these world-class universities.
A critical finding of [2] is a relatively low productivity of Australian academic economists in comparison with their counterparts in Europe and USA -mainly in terms of their publication in the top 88 journals. It is hence imperative to cast a look at the productivity figures of NSW economists. A list of these high performers is provided in Table 2.

CONCLUSION
We argue that a simplistic classification of Australian universities will hide more and reveal little. Possibly it will give rise to serious problems of asymmetric information and "strategic piggybacking." In a wellreputed university a weak discipline will have little incentive to improve. In a weak university, a strong school will perish. It is therefore argued that there should be an appropriate sequencing of future steps: first and foremost, classify and rank departments/disciplines so that the stakeholders know strengths and weaknesses of each university. In the second stage bring out the classification of universities and use the rankings of universities and rankings of disciplines for funding allocation. In the US higher education, stakeholders use both a classification (rankings) of universities and rankings of disciplines. The use of one at the expense of the other can create enormous problems as highlighted in the index of mismatch, as reported in this study, between these rankings. We examine in detail the economics departments of New South Wales. We note that university classifications do not fully reflect the rankings of departments. More importantly, we see two cleavages: first,, two economics departments of the top 50 universities of our globe, namely University of New South Wales and University of Sydney, enjoy significant productivity advantages in terms of IAJP. Other departments in the region have almost comparable productivity in terms of IAJP, though some of them are in the rankings of the top universities of the world [1] . The second cleavage is that in each university only a handful of researchers determine their relative productivity while the majority in each department has a low productivity. These rankings will be seriously affected if some of these highly productive ones decide to move. The broad picture is interesting: at UNSW's department of economics 45% staff are productive that goes down to less than 30% for university of Sydney and less than 10% for UWS. Further research is necessary to understand the low productivity of the majority of academic economists working in the eight universities in New South Wales, Australia.