Quality research framework: Enumerative study versus analytic study from deming’s perspective

Problem statement: The right research framework is the only way to lead us to become champions in our respective disciplines and certainly to be recognized as an excellent referral center of knowledge for the world. Deming, a well-known quality guru, has defined two research frameworks, namely enumerative study and analytic study. They are totally different in their applications and implications. He criticizes analytic study as it is the most misguided and poorly taught research approach in most universities, as compared to enumerative study. Approach: Intensive literature reviews were conducted to obtain in-depth understanding concerning Deming’s teachings through books, websites and journals. Results: Enumerative study aims to describe the magnitude of the study problem with its possible correlated variable (s). The study objectives, its variables’ scope, hypothesis and research method are predetermined earlier. Common statistical tools in used are the t-test, chi-square, correlation and frequency. Outcome studies involve percentage prevalence data, new standard formula and linear modeling. The major pitfall of enumerative study is that its findings, recommendation and action for improvement could be wrongly interpreted, misleading and wrongly guided. It is because the statistically significant correlated variables do not truly support the cause to existing study problem. In analytic study, the main purpose is to answer the "why" question. It tries to identify the real variable(s) incur to the study problems. Statistical Process Control is the only statistical analytic tool in this particular research question framework. Conclusion: In summary, Deming’s emphasis is to focus on analytic study as it helps to generate new knowledge, rather than enumerative study. The latter is passive and does not help much either to prevent the problem encountered or to improve on the existing problematic situation.


INTRODUCTION
New knowledge can only be obtained through the use of relevant research frameworks. Continually searching for new knowledge means we are in a better position compared to our competitors. Perhaps the most interesting point to note is to learn from the fact that the manufacturing industries in Japan succeeded in penetrating the world market even after their defeat and surrender in World War II. The quality of Japanese products were previously inferior, but were soon in the short time frame of a decade famous because of they were cheap, reliable and excellent in quality. The reason behind such a drastic change was soon discovered in the late 70s by US academic researchers and industrial managers. Deming W. E., the famous quality guru was primarily responsible for many of the insights into this Japanese phenomenon and his writings have a wide influence in most advanced countries. The best examples are to be found in the various international standards he introduced, namely Product Quality 9001, Occupational Safety and Health 18001, Environmental 14001, etc. that were accepted world-wide by the International Standard Organization (ISO) and International Labour Organization(ILO) (Dyjack & Levine, 1996;Machida & Baird, 2001;Heizer & Render, 2000;Kingdom Management Training 2003). It is hoped that this brief discussion of Deming's quality perspectives would enlighten all parties concerned especially academicians, consultants, government agencies and auditors to better appreciate the importance of selecting the right research framework.

PURPOSE OF THIS ARTICLE
The purpose of this article is to highlight the importance of Deming's insights on both enumerative study and analytic study, and help clear the confusion most if not all academicians and industry researchers have with regard to these two research frameworks. This critical review is aimed at engaging local researchers and academicians so that they may become aware of the need to re-evaluate the choice of their research paradigm.

METHOD
In order to substantiate some of the points raised in this paper, the relevant books, journal articles, websites and electronic database system in Universiti Utara Malaysia and Universiti Kebangsaan Malaysia were used as reference sources.

LITERATURE REVIEW
Deming's influence on Japanese industries and community was first established in 1950. Since then, Deming's Prize for Quality has been awarded by the Japanese Union of Scientists and Engineers (JUSE) and it has played an important role in rebuilding the Japanese economy after the World War II (Latzko & Saunders, 1995). Japanese manufacturing industries benefited the most from Deming's proposal to introduce a new management theory, a move that finally led them to have the edge of product quality over their main US competitor. Wadycki & Sclove (1991:1) reckoned Deming's contribution as "Dr. W. Edward Deming, a PhD statistician, help Japan rebuild." In common practice, Deming defined two major research frameworks, namely enumerative study and analytic study, as shown in Table 1. The main focus of enumerative study is to describe the existing situation based on specific criteria. In contrast, analytic study serves to identify the reason(s) which cause a situation to happen.  Latzko & Saunders (1995:192) In describing an analytic study, Deming (1950:249) points out that "in analytic problems the concern is not this one bowl alone but the sequence of bowls that will be produced in the future". One bowl measurement is not enough, further information must be supplied by samples from other bowls. However, Deming (1992:132) criticized the extant practice whereby "teaching of pure statistical theory in universities, including the theory of probability and related subjects, is almost everywhere excellent. Application to enumerative studies is mostly correctly, but application to analytic problems -planning for improvement of tomorrow's run, next year's crop -is unfortunately, however, in many textbooks deceptive and misleading." Deming (1992:132) further added that "analysis of variance, t-test, confident intervals, and other statistical techniques taught in the books, however interesting, are inappropriate because they provide no basis for prediction and because they bury the information contained in the order of production." On the other hand, with regard the weakness of an enumerative study, Deming is of the view that "many surveys are flawed" (Latzko & Saunders, 1995:192). Respondents tend to be bias based on current perceived social values. Deming further pointed out that respondents also "tell us what they think we want to hear". Due to such flaws, he commented that the survey study is heavily perception biased and bring no significant value in improving the questions in the research study. From the Deming's thorough assessment of the situation, analytic study has been poorly taught in most universities.

DATA ANALYSIS
Based on the various sources of literature reviews in particular, Deming's work, a comparative study of the two research methods is presented in Table 2. It can be summarized that: a. The enumerative study does not identify any variable that is associated or considered the cause in the existing problem system. It only studies the magnitude of the problems with the "possible/likely" variables that can be correlated to the research problem. The most common statistical tools applied are t-test, Chi-Square, frequency, scatter diagram, correlation, etc. Subsequently, any findings or solutions still remain as the most "likely" workable plan and do not provide a 100% assurance of success.
b. An analytic study is used to determine the variable(s) that can be seen as the actual cause of the existing study problem. The Statistical Process Control (SPC) is the only analytic tool used in this context to determine the root cause variable. Using the SPC tool, one is able to differentiate between the common cause and special cause variables which exist in a system. Identifying the special cause variable is the main task. The common cause variable is the random variation that can only be improved unless the entire system process/performance was upgraded by the top management. The drawback of enumerative study is that it mixes both the common cause and special cause. Confusing both causes will result in the wrong corrective action which in turn might worsen the performance of the system. Therefore, an enumerative study is not an analytic method because it does not reveal which variable is confounder to the existing problem. A confused understanding of the characteristics of both the research frameworks will have serious implications. Using an enumerative study framework that only explains "what" to find out "why" will thus lead to the wrong solution proposed by researcher/consultant. The action taken for corrective action, recommendation is still based on judgment; often confuse and mixing up common and special cause variables The SPC's in statistical control performance will guide to determine the right corrective action and recommendation is effective implemented; clearly differentiate common and special cause Effectiveness of project carried out t-test or chi square does not guide to determine the project is implemented cost effectively SPC will help us to monitor and improve the variable(s) implemented, in an economically, timely and optimized manner for the whole balanced system

DISCUSSION AND IMPLICATION
The choice of a research framework, either enumerative study or analytic study, depends upon the type of research question one intends to answer. For instance, in order to know the magnitude of certain problems (or what is the existing situation), it is best to adopt the enumerative study that has been widely taught in most universities. However, if the goal is to identify which variable contributes/causes the current study problem, an analytic study is the best selection in this context.
• The Analytic approach is the most relevant paradigm in the current research context The effort to identify which variables contribute to daily problems is the biggest challenge. For example, finding out the reasons for the high rate product defect, identifying the best effective ways to overcome the current economic crisis, seeking out the type of virus responsible for the current disease outbreak, pinning down the reasons for students' poor academic performance, etc. easily appear as an analytic question. We often experience immense difficulty in identifying the real reason(s) which may assist us in resolving the problematic situation effectively. Such a challenge is indeed much more difficult compared to the task of the enumerative study in only studying the known and predetermined variables.
The enumerative study does not guide us to face the real world because it does not seek to identify the real root cause; too often the mindset of the whole paradigm still assumes a fantasy world. In contrast, the main objective of the analytic study is to identify the actual cause so that the research problem could be overcome effectively. It is indeed sad to note the unfortunate state of affairs whereby many graduated students across all disciplines are well trained in conducting the enumerative study, but the analytic study is totally absent in most of the universities today. Therefore, it is not surprising to observe that the enumerative study research framework is wrongly applied to solve what is essentially an analytic problem. A real example was described by a senior Professor in a local conference at the Putra World Trade Centre, Kuala Lumpur in 2009. With over twenty years of research experience, he admitted that his research findings and subsequent solutions proposed do not seem to achieve a measure of success to be proud of. It would seem to me, in light of the foregoing discussions that this senior academician has wrongly applied the enumerative research framework to solve an analytic problem. This is congruent with Deming's following remarks "twenty years of experience is meaningless, it may be just one year of experience repeated twenty times" (Latzko & Saunders 1995:77).
• Depending solely on the findings of an enumerative study is often insufficient in seeking any corrective action or improvement From the Deming's red bead experiment (Crayson;Deming 1994), we can identify various management concepts; namely, intensive inspection by workers, zero defect slogans, reward and punish scheme for workers, ranking worker performance, etc. These, in actual fact contribute no added value or a better improvement in product quality despite various corrective actions had been taken. The experiment showed the product defect rate actually appeared in the statistical control mode. Any effort that targeted solely the worker without top management involvement to improve the product quality will only result the greater losses. Besides, SPC offers the greatest advantage to differentiate the responsible party, either worker or top management, that should be accountability to product quality problem without blindly accused each other.
On the other hand, frequency, t-test and chi square in the enumerative study does not provide sufficient information. Much information is contained inside the data and do not reveal the entire story (Deming 1992(Deming , 1994. For instance, in an enumerative study, every data point (show increasing or decreasing rate) is treated as a special cause, and immediate remedial action is taken with the assumption that the system will become better. However, if the data is plotted using the SPC analytic tool, the data might appeared in the statistical control situation. It means that no adjustment is required on the system performance despite the existence of some variation/defects. Therefore, relying only on the enumerative study is insufficient and it could lead to wrong decision making.
• The enumerative study findings do not always provide the right answer, and is of too narrow a perspective The analogy here is like that of a blind researcher who is interested to know what the shape of the earth is. A thousand years ago people might not realize the actual shape of the earth. Using qualitative or quantitative perception studies, with the full participation of the population will finally form the theory that the earth is flat. The enumerative study serves to compile and aggregate the magnitude of general population perceptions into different defined categories. Such a study does not assure the correctness of the findings. As Deming has aptly pointed out, "every theory is correct in its own world, but the problem is that the theory may not make contact with this world" (W. Edwards Deming Institute 2003). For instance, the study of best ethical practice at Cyber Cafes among junior students could be defined differently for the Malay, Chinese and Indian community in Malaysia based on their different beliefs, culture and religious backgrounds. The selection of any ethnic community as the benchmark will not satisfy the other ethnic group. Deming (1992:29) has emphasized that the only way to overcome the different opinions of various groups is to bring them into the statistical control environment, pointing out that "inspectors fail to agree with each other until their work is brought into statistical control." On the other hand, adopting the annual income of advanced countries as an indicator of the high quality of life might only measure the passive monetary and physical property setting. It has failed to indicate the internal happiness among the low income community (developed countries) that might enjoy better spiritual and intrinsic happiness although their income revenue is relatively lower. This is the why research findings based on enumerative study has poor credibility due to its limited scope and it does not provide a holistic viewpoint. In short, rigidly applying the international gold standard instruments and too narrow a perspective as in an enumerative study tends to lead to the conclusion that the earth is still flat.
• Limitation of standard and model derived from enumerative study The attributes in the standard model is based on a set of statistical significant variables to describe or measure a particular scenario. This drawback in the enumerative paradigm will always influence its real life application. For instance, an audit based on the standard elements in an Occupational Safety Health Management System (OSHMS), diagnosing patients based on gold standard upon the predetermined variables does not assist much in real problem solving. The conventional statistical method assumes that each respondent in a sample is homogeneous. Consequently, the researcher, consultant, practitioner in most disciplines will treat every entity, company or patient as tending to behave similarly and does not consider any entity to have its own unique requirement(s) that cannot be benchmarked in an apple to apple approach.
At this juncture it is perhaps appropriate to highlight the often overlooked fact that Chinese Medicine theory, an ancient discipline which has been in existence for few thousand years, has a excellent approach that provides an accurate and realistic standpoint. It treats every patient as behaving uniquely, and always subject to various factors which "dynamically change" via a five-organ interrelation, and takes into consideration environmental, living styles, belief, diet habits and personal behavior. Each patient should therefore, be diagnosed and treated differently. Moreover, Chinese medicine theory adopts a holistic approach and does not rigidly apply any defined gold standard. Under the theory of variation as described by Deming, there is never any is exactly alike, even the twins (Beauregard et al. 1992).
To elaborate this point, an example is shown in Figure 1 where the data shows a normal distribution, comprising the following categories of excellent, average and poor performance workers. We take the statistical "mean" or average group as our main focus to design the training program to enhance worker performance. It is common practice that the "average" performance worker is selected because the category has the highest frequency samples. Subsequently, a training program was designed to suit the "average" performance worker and this leads to the neglect of the specific requirements from the excellent and poor performance worker groups. Consequently, 30%-40% of the two sample groups felt that the training program was not challenging (excellent performance worker) or too demanding (poor performance worker) respectively.
• The most worrying factor is the unknown variable The enumerative study deals with the known variables. As a result, a new gold standard, formula, checklist, etc was applied by industries/ practitioners in solving their problems. This type of chronological thinking has resulted in the whole related industries comprising universities, government agencies, industries to only deal with the known variables. No further effort is expanded to identify the unknown variable(s), and this may result in none of the concerned parties actually solving the real problem. The short term stamping out of the fire is their main goal. Such conventional efforts only maintain the status quo without any real improvement. Neave (1990:53) criticized the use of military standard and tables, for acceptance "implies that there is an acceptable level of errors and faults, thus denying the need for improvement". The Japanese manufacturer has learnt from Deming that there is the need to keep on improving all processes in the system. Western management style is still dealing with the known variables. As a result, it comes as no surprise that the Japanese are able to sell their products at a price that is the equivalent of the American production cost (Wadycki & Sclove, 1991). As Dr. Deming quoted by Dr. Nelson is said to have remarked "the most important problems of management are unknown and unknowable" (Latzko & Saunders, 1995: 111).

CONCLUSION
In an enumerative study, one does not differentiate the common cause and special cause. Subsequently, any corrective or improvement action is inadequate and serves no value added purpose. In reality, most research questions require the use of an analytic research framework rather than an enumerative study. The total transformation of the higher education system through research using the analytic research paradigm seems critical and mandatory if we aim to be the center of excellence. We are now living in a very highly competitive market and this requires every individual, industry or government to keep researching the unknown variables. A success in reducing 1% variation means 1% cheaper sales product and therefore, better survival. Lastly. if Deming has underscored the point that "information is not knowledge", then certainly the enumerative study provides passive "information" that might only solve symptom while an analytic study generates new "knowledge" that will solve real problem.