I. A Framework for Management Research
- A conceptual framework for good management research
- Challenges to conducting management research
- External review considerations
- Responding to reviewer critiques: a case example
- >>Using qualitative research for hypothesis testing
III. Resources
Management Research in VA
II. Methodological Challenges to Conducting Management Research
One important methodological option in conducting management research is the use of qualitative methods for data collection and analysis. Qualitative research, with its emphasis on understanding complex, interrelated and/or changing phenomena, is particularly relevant to the challenges of conducting management research. Qualitative methods combined with quantitative ones can provide particularly rich and robust inquiries. Either alone or in combination, qualitative research must be conducted with methodological rigor.
This section does not attempt to provide a primer on qualitative methods. The role, benefits and appropriate use of qualitative research have been discussed extensively in the literature. Several references to excellent articles can be found in the references section and links to references. .
Our more limited aims here are:
- To offer, for those who are not familiar with qualitative methods, a brief overview of how they are used and what value they offer, drawing heavily from articles by Shoshanna Sofaer. 6
- To propose the use of qualitative methods in hypothesis testing. Qualitative methods are often used inductively, for exploration, theory building and description. Less attention has been given to their use in deductive hypothesis testing. The white paper, prepared by Brian Mittman 7 discussed in this section explores those potential uses.
Specially this section addresses four questions:
What are the uses and value of qualitative research?
Qualitative research is characterized by an emphasis on describing, understanding, and explaining complex phenomena - on studying, for example, the relationships, patterns and configurations among factors; or the context in which activities occur. The focus is on understanding the full multi-dimensional, dynamic picture of the subject of study.
Its approaches contrast with quantitative methods that aim to divide phenomena into manageable, clearly defined pieces, or variables. Quantification is good for separating phenomena into distinct and workable elements of a well-defined conceptual framework. But when we focus research on what we already know how to quantify, (e.g., what can be reliably quantified), we may miss factors that are key to a real understanding of the phenomena being studied. The downside of quantification is that it does not always support (as well as qualitative methods) understanding of complex, dynamic, and multi-dimensional wholes.
Qualitative methods are useful, not only in providing rich descriptions of complex phenomena, but in constructing or developing theories or conceptual frameworks, and in generating hypotheses to explain those phenomena.
What are the methodological challenges in qualitative research techniques?
Key challenges to conducting rigorous qualitative research range from instrument development through data collection to data analysis. In addition, results need to be documented and reported using formal accepted methods.
For example, typical deficiencies are unfocused instrument development and lack of supporting theory. Rigor related to instrument protocol development requires attention to validity, intrusiveness (the Hawthorne effect) and triangulation. In addition, attention must be paid to distinguishing between collecting subjective and objective data, information on the formal vs. the informal organizational structures and processes and the differences between collecting facts vs. opinions vs. interpretations.
Planned, systematic, comprehensive data collection requires variable definitions and measures, document coding form protocols, administrative database specifications and survey instrument question libraries. In the data collection phase, problems can be minimized through pilot-testing and pretesting, validity/quality checks, triangulation and monitored flexibility. Sole reliance on subjective data, self-reports, etc. can reduce validity. Some tips to insure rigor in data collection management include training of all data collection staff and conducting immediate post-collection coding for time/memory sensitive data. Other methods to ensure the validity of data include tape recording interviews, performing real time data entry and editing, using paired interviewers, and implementing quality assurance fo each instrument. And, to avoid further problems, incomplete, missing or unusable data should be corrected immediately.
Pitfalls related to data analysis include using ad hoc, emergent, exploratory, informal analyses that may lead to inappropriate conclusions and unpublishable results. Rigorous analysis requires an a priori theoretical model and hypothesis, a formal framework guiding data collection and analysis and adherence to the formal framework and research best practices.
Finally, reporting requires results structured by hypotheses and an analysis plan. Reports need to include data syntheses and summaries with a focused analysis of the data. Conclusions must have a documented basis and systematic formal analysis methods, and validity must be documented.
What are some key qualitative research methods?
A wide range of tested qualitative research methods are available to address these challenges. The selection of method, or combination of methods, will be tailored to the questions being studied and the setting for research. Typical methods include:
What is the role of qualitative research in hypothesis testing?
The origins and development of qualitative research methods and their close association with inductive, interpretive and historical research have led many researchers to associate these methods exclusively with these forms of research and to fail to recognize their value in conventional deductive empirical research.
Some investigators, however, contend that hypothesis-testing, deductive research can benefit from the use of qualitative research methods - and that these methods can be used in a manner consistent with accepted standards of rigor and validity. In particular they believe that the acknowledged strength and unique contribution of qualitative methods in developing insights into actors' values, beliefs, understandings and interpretations of events and other phenomena, or in explaining historical occurrences, can enhance "conventional" forms of empirical research.
Brian Mittman, in a white paper prepared for the MDRC workshop on Management Research in VA, argues for the use of qualitative methods in hypothesis testing, and outlines the key components of the rigorous approach needed to use these methods successfully. His paper is motivated by two interests: first, convincing researchers not experienced in qualitative methods that they can enhance their empirical, deductive work, and, second, minimizing the misuse of qualitative methods in ways that threaten the validity of studies. Dr. Mittman's paper is linked here.