Saint Louis University John Cook School of Business Researcher Helps Publication Address Replications in Business Research
New Publication Presents Forum for Examination of Previously Published Findings
ST. LOUIS— A recent technical note in the journal Science continued a debate about the reliability of scientific findings. In essence an open science consortium reported reconfirmation of only about half of the papers published in top psychological journals, however using a different approach another group of scientists found closer to 85% of studies to yield essentially the same as initial results.
Fred Niederman, Ph.D., the Shaughnessy chair of management at Saint Louis University’s John Cook School of Business, has been invited to join the board of senior editors for a new journal, Transactions on Research Replication sponsored by the Association for Information Systems (AIS), the leading academic society for the field of management information systems. The journal is dedicated to testing the confirmation of results of prior published studies in the management information systems domain.
Niederman has, with colleague Dr. Salvatore March from Vanderbilt University, already published one paper in the new journal examining the wide variety of types of replications that are possible and the effects of different kinds of variation in approach relative to the initial published study. A second paper is currently under review that describes the discussion and argumentation pertaining to replications from a panel on which he participated at the 2015 European Conference on Information Systems in Muenster, Germany.
“The scientific method requires conceiving of patterns likely to be observed in practice in the form of theory,” Niederman explained. “These theories are used to make predictions about what we will see on the next trial. We conduct or observe what happens and contrast the actual results to what were predicted. It is critical, particularly for phenomena that are important and consequential, that we do not stop there but continue testing to see how much confidence we should have that the next trial will produce the same results as those previously seen.”
The follow up of prior studies can show similar or identical results, adding support for prior conclusions; alternatively it can show completely opposite or partially supportive results.
“What’s most important,” Dr. Niederman continued, “Is not the dismissal of prior theory because results are not completely supported, but rather the opportunity to structure new questions and reformulate original theory into richer configurations.”
For example, key differences in the setting of two studies may show that a pattern is likely to repeat in one condition but not the other. Differences in findings might also reflect tendencies in different methodologies. Where surveys are likely to capture overall judgments of individuals responding to particular questions, interviews might capture difficult to compare, but richer and more nuanced explanations of attitude toward particular questions.
In order to create a stronger overall body of knowledge where observed patterns stated in the form of theories can grow in stature and usefulness, the AIS group has added this journal focused on replication to its family of publications that include a top theory building and testing journal, the Journal of AIS and a community building journal, the Communications of AIS.
“This journal may be the first of its kind specifically dedicated to replications,” said Niederman. “It is an honor to have the opportunity to contribute to an innovative research approach that has the potential to be replicated itself for use in other academic fields.”
Fred Niederman, PhD. is the Shaughnessy Endowed Professor of management in the John Cook School of Business at Saint Louis University. He has devoted many years to the study of personnel who work in information systems and the management of information systems and projects more generally. Distinguishing him as the first coordinator of the AIS senior scholars, a group of 150 research and service leaders in the information systems academic field, Niederman recently collaborated in the publication of a history of co-evolution of information systems and the personnel who create and maintain them.View All News