How to read a behavioral research paper

1/09/04

Prepared by Prof. Lee Sproull and Prof. Natalia Levina

For the “IS Research: Behavioral Perspectives” doctoral seminar

New York University, Stern School of Business.

In this seminar you must read many papers. “Reading” is an activity that requires critical skills (skills of criticism and analysis) that you will develop over the course of the semester. You should plan to read each of the required papers as least twice. In the first reading you should ask yourself: What is the author saying? After having read the paper once, you should be able to produce a one-paragraph summary of the main message of the paper. In the second reading you should ask more sophisticated questions (examples of types of questions below) that draw upon more sophisticated cognitive skills. At the end of the second reading, you should be prepared to participate in a fruitful discussion of the paper and to write a thoughtful critique of it.

Levels and types of questions

Exploratory questions probe facts and basic knowledge: “What evidence supports the hypothesis?”

Challenge questions examine assumptions, conclusions, and interpretations: “How else might we account for the findings of this experiment? Would the conclusions hold if a different method was used?”

Relational questions ask for comparisons of themes, ideas, or issues: “What premises of Sproull/Kiesler (86) did Walther (96) challenge? How does the concept of interface differ among Authors A, B, and C? How does this work relate to the themes from prior classes?”

Diagnostic questions probe motives or causes: “Why did author A use method B in this study? What are authors’ assumptions about human nature, technology, and knowledge?”

Cause-and-effect questions ask for causal relationships between ideas, actions, or events: “If the technology artifact were changed in the following ways, how would the relationships proposed by author A change?”

Extension questions: “How does this paper relate to previous papers?” “What are the practical implications of this work?”

Priority questions: “What is the most important or most fundamental theoretical issue emerging from this paper? This set of papers?

Cognitive skills required by different types of questions

Knowledge skills (remembering): How does author A define outsourcing? How is IT productivity measured in paper B?

Comprehension skills (understanding the meaning of remembered material, usually demonstrated by restating or citing examples): Explain the process by which IT investment increases productivity according to paper B. Give examples of boundary processes in IT development.

Application skills (using information in a new context to solve a problem or answer a question): how could the method of paper A be used to investigate the problem posed in paper B?

Analysis skills (breaking a concept into its parts and explaining their interrelationships): What are the components of mutual knowledge [according to author A? according to you?] How do they interrelate [according to author A? according to you?] How do they contribute to group collaboration [according to author A? according to you?]

Synthesis skills (putting the parts together to form a new whole; solving a problem requiring creativity or originality): “How would you design a structuration study of the “productivity paradox?”

Evaluation skills (using a set of criteria to arrive at a reasoned judgment of the value of the paper): “To what extent does the paper advance understanding of behavioral IS research?”

(modified from Barbara Gross Davis, 2001, Tools for Teaching, Jossey-Bass, pp. 83-85)