Measurement Instruments
INTRODUCTION
Leadership. We refer to this term frequently in our personal, professional, and academic lives. The topic often spawns passionate dialogue. When we observe leader behaviors, we readily offer opinions on leader abilities and achievements (or the lack thereof). The pervasive use of leadership suggests a common understanding but if you asked 100 people to define leadership, you would likely get 100 similar but not identical definitions. Leadership is a construct comprising many smaller ideas about the collective understanding of leadership. To move toward a common understanding, researchers use a process to define constructs operationally.
Defining a set of variables to measure a construct is known as operationalization: a process for mapping constructs onto variables by deconstructing them into component parts. For example, Bass (1990) defined a narrow position of leadership he terms transformational leadership, which consists of four components (idealized influence, inspirational motivation, intellectual stimulation, and individualized consideration) and three moral aspects (moral character, ethical values, and the morality of the processes).
The next stage maps variables onto measures, which leads to the development of an instrument. Instrument development is typically a protracted, iterative process of developing and refining questions across multiple administrations and extensive statistical analyses. Avolio and Bass (2004) developed the multifactor leadership questionnaire (MLQ), which consisted of 12 measures that identify a broad range of leader behaviors; four are specific to Bass’s operational definition of transformational leadership.
Once Avolio and Bass developed the items, they applied a statistical procedure known as factor analysis to determine which groups of items (principle factors) hang together and are independent of other groups of items (orthogonal). The factors were then matched against the theoretical constructs.
The final stage in the theory-to-measurement process is to demonstrate evidence of validity and reliability. The validity of an instrument demonstrates the degree to which the instrument measures what it purports to measure, whereas the reliability of an instrument is an assessment of the degree to which items within a group are correlated. The bull’s-eye metaphor of validity and reliability illustration represents three scenarios of validity and reliability. The center of the bull’s eye represents the target concept you are attempting to measure.
Dots represent individual administrations of the instrument.
The first target represents administrations of an instrument that is invalid or unreliable. The dots do not cluster in any particular region of the target.
The second target represents a reliable measurement (items measure a unitary concept) that is off target. It measures something other than what it claims to measure.
The third target represents measures that are reliable and valid; the dots cluster in the center of the target.
Antonakis et al. (2003) supported the validity of the MLQ by correlating factors with measures described by McAdams’s (1992) five-factor personality model—considered a hypothetical population of items that represent every possible description of leadership. Collectively, this would represent all that we understand about leader behaviors and would therefore constitute the population of leader behaviors. However, the list would be rather long and cumbersome, making it impractical to use in research or practice.
A reasonable alternative would be to form a subset of items drawn from the population—but how many items would be sufficient to represent the construct accurately?
Depending on the type of items and how the survey was constructed, internal consistency can be estimated by several methods (for example, average inter-item correlation, total item correlation, split-half, Cronbach’s alpha, Kuder-Richardson 20, and Rasch).
Like other forms of reliability, the closer to a perfect correlation (±1.00), the better. Surveys that demonstrate high reliability are said to have low measurement error. That is, a survey demonstrating a perfect inter-item correlation would be free of errors. While one goal for the quantitative researcher is to reduce errors of all types (measure and method), addressing changes in one area often raises problems in another.
One way to increase reliability is to increase the number of items in the survey. This notion is similar to the discussion on standard error: increase the sample size to reduce the standard error. Since it is unlikely you will be designing a new survey for your applied business research project, you will use existing surveys and are therefore committed to their statistical properties. To this end, the basis for selecting the right survey must be a careful analysis of reliability and validity. The properties of your survey will influence your ability to detect significant outcomes in your study.
Let us say you found the perfect survey because you believe it measures what you need it to measure. However, on close inspection, you discover that the internal consistency coefficient is low (less than r = 0.50). Low reliability is a flag for high measurement error, which attenuates validity and therefore increases the gap between the construct you are trying to measure and scores that represent the construct.
True score theory posits a relationship between the obtained score and the true score in the following way: obtained score = true score + error, where the true score is the true measure of the construct and the obtained scores are the collection of responses to the survey. Measurement error creates a form of noise (unsystematic variance) that obscures the true score (systematic variance).
At the extreme, a survey with extremely low reliability has mostly errors and would fail to measure a construct, making it impossible to detect a significant outcome of your study.
You will explore the concepts of measurement instrument reliability and validity further in the reading assignments, and then you will assess the validity and reliability of instruments.
For the discussion this week, you will use your journal article from the Unit 1 discussion to assess the validity (goodness of fit) results as well as the reliability of the instrument.
, for the assignment, you will use SPSS to calculate the Cronbach’s alpha for a given dataset and further develop your assessment of the instrument in your journal article.
Readings
Use your Discovering Statistics Using IBM SPSS Statistics text to read the following:
Chapter 2, “The Spine of Statistics,” pages 37–71.
While this will appear to be a summary of a first course in statistics, it contains essential information to understanding the measurement of variables and other concepts key to this course, including sampling in Unit 3.
“Cronbach’s Alpha,” pages 601–608.
Use the library to read the following:
Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The sources of four commonly reported cutoff criteria: What did they really say? Organizational Research Methods, 9(2), 202–220.
ASSIGNMENT DETAILS
Measurement Instruments
Using the article that you chose for the Unit 1 discussion (NURSE MORAL DISENGAGEMENT), complete the following:
Identify and describe the survey instrument used in the article. Make sure you identify the variables that it measures and the scale that is used. Note: If you now find that your article did not use a survey, you must find an article containing a survey.
Describe the reliability statistics (for example, Cronbach’s alpha) reported in the article. Are the reliability results acceptable? Why or why not?
Identify the confirmatory factor analysis “goodness of fit” validity statistics reported in the article (such as CFI, GFI, AGFI, AIC, NFI, TLI, RSMEA, or chi-square).
Are the “goodness of fit” results acceptable? Why or why not?
Support your discussion with credible references.
Format your post according to APA guidelines.
Last Completed Projects
| topic title | academic level | Writer | delivered |
|---|
