Demand Characteristics of Respondents and the Reliability of a Survey Instrument
Type of Submission
Podium Presentation
Campus Venue
Dixon Ministry Center, Room 102
Location
Cedarville, OH
Start Date
4-10-2013 1:20 PM
End Date
4-10-2013 1:45 PM
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.
Apr 10th, 1:20 PM
Apr 10th, 1:45 PM
Demand Characteristics of Respondents and the Reliability of a Survey Instrument
Cedarville, OH
Comments
Abstract:
The Bible Scale developed by Village (2005) to assess religious beliefs was administered to a sample of 226 CU students and 237 alumni through Survey Monkey in April, 2012. We used a principal components factor analysis with promax rotation as the primary technique to determine the dimensions of the 33 Items on Village's Instrument based on this sample. We then selected the three factors with Internal consistency reliabllty, measured by Cronbach's alpha, greater than .70 for subsequent analyses. We named the first Conservative lifestyle because the items dealt with divorce, cohabitation, premarital sex, abortion, and homosexuality. We labeled the second factor Religious Certainty because the statements eschewed doubt, liberalism, and other faiths in contrast to Christianity. We called the third scale Biblical Authority because It affirmed the Bible as the sole source for teaching and belief. The three factors together accounted for approximately 56 percent of the total variance. The Village scale was again administered to a new sample of 682 CU students in November, 2012. The same statistical analyses was applied to the data. It was expected that with a 1arger sample size, both validity and reliability would Improve. However, the factor structure failed to hold and the internal consistency reliabilities of the scales dropped significantly. Closer examination of the data showed a higher percentage of missing responses in the November sample than in the April sample. Feedbacks from some of the students suggested that they were reluctant to answer questions regarding their beliefs of the Bible during the time of data collection. The psychometric discrepancy can be attributed primarily to the differences in attitude while taking the survey. This is an example of how demand characteristics in the respondents affect the validity and reliability of survey results.