Can we augment web responses with telephonic responses to a graduate destination survey?. Issue 4 (18th May 2016)

• Record Type: Journal Article
• Authors: du Toit, Jacques
• Is Part Of: Assessment & evaluation in higher education. Volume 41:Issue 4(2016)
• Page Start: 560
• Page End: 574
• Publication Date: 2016-05-18
• Subjects: alumni survey -- graduate destination survey -- web survey -- telephonic survey -- mode effect; Education, Higher -- Evaluation -- Periodicals; 378
• DOI: 10.1080/02602938.2015.1033613
• Languages: English
• ISSNs: 0260-2938

Abstract

Abstract : The 2010 Western Cape graduate destination survey utilised a sequential mixed-mode design in which an initial web survey was augmented with an equivalent telephonic survey. This article examines mode effect in the Western Cape survey in terms of overall effect size and the bearing it had on the main outcome of the study. Standardised residuals and Cramér's V are used to determine mode effect across two scenarios, a full sample vs. a subsample, and using two categorical questions with different numbers of response categories. Overall effect size appears to be small in the first question, but increases noticeably together with non-responses in the second question that has many more response categories. Web responses to alumni or graduate destination surveys can perhaps be augmented with telephonic responses if necessary, provided response categories are kept to a minimum, and interviewers are trained properly and monitored for possible interviewer misbehaviour. The benefit of obtaining larger samples should then also outweigh the benefit of using web surveys alone.