Surveys: Getting better return rates through incentives


One of the great things about working at a university library is you can get to along to lunchtime lectures on Wednesdays to hear academics talking about their research. The Dept of Communication, Journalism and Marketing here at the Turitea campus of Massey University has been running an excellent series of talks this year. This week Dr Mike Brennan spoke about “Doing research on the cheap”.

Mike spoke about using surveys to conduct experimental studies on how to improve return rates (this was for mail surveys). One of the key ways to improve return rates is to use an incentive, but what works best? As part of this experiment surveys had 20 cents, 50 cents, or $1 attached to them, or a chance to go into the draw for $200 or a $200 voucher, and there was a control with no incentive.

Return rates are improved by providing the incentive with the survey, rather than the promise of a prize draw or voucher. Turns out the 50 cent incentive got the best return from the first mail out in this experiment:

 

Mail 1

Mail 2

Mail 3

Control

24.7

46.6

57.5

20c mailout 1

27.1

43.5

54.1

50c mailout 1

46.0

66.7

74.7

$1 mailout 1

42.3

59.2

69.0

20c mailout 2

28.9

51.8

63.9

50c mailout 2

15.7

39.8

54.2

$1 mailout 2

23.5

51.9

69.1

$200 prize draw

25.6

43.6

57.7

$200 voucher

18.3

46.3

61.0

(NB We didn’t get a date that this research was done. Many thanks to my colleague Jane Brooker for noting the figures for this table.)

Obviously a small cash incentive is not true compensation for someone’s time, so wording such as “please accept this as a token of our appreciation” in the covering letter works well.

These days NZ Post doesn’t allow cash to be sent through the mail. Intrepid researchers have tried alternatives to cash in postal surveys. These include:

  • Pens
  • Tea bags, coffee bags, or both
  • Scratch and win cards
  • Stamps
  • Golf balls (!)
  • Turkeys (presumably vouchers for them!)

                                        choc2                                                                                                                                                                                                         Gold foil wrapped chocolate coins have also been tried, but a better option is the chocolate squares from Whittakers. Judging by the murmurs of approval from the audience this is likely to be a great incentive!

 

 

The other important thing is not to use just one mail out, but to send a reminder or another copy of the questionnaire in subsequent mailouts. As the table above shows this will increase response rates. Various combinations have been trialled and the 3 stage combo of questionnaire with chocolate/replacement questionnaire/follow up letter was mentioned as being successful.

Other external treatments have also been researched – these include using stamped v franked envelopes, brown v white envelopes, tone of the cover letter, status of the researcher (professor v student), colour of the questionnaire. Mike’s profile page details the research he and colleagues have published in this area.

PS –  I see there was a session at the recent LIANZA conference on designing effective surveys by Rachel Esson from Victoria University of Wellington, so that’s one conference paper I’ll be looking out for.

Advertisements

3 thoughts on “Surveys: Getting better return rates through incentives

  1. vintagekat says:

    I found Rachel Esson’s session at conference really interesting. Some of it was common-sense stuff (eg. better rates of return with less than 30 questions. One only needs to listen to our students grumbling about the approx 60 or so questions they need to answer for one of our student surveys to know that!). Rachel found that anything over 30 qustions and response rates will drop dramatically.

    Some of her other findings I found quite surprising. For example paper surveys reported a consistently higher response rate than electronic surveys. Although I suspect this would be dependent on the demographic being surveyed (I would expect students to have a higher electronic response rate in general than paper; but perhaps paper is best for Gen Y and up given the findings?).

    You mentioned briefly in your post about the colour of the survey paper. This was another aspect I found interesting and hadn’t considered. Rachel didn’t say what colour works best, but said it might be a useful exercise to experiment with different colours when sending a survey and compare response rates of the different colours. Colour can also apply to electronic surveys in regard to background and font colours.

    Her findings were also consistet with your mention about including incentives at the time of the suvey – rather than the promise of going into a prize draw or incentive on return. She did say though that the incentive has to be balanced in its worth. Findings suggest that an incentive that is too high is often percieved as bribery and motivation to participate in the survey can be lost.

    Finally, she found from her research that survey response rates are generally higher if participants can also see some kind of wider ‘community good’ gained as a result of the survey – not just a narrow focus about the library (although I’m not 100% sure how you could crack that one every survey as some naturally do have to be of a narrow focus at times).

    Nice to see that much of Rachel’s resarch is consistent with other research. All interesting stuff.

    • vintagekat says:

      Oops. I meant to say Gen X in reference to paper surveys. Gen Y would probably roll their eyes at anything paper based!

  2. Alison says:

    Thanks for that summary Kathy – it sounds like it was a really useful session that backs up what I have learnt from my studies and also heard from the likes of Mike. 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s