7th Biennial ACSPRI Social Science Methodology Conference

Dual frame surveys using mobile and landline telephone numbers have been the predominant method of sampling for CATI interview studies in Australia for several years now, however neither frame is without its problems. Landline telephones can only be used to access 49% of the population, with coverage heavily skewed towards older age groups. Mobile telephones are not inherently linked to a geographic location meaning that obtaining geographically targeted samples is expensive, often prohibitively so. Commercial sample providers have recently improved their offering for “listed” mobile phone numbers with geographical information attached, however these lists also have coverage errors particularly for younger age groups. This presentation reports on work carried out for a large state-level survey to determine an optimum blend of sample sources including listed and RDD mobile phones alongside landline sample. The survey is state-wide, but includes a sample quotas for each of the 79 LGAs in the state, so requires a large degree of geographical targeting. Several simulations were carried out using a variety of different sample mixes to determine the most cost-effective solution.

We present results of our work on the impact of call cycle and refusal conversion on weighted estimates from a large state-level CATI health survey. We examined estimates from truncating the call routine at 1, 2, etc. calls, where each estimate was weighted using standard procedures for the survey in question. We find that estimates for most outcome variables quickly stabilise: making additional calls does not yield improved estimates. For mobile phones, the vast majority of the variables examined were stable from one call attempt. It did not matter if call attempts stopped at one, four or ten; the estimates would be unchanged with respect to age of the respondent, gender, having children, local government area, language spoken at home, having anxiety or depression, arthritis, asthma, cancer, diabetes, heart disease, hypertension, osteoporosis, a stroke or multiple diseases, smoker status, life satisfaction, feeling one’s life is worthwhile, multiculturalism or fruit and vegetable consumption. A limited number of variables stabilised at the second call attempt: lifetime risk from alcohol consumption, BMI category, dental health and generalised trust. Others stabilised after the third call attempt: employment, immediate risk from alcohol consumption, self-assessed general health status, sedentary behaviour and avoiding visiting a dentist. The only variables we checked that stabilised at the fourth call attempt were household income and psychological distress. In a similar analysis to that for mobile sample, the variables that stabilise for landline sample beyond four call attempts are as follows. Stabilising at five attempts were respondent age, short-term risk from alcohol consumption and sedentary behaviour. Stabilising at six attempts were lifetime risk from alcohol consumption, generalised trust and vegetable consumption. Stabilising beyond six call attempts were household income, having multiple diseases and employment status. Our work shows that extended call cycles no longer reduce non-response error and casts into question the desirability of extended call routines in Australian telephone surveys.