8th Biennial ACSPRI Social Science Methodology Conference

Life in Australia™ is Australia’s only probability-based online panel, in operation since 2017. The panel was initially recruited in 2016 using dual-frame random digit dialling (RDD), topped up in 2018 using cell phone RDD as a single frame, expanded in 2019 using address-based sampling (A-BS), and topped up in late 2020 using a combination of A-BS, interactive voice response (IVR) calls to cell phones, and SMS push-to-web (i.e., invitations using only SMS) and expanded again in late 2021 using A-BS.
We present our findings with respect to recruitment and profile rates, retention, and completion rates. We also present the demographic profile of panel members compared to Census benchmarks with respect to age, gender, education, and nativity. We also discuss projections of amortised recruitment cost allowing for more frequent top-ups and higher attrition of SMS push-to-web sample to shed further light on cost considerations.
The yields from IVR and SMS push-to-web sample were below that of ABS, however the costs for IVR and SMS push-to-web were well below those of ABS and the less expensive modes actually delivered a more desirable panel member profile with respect to age and nativity, though not education. Our research raises interesting questions as to the trade-off between bias, cost, and face validity in the form of response rates.
This paper contributes to the international body of research on recruitment methods for probability-based online panels (see, e.g., Bertoni 2019; Bilgen, Dennis, and Ganesh 2018; Blom, Gathmann, and Krieger 2015; Bosnjak et al. 2018; Jessop 2018; Knoef and de Vos 2009; Meekins, Fries, and Fink 2019; Pedlow and Zhao 2016; Pew Research Center 2015, 2019; Pollard and Baird 2017; Scherpenzeel and Toepoel 2012; Stern 2015; Vaithianathan 2017; Ventura et al. 2017).

Recording link: https://acspri-org-au.zoom.us/rec/share/sFS4E0Eva6L3CETeEbh9cr8bLW7jHf3BqadVrD3ZBgTXkIpMYjtO_h9UWipxxHi-.ZbxNTlh6Oe1hAEZ5?startTime=1669255431000

The Integrated Public Number Database (IPND) provides access to all Australian mobile telephone numbers. Unlike random digit dialling (RDD), the IPND also provides the postcode at which the number is registered. The availability of postcodes is critical for affordable research at the state level and below because Australian mobile numbering is not related to geography. As a result of the cost of screening for sub-national samples, contemporary Australian telephone surveys typically use a blend of mobile RDD and listed mobile sample for cost control. (Landline RDD has largely been discontinued due to declining coverage, increasing coverage error and increasing cost.) Although the IPND has been available for some time for a limited set of use-cases (public health research, Commonwealth public policy research and party political research for candidates or parties), there has been no systematic investigation into the quality and efficiency of the IPND relative to RDD; an absence that this paper aims to remedy. We describe the results of (1) a systematic trial conducted on the IPND by the Social Research Centre, in partnership with various state and territory health agencies and Cancer Council Victoria (CCV) and (2) parallel administration of RDD and IPND in the CCV’s 2022 Victorian Smoking and Health Survey.
We first present a brief overview of the IPND and its limitations. The accuracy of geographic information from the IPND trial is shown, at levels from state (correct in 93% of cases) to postcode (correct in 75% of cases) and it is demonstrated that no appreciable coverage error was identified. The results of RDD and IPND samples on demographics and selected health outcomes and behavioural measures are compared. With the possible exception of the proportion of very young adults (i.e. 18-19 year-olds) responding, both unweighted achieved sample and weighted estimates are consistent between RDD and IPND.
The results of this paper show that the IPND is fit-for-purpose for public health research that has to date used RDD-based designs. Compared to a hypothetical single frame mobile RDD design for a state-level survey, IPND will yield similar estimates and sample characteristics but at reduced cost, due to reduced need for screening. Compared to the dual-frame mobile RDD and listed designs that are actually in use, costs should be broadly similar but the IPND is preferable because it offers the benefits of a pure probability design and higher weighting efficiency compared to the dual-frame alternative.

Recording link: https://acspri-org-au.zoom.us/rec/share/sFS4E0Eva6L3CETeEbh9cr8bLW7jHf3BqadVrD3ZBgTXkIpMYjtO_h9UWipxxHi-.ZbxNTlh6Oe1hAEZ5?startTime=1669255431000