BEGIN:VCALENDAR VERSION:2.0 PRODID:-//pretalx//conferences.acspri.org.au//2026//speaker//HXRLY3 BEGIN:VTIMEZONE TZID:AEST BEGIN:STANDARD DTSTART:20000326T040000 RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=3;UNTIL=20050326T170000Z TZNAME:AEST TZOFFSETFROM:+1100 TZOFFSETTO:+1000 END:STANDARD BEGIN:STANDARD DTSTART:20060402T040000 RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4;UNTIL=20060401T170000Z TZNAME:AEST TZOFFSETFROM:+1100 TZOFFSETTO:+1000 END:STANDARD BEGIN:STANDARD DTSTART:20070325T040000 RRULE:FREQ=YEARLY;BYDAY=4SU;BYMONTH=3;UNTIL=20070324T170000Z TZNAME:AEST TZOFFSETFROM:+1100 TZOFFSETTO:+1000 END:STANDARD BEGIN:STANDARD DTSTART:20080406T040000 RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=4 TZNAME:AEST TZOFFSETFROM:+1100 TZOFFSETTO:+1000 END:STANDARD BEGIN:DAYLIGHT DTSTART:20000827T030000 RRULE:FREQ=YEARLY;BYDAY=4SU;BYMONTH=8;UNTIL=20000826T170000Z TZNAME:AEDT TZOFFSETFROM:+1000 TZOFFSETTO:+1100 END:DAYLIGHT BEGIN:DAYLIGHT DTSTART:20011028T030000 RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10;UNTIL=20071027T170000Z TZNAME:AEDT TZOFFSETFROM:+1000 TZOFFSETTO:+1100 END:DAYLIGHT BEGIN:DAYLIGHT DTSTART:20081005T030000 RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=10 TZNAME:AEDT TZOFFSETFROM:+1000 TZOFFSETTO:+1100 END:DAYLIGHT END:VTIMEZONE BEGIN:VEVENT UID:pretalx-2026-UKCQSB@conferences.acspri.org.au DTSTART;TZID=AEST:20261126T111500 DTEND;TZID=AEST:20261126T113000 DESCRIPTION:Large language models (LLMs)\, such as ChatGPT\, are increasing ly being considered as tools for coding in evidence synthesis. However\, e valuations of LLM coding often rely mainly on overall agreement scores\, w hich can conceal how and why model decisions differ from researcher judgem ent. This is especially problematic in conceptual reviews\, where definiti ons\, operationalisations and relationships between concepts may be implic it\, spread across a text or open to more than one reasonable interpretati on. This paper asks where the two coding approaches diverge\, whether disa greements are systematically associated with particular coding functions a nd concepts\, and whether they lead to different substantive interpretatio ns.\nThe study is embedded in a large-scale scoping review of 6\,205 peer- reviewed articles on refugee integration published between 2001 and 2025. For the comparison\, 15 articles were purposively selected from 200 full t exts coded with the support of NVivo to maximise variation in period\, met hod\, discipline\, geography and conceptual engagement. ChatGPT applied th e same codebook to the NVivo-identified passages\, using each extract with out the wider article context.\nThe paired dataset comprised 296 coding un its covering eight concepts: integration\, settlement\, belonging\, citize nship\, adaptation\, acculturation\, social capital and social inclusion. Three coding functions were assessed: whether a passage defined a concept\ ; identified how it was studied or measured\; or explicitly related it to another concept. The analysis combined exact agreement\, Cohen’s kappa\, the direction of disagreement\, concept-level cross-tabulations and compa rison of the resulting qualitative syntheses.\nExact agreement was similar across functions\, ranging from 53.4% to 54.4%\, while agreement beyond c hance was slight (κ = .064–.132). Yet these headline coefficients conce aled strongly directional differences. NVivo-assisted researcher coding wa s broader for definitions (56.1% of passages compared with 19.6% for ChatG PT) and operationalisation (49.7% compared with 25.3%). ChatGPT applied a broader threshold to relationships between concepts (57.4% compared with 3 2.8%). Divergence also varied across concepts: ChatGPT identified no opera tionalisations for settlement or citizenship\, whereas belonging and socia l capital showed especially strong substantive convergence.\nDespite low a greement on individual passages\, the two approaches produced similar high er-level interpretations. Both described integration as multidimensional\, processual\, relational and institutionally conditioned\, and produced cl osely aligned accounts of belonging\, social capital and the distinction b etween formal citizenship and lived membership. They often reached similar conceptual conclusions through different allocations of material across c oding functions.\nThe paper argues that evaluating LLM coding requires mor e than a single agreement coefficient. It proposes a multilevel assessment combining code prevalence\, direction of disagreement\, concept-specific patterns and substantive convergence. Because the study used a researcher- derived passage set\, a single model and prompt format\, and no independen tly adjudicated standard\, it evaluates consistency and coding thresholds rather than comparative accuracy. The findings show that LLM coding shoul d not be treated as interchangeable with researcher judgement\, but may of fer a complementary perspective when its role\, access to context and limi tations are explicitly documented. DTSTAMP:20261013T070101Z LOCATION:Sutherland Room SUMMARY:Where NVivo-assisted researcher coding and LLM coding diverge: Evid ence from a large-scale scoping review - Gabriella D'Avino URL:https://conferences.acspri.org.au/2026/talk/UKCQSB/ END:VEVENT BEGIN:VEVENT UID:pretalx-2026-BEH7LV@conferences.acspri.org.au DTSTART;TZID=AEST:20261126T120000 DTEND;TZID=AEST:20261126T121500 DESCRIPTION:Egocentric network data are often elicited rather than directly observed. Name generators and alter-level questions require participants to define boundaries\, retrieve alters\, retain an expanding list and\, wh ere local structure is collected\, assess a rapidly increasing number of p ossible alter–alter ties. These demands intensify when interviews are mu ltilingual\, interpreter-mediated\, ethically sensitive\, conducted in dom estic settings and embedded in longer qualitative encounters. Participant- aided sociograms may reduce some of this burden\, but they are often descr ibed mainly as engagement tools. Less attention has been paid to how they transform the production\, observability and interpretability of network e vidence.\n\nThis paper develops and evaluates a qualitative-led mixed-meth ods protocol centred on a low-technology\, participant-aided target sociog ram. It asks: how do participant-aided sociograms transform egocentric net work elicitation under high-constraint field conditions\, and what are the implications for data quality\, boundary specification and interpretation ? The five-stage protocol combines: (1) an ethically permissive contextual opening\; (2) affective and hypothetical exchange name generators\; (3) q uestion-wise composition measures\; (4) participant placement and movement of alters across four perceived-closeness rings\; and (5) participant map ping of salient alter–alter ties and clusters\, followed by qualitative probing.\n\nThe protocol was piloted and applied in 17 interview sessions with 23 Arabic-speaking refugees in England\, including 10 sessions mediat ed by a trained Arabic–English telephone interpreter. The methodological evaluation integrates transcripts and field notes\, photographed sociogra ms\, documented additions and repositioning of alters\, case-by-case audit s of maps against node and edge files\, and descriptive summaries of netwo rk size and map complexity.\n\nThe findings show that the sociogram redist ributes cognitive and interactional work across participant\, visual artef act\, interviewer and interpreter through four linked mechanisms. First\, the visible map supports memory offloading by keeping previously elicited alters available for inspection and iterative recall. Second\, it enables comparative calibration: participants judge and revise the closeness of on e relationship in relation to others. Third\, spatial grouping makes perce ived local structure feasible to elicit without a complete dyadic census. Fourth\, interaction around the map makes parts of the reporting process v isible. Hesitation\, clarification\, movement and decisions to omit a tie can be qualitatively probed\, helping distinguish forgetting\, boundary mi sunderstandings\, taken-for-granted household relations\, ethical withhold ing and uncertainty.\n\nThese gains do not produce a complete or objective ly accurate network. The maps may privilege strong\, salient and coherent ties\; question-wise composition reduces repetition but sacrifices alter-l evel cross-classification\; very large networks crowd the paper target\; a nd a telephone interpreter cannot share the visual field. The protocol the refore supports claims about elicited network size\, broad role compositio n\, ordinal perceived closeness and salient perceived structure\, but not total degree\, valued ties or a complete adjacency matrix.\n\nThe paper co ntributes a replicable five-stage protocol\, explicit conversion and repor ting rules\, and an implementation-based framework for evaluating particip ant-aided sociograms as measurement interventions as well as visual elicit ation tools. It argues that\, in high-constraint fieldwork\, data quality depends not on maximising standardisation or detail in isolation\, but on the documented fit among the research question\, elicitation instrument an d field encounter. DTSTAMP:20261013T070101Z LOCATION:MacCallum Room SUMMARY:Participant-aided sociograms in high-constraint fieldwork: A qualit ative-led mixed-methods protocol for egocentric network elicitation - Gabr iella D'Avino URL:https://conferences.acspri.org.au/2026/talk/BEH7LV/ END:VEVENT END:VCALENDAR