Comparative Efficacy of AI LLMs in Clinical Social Work: ChatGPT-4, Gemini, Copilot

TAŞKIRAN TEPE H., ASLANTÜRK H.

Research on Social Work Practice, 2025 (SSCI) identifier

  • Yayın Türü: Makale / Tam Makale
  • Basım Tarihi: 2025
  • Doi Numarası: 10.1177/10497315241313071
  • Dergi Adı: Research on Social Work Practice
  • Derginin Tarandığı İndeksler: Social Sciences Citation Index (SSCI), Scopus, Academic Search Premier, ASSIA, Child Development & Adolescent Studies, CINAHL, EBSCO Education Source, Education Abstracts, ERIC (Education Resources Information Center), Psycinfo, Public Affairs Index, Social services abstracts, Sociological abstracts, Violence & Abuse Abstracts
  • Anahtar Kelimeler: artificial intelligence, ChatGPT, clinical social work, Gemini, Microsoft Copilot
  • Bilecik Şeyh Edebali Üniversitesi Adresli: Evet

Özet

Purpose: This study examines the comparative efficacy of three AI large language models (LLMs)—ChatGPT-4, Gemini, and Microsoft Copilot—in clinical social work. Method: By presenting scenarios of varying complexities, the study assessed their performance using the Ateşman Readability Index and a Likert-type accuracy scale. Results: Results showed that Gemini had the highest accuracy, while Microsoft Copilot excelled in readability. Significant differences were found in accuracy scores (p =.003), although readability differences were not statistically significant (p =.054). No correlation was found between case complexity and either accuracy or readability. Discussion: Despite the differences, none of the models fully met all accuracy standards, indicating areas for further improvement. The findings suggest that while LLMs offer promise in social work, they require refinement to better meet the field's needs.