09/09/2026
🤖📋 Can AI become a survey methodologist?
A fascinating topic from our GESIS Summer School Lunchtime Talks this year, where Lydia Repke explored how AI can support questionnaire evaluation and help us design better questions – without replacing survey methodologists.
👉 Read more about The Algorithm Behind SQP 3.0 (sqp.gesis.org) in the original post. 👇
𝗛𝗼𝘄 𝗖𝗮𝗻 𝗪𝗲 𝗣𝗿𝗲𝗱𝗶𝗰𝘁 𝗦𝘂𝗿𝘃𝗲𝘆 𝗤𝘂𝗲𝘀𝘁𝗶𝗼𝗻 𝗤𝘂𝗮𝗹𝗶𝘁𝘆? 𝗧𝗵𝗲 𝗔𝗹𝗴𝗼𝗿𝗶𝘁𝗵𝗺 𝗕𝗲𝗵𝗶𝗻𝗱 𝗦𝗤𝗣 𝟯.𝟬
https://sqp.gesis.org/
𝗖𝗮𝗻 𝘄𝗲 𝗽𝗿𝗲𝗱𝗶𝗰𝘁 𝗵𝗼𝘄 𝘄𝗲𝗹𝗹 𝗮 𝘀𝘂𝗿𝘃𝗲𝘆 𝗾𝘂𝗲𝘀𝘁𝗶𝗼𝗻 𝘄𝗶𝗹𝗹 𝗺𝗲𝗮𝘀𝘂𝗿𝗲 𝘄𝗵𝗮𝘁 𝗶𝘁 𝗶𝘀 𝘀𝘂𝗽𝗽𝗼𝘀𝗲𝗱 𝘁𝗼 𝗺𝗲𝗮𝘀𝘂𝗿𝗲 𝙗𝙚𝙛𝙤𝙧𝙚 𝘄𝗲 𝗲𝘃𝗲𝗻 𝗰𝗼𝗹𝗹𝗲𝗰𝘁 𝘁𝗵𝗲 𝗱𝗮𝘁𝗮?
The idea behind SQP is to use the formal and linguistic characteristics of a survey question—such as its response scale, wording, or the number of response categories—to predict its reliability and validity.
With the release of SQP 3.0, the database had grown substantially. That meant it was time to rethink the statistical model behind the predictions. Barbara Felderer, Lydia Repke, Wiebke Weber, Jonas Schweisthal, and Ludwig Bothmann, compared four machine-learning approaches:
🔹 LASSO
🔹 Elastic Net
🔹 Boosting
🔹 Random Forest
And the winner? Random Forest.
It is now the prediction algorithm implemented in SQP 3.0.
But prediction accuracy is only part of the story. The authors also wanted to understand 𝙬𝙝𝙞𝙘𝙝 𝗰𝗵𝗮𝗿𝗮𝗰𝘁𝗲𝗿𝗶𝘀𝘁𝗶𝗰𝘀 𝗼𝗳 𝘀𝘂𝗿𝘃𝗲𝘆 𝗾𝘂𝗲𝘀𝘁𝗶𝗼𝗻𝘀 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗱𝗿𝗶𝘃𝗲 𝘁𝗵𝗲 𝗽𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻𝘀. Looking at feature importance in the Random Forest model gives valuable insights into the relationship between question design and measurement quality.
📄Felderer, B., Repke, L., Weber, W., Schweisthal, J., & Bothmann, L. (2026). Predicting the validity and reliability of survey questions. 𝘚𝘶𝘳𝘷𝘦𝘺 𝘙𝘦𝘴𝘦𝘢𝘳𝘤𝘩 𝘔𝘦𝘵𝘩𝘰𝘥𝘴, 20(2), 133-145.
👉https://doi.org/10.18148/srm/2026.v20i2.8453
