Relevance & Research Question

Segmentation is a challenging, core strategic research task, enabling businesses to understand the many needs of diverse customer groups. Yet, its effectiveness lies in its adoption within an organization. Despite AI's pervasive influence, its potential benefit in segmentation studies remains underutilized. This prompts us to explore: To what extent can AI help us socialize segmentation research, enabling stakeholder interaction with data and driving organizational adoption to influence business outcomes?
Methods & Data

Our research introduces an innovative approach leveraging AI-driven persona chatbots, tapping into Language Model-based systems like ChatGPT. Our aim is to create an interactive chatbot that can be shared across organizational departments, facilitating in-depth familiarization with customer segments. To do that, we train a GPT model on a comprehensive dataset combining quantitative and qualitative research findings from a study on segmenting online travel booking site users. This will enable us to evaluate its potential as a tool to humanize research findings and share accurate information about identified segments.
Results

Our research plans to assess the chatbot's capacity to uphold factual accuracy based on its training data while also exploring its ability to generate creative yet aligned responses consistent with the characteristics of the segmented customer groups it represents. Initial assessments showcase promising signs of the chatbot's capacity to navigate between factual accuracy and creative engagement, aligning well with the segmented customer profiles it represents. However, its effectiveness heavily depends on well-engineered prompt design.

Added Value

In embracing this innovative approach, our goal is to create a tool that aids organizations in unlocking the full potential of segmentation. By encouraging greater immersion and fostering deeper empathy with consumer segments, our persona chatbot aims to make research findings more accessible to wider, less research-savvy audiences and enables them to get to know segments in a more playful and engaging way.

Relevance & Research Question

Conversational agents such as chatGPT open up new ways for online qualitative research. On the analysis side, they may be used to extract key ideas and to provide participants’ quotes illustrating those. On the field management side, they may be used for moderation, to help dig deeper into what participants think. However, beyond the obvious advantages in terms of feasibility, numbers, speed and costs, the question how AI supplemented research design fares compared to purely human driven research appears as a pressing and hard to address question. Our goal in this research is more precisely to assess the quality of AI supplemented qualitative research for analysis and moderation, by comparison with human standards.

Methods & Data

We shall compare results obtained with and without a chatGPT powered AI assistant on a recently launched qualitative research platform www.bilendi.de/static/bilendi-discuss enabling the use of such an assistant. Regarding analysis, we qualitatively compare the results of a pure human analysis with those of the ChatGPT powered analysis. Regarding moderation, we quantitatively compare the response rate of participants to human moderators vs the chatGPT powered moderator. The data will consist in two data sets, a first study run in Finland in September 2023 gathering 22 participants, and a second study run in France and Germany and the UK over November-December 2023 gathering 30 participants per country. A pilot was run for this second demo in France in November 2023 with 225 participants.
Results

In the Finnish study (analysis only), ideas provided by the ChatGPT powered assistant were found to be 70% consistent with those of the human analysis, 20% consistent but ‘not usable as such’ and 10% inconsistent. In the pilot study for the second demo (moderation only), response rate to human moderators was 85% and response rate to the ChatGPT powered assistant was 74.63%.

Added Value

Recent research by Chopra and Haaland (‘Conducting Qualitative Interviews with AI’, Cesifo Working Paper, 2023) provides encouraging evidence in terms of participants engagement and generated insights. The present research develops on those results by coming with systematic comparisons between human and machine performance.

Thanks to GPS tracking with a smartphone app, a person's mobility behavior can be tracked in great detail. The information obtained on stages, routes, transport use and mobility purposes offers real added value in many areas of research. In this presentation, Beat Fischer explains the methodology, provides insights into the data science behind it and shows case studies with data from the Swiss Footprints Panel.