Exploring the Impact of User-Participated Customization in Experiencing Chatbot Failure
Extended Abstracts of the CHI Conference on Human Factors in Computing Systems 2024
ChatbotCustomizationFailureUser Experience
Abstract
Traditionally, personalized AI systems have offered customized experiences by inferring user preferences from their system usage data, which can be referred to as data-driven personalization. With the advent of GPTs, opportunities for users to directly engage in the customization process have increased, which we term user-participated customization. In this study, we explored the impact of user-participated customization on user reactions to chatbot failures. We involved twenty-two participants in total, with fourteen for user-participated customization and eight for data-driven personalization. Both groups experienced frustration with chatbot failures, yet their responses differed. Those who customized their chatbots predominantly displayed retrying intentions, in contrast to those using pre-personalized chatbots, who primarily showed giving-up intentions. Moreover, we noted different customizations were preferred for task-oriented and social-oriented chatbots. This study suggests that user-participated customization has the potential to foster the Ikea effect. This effect is expected to mitigate negative experiences with chatbots.