Bias Alert: LlamaChat AI Chatbot Favors Pro-European Views, Study Shows

Bias Alert: LlamaChat AI Chatbot Favors Pro-European Views, Study Shows

(IN BRIEF) A recent study conducted by researchers from the University of Copenhagen sheds light on the political bias embedded within LlamaChat, a popular AI chatbot developed by Meta, particularly in its responses to European Union (EU) political issues. According to the study, LlamaChat demonstrates a predisposition towards pro-European and left-wing viewpoints, as evidenced by its alignment with groups such as the Greens/EFA and the S&D group in the European Parliament. The researchers tested the chatbot’s political inclinations using a questionnaire related to EU politics, revealing its tendency to lean towards certain ideologies. The study attributes this bias to various factors, including the datasets used to train the model and Meta’s ethical guidelines, which may favor non-controversial positions. However, the researchers also found that it is possible to modify the model’s biases through additional training and by circumventing its built-in ethical constraints. They hope that their findings will prompt further collaboration to develop more balanced and informative language models that can better serve citizens in political decision-making processes.

(PRESS RELEASE) COPENHAGEN, 6-Jun-2024 — /EuropaWire/ — In the midst of the European Parliament elections, where citizens are grappling with decisions on party representation, a recent study sheds light on the political bias embedded in AI chatbots. LlamaChat, a prominent alternative to ChatGPT, has been found to exhibit a clear left-leaning and pro-European stance, according to research conducted at the University of Copenhagen.

Led by postdocs Ilias Chalkidis and Stephanie Brandl from the Department of Computer Science, the study delved into LlamaChat’s responses to EU policy issues. Testing the model on a questionnaire related to European politics, researchers discovered a notable inclination towards pro-European and left-wing viewpoints. Notably, LlamaChat’s alignment was observed to be closer to the Greens/EFA and S&D groups, rather than the EPP or ID group.

The investigation highlighted two primary factors contributing to LlamaChat’s political bias. Firstly, the model’s training data, sourced from internet datasets, may inherently contain biases. Secondly, the influence of Meta’s ethical guidelines during model optimization could steer it towards non-controversial, left-leaning positions.

Utilizing an open-source version of LlamaChat based on Meta’s Llama2 model, the study employed the “euandi2019” questionnaire, designed to assist EU citizens in identifying preferred candidates during the 2019 elections. Additionally, the researchers fine-tuned the model using 87,000 speeches from European Parliament plenary sessions spanning 2009-2023.

Stephanie Brandl expressed concern over the lack of external influence on the development of such language models, emphasizing the need for public agencies to fund initiatives aimed at better controlling training datasets and guidelines.

While previous studies have unearthed political biases in AI chatbots like ChatGPT, this research marks the first exploration of such biases within an EU context. Moreover, the study demonstrates the potential for mitigating biases through additional training and adjustments to ethical guidelines.

Chalkidis and Brandl hope their findings will pave the way for the development of more neutral and informative language models, capable of providing nuanced insights into political matters. They envision a future where AI chatbots can empower citizens with accurate information, guiding them through complex political landscapes and fostering greater democratic engagement.

Media Contacts:

Ilias Chalkidis
Postdoc
Department of Computer Science
University of Copenhagen
Ilias.Chalkidis@di.ku.dk

Stephanie Brandl
Postdoc
Department of Computer Science
University of Copenhagen
brandl@di.ku.dk
+45 35 32 88 58

Maria Hornbek
Journalist
Faculty of Science
University of Copenhagen
maho@science.ku.dk
+45 22 95 42 83

SOURCE: University of Copenhagen