MIT researchers studied 16 million election-related AI responses. They found chatbots are 'sensitive to steering,' raising questions about LLMs' neutrality

It’s July 2024. Vice President Kamala Harris just kicked off a blitz run for the White House after a shock switch-up.

Meanwhile, a team of MIT researchers was working to better understand how chatbots perceive this political environment. They fed a dozen leading LLMs 12,000 election-related questions on a nearly daily basis, collecting more than 16 million total responses through the contest in November. Now they are publishing some conclusions from that process.

As the first big US political race to occur since generative AI went mainstream, the 2024 presidential campaigns happened in a media environment in which the average voter was increasingly looking to chatbots for election information.

The authors wanted to study the impact that shift had on the information voters saw, i