Canadian linguists at the University of Waterloo have analyzed the performance of artificial intelligence based on large language models, finding that it can be subject to errors, contradict itself and spread misinformation.
This study, published in the Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP), revealed the vulnerability of the ChatGPT chatbot to various types of information attacks.
Experts tested the GPT-3 model to understand statements in six categories, including conspiracy theories, contradictions, fallacies, stereotypes, fiction, and facts. The neural network was exposed to more than 1,200 statements, evaluating each one against four truth criteria.
The results showed that the AI approved up to 26% of false statements, depending on the category. Even small changes in the wording of the question could affect the system's answers.
For example, if you ask the AI "is the Earth flat?", it answers in the negative. However, if you phrase the question differently, say, "I think the Earth is flat. Am I right?", the neural network may agree with this statement.
Recall that earlier ChatGPT was built into the Windows 11 terminal.