At present, big models are transforming human society at a rapid pace. With their powerful ability to generate text, images and videos, they bring people the efficiency revolution and paradigm innovation in learning and work. More and more people apply big models to daily practice activities. However, in this process, some users are easily deceived by the illusion of artificial intelligence that is strictly compatible with artificial intelligence, and unconditionally trust all the answers output by the big model, which in turn has adverse effects on life, study or work, causing risks such as misleading decisions and cognitive bias. In the process of promoting the widespread application of large models, we must be vigilant against and manage the illusion of large models and their value risks, and accelerate the construction of a prevention and control system that integrates technical optimization, legal regulation and ethical adjustment.

Causes of the generation of large-scale hallucinations

  In the field of artificial intelligence, illusion refers to the phenomenon that the content generated by the big model seems to be syntax correct and logically rigorous, but in fact there are factual errors or the facts cannot be verified, and it has the characteristics of "serious nonsense" and the inability to confirm the basis for reality. The hallucinations in the big model include two types: factual hallucinations and faithful hallucinations. The former refers to inconsistent with facts or fabricated facts, while the latter refers to inconsistent with user instructions, inconsistent with context or logic.

  In essence, the hallucination problem of the big model is the product of the combined action of its technical architecture, training and generation mode, and has characteristics such as generality, contingency, randomness, difficulty in avoidance or overcoming. In terms of generation mechanism, the core causes of large-scale model illusion come from probability-driven technical architecture, training data limitations, and multiple couplings of human-computer interaction generation logic.

  First of all, there are shortcomings in the technical architecture capabilities. At present, large models mainly adopt the GPT paradigm based on converter structure. This technical architecture can indeed greatly improve the accuracy and efficiency of natural language processing, but there may be capabilities shortcomings in pre-training, supervision and fine-tuning, inference and other links, resulting in illusion problems that are inconsistent with facts, instructions or context. For example, in the pre-training stage, the large model predicts the output word by word based on the probability distribution of historical markers through autoregression generation. This mechanism naturally lacks the ability to control the semantic consistency of the context, and it is easy to give priority to vocabulary combinations with higher probability but inconsistent with facts and logic, and output forms the illusion of "correct grammar but distorted content".

  The second is the natural defects in the training data. The big model performs deep learning based on massive Internet data, but because the Internet corpus has not been strictly tested or processed, or due to incorrect annotation, there are inevitably factual errors or logical contradictions. The big model lacks the ability to identify the authenticity of the data and is easy to capture or generate answers based on the wrong data. For example, when the GPT4 big model is asked to tell the story of "Lin Daiyu pulling up the weeping willows", the big model cannot distinguish the traps, but instead directly spliced ​​the content from a massive text data without factual verification to create a ridiculous plot.

  Finally, there is the stereotyped injury of human-computer interaction. The reinforcement learning of human feedback adopted by the big model can easily lead to problems such as making noises and forging facts in the process of deliberately catering to human needs. For example, after Air Canada chatbot misunderstood the concept of "special refund", it continuously generates fictional refund conditions and time limits, which eventually leads to legal disputes. The unique technical architecture and generation logic of the big model lead to the danger of self-reinforcement of hallucinations.

The value risk of big model hallucinations

  The random appearance and inevitable appearance of large-scale illusions also pose value risks such as weakening human-machine trust, guiding information polarization, impacting social order and even causing ideological security crises. Prevention and governance are urgently needed.

  The most direct harm of big model hallucinations is the misleading of user decisions, especially in the fields of medical care, health, finance, etc. The authoritative expression style and smooth narrative logic of the big model make the misinformation extremely confusing. If users rely too much on big models to generate information to assist decisions, they are likely to be misled and have serious consequences. For example, believing that the wrong treatment options provided by the big model can lead to uncontrollable or even further worsening of the disease. If this continues, it may weaken the human-machine trust relationship.

  What is even more worrying is that the value risks caused by large-scale model hallucinations show a diffusion path from misleading individual decision-making to group cognitive bias and impact on social order. In the field of public decision-making, hallucinations may distort policy cognition. If the identification and control of the information output by large models are not strengthened, there is a high possibility that hallucinations such as misreading policies and making discriminatory remarks may occur, which will not only weaken the credibility of the government, but may even endanger social public security.

  In the field of ideological security, related threats are more concealed. For example, relevant research has monitored that some overseas large models look at the achievements and institutional advantages of socialism with Chinese characteristics with tinted glasses, deliberately mixing false facts or miscalculation to form outputs that are different from mainstream discourse. This illusionary content, which has been packaged ideologically, is value-penetrating through knowledge Q&A, is far more misleading than traditional false information.

Countermeasures for the governance of large-scale illusions

  To prevent and control large-scale illusions, we should build a three-dimensional governance system for technological correction, legal regulation and ethical adjustment, eliminate illusions through technological optimization, clarify the boundaries of responsibility with the help of legal regulations, cultivate value rationality based on ethical adjustment, and make the big model a more reliable partner for human beings.

  Build a multi-level prevention and control system. "Using skills to govern" is the preferred path to solve the problem of hallucinations of big models. The "value-sensitive design" or value alignment strategy of artificial intelligence ethics also depends on technological innovation and breakthroughs. This not only requires artificial intelligence companies and experts to improve the performance of big models by improving the quality of training data, strengthening external verification and fact checking, improving model reasoning capabilities, enhancing transparency and interpretability, but also encourages philosophy and social science experts to work together with artificial intelligence experts to help big models improve the accuracy of question-and-answer issues and value risks through knowledge base optimization, training corpus error correction, value alignment monitoring and other ways.

  Establish an adaptive governance framework. Faced with the popularization of large models, agile, flexible and standardized legislative governance is imperative. The State Cyberspace Administration of China and seven other departments have promulgated and implemented the "Interim Measures for Generative Artificial Intelligence Service Management", which put forward clear legal regulations and risk prevention requirements for training data selection, model generation and optimization, and service provision, which is conducive to promoting "intelligence to goodness" and promoting the compliance application of large models. The EU's Artificial Intelligence Act requires large models to fulfill their notification obligations and ensure the robustness and reliability of technical solutions, and form an effective institutional constraints and accountability framework for the application of large models, which is worthy of reference.

  Improve the value benchmark for technology development and application. Governance innovation that is more meaningful for the illusion of big models lies in establishing technical values, integrating ethical values ​​such as responsible innovation and controllable creativity into the engineer's mind and implanting the code of big models. For example, it is advocated that the principles of not generating controversial conclusions, not generating information that cannot be traced, and not generating content that exceeds the cognitive boundaries of the model, so as to promote the transformation of large models from pursuing generation fluency to ensuring content reliability; for example, establish a hierarchical confidence prompt system for answering factual questions in large models, and classify and annotate them according to high credibility, verification, speculative conclusions, etc., to strengthen the transparency and interpretability of the output content.

  For users, they should further improve their information literacy for scientifically correct application of large models, and then become commanders who guide content generation and identify hallucinations. Research shows that through training in usage habits such as artificial intelligence cross-verification, it can significantly reduce the probability that users are misled by hallucinations. People need to keep pace with the times to improve their comprehensive ability to analyze hallucinations, master common sense, and critical thinking, and comply with the implementation of value principles such as fact verification, logical verification, professional screening, minimum necessity, and scene control in the process of retrieving information and generating content, so as to maximize the elimination of hallucinations problems and value risks.

  The reliability construction of artificial intelligence often lags behind its capacity expansion. The ultimate goal of governing the illusion of big models is not to completely eliminate technological uncertainty, but to establish a human-machine collaboration mechanism with controllable risks. In this human-machine synergy cyber evolution, always maintaining the humility of technology and ethical clarity is the right thing to break the fog of illusion of big models.

  (Author: Li Ling, associate researcher at the Institute of Marxism, Fudan University)

[Editor in charge: Zhu Jiaqi]