At present, “asking AI for everything” seems to be becoming the norm.
From daily life to public affairs, when asking about AI decisions, should we ask: Should humans maintain the necessary understanding, verification and judgment when making decisions? When the explanation of the problem, alternatives and reasons for choice are all provided by AI, what kind of judgment does the human final decision include?
who is forming the judgment
A working paper published by Harvard Business School in 2025 shows that in the process of 244 Boston Consulting Group consultants completing simulated investment advice tasks, some mastered their own analysis ideas and selectively used AI; some repeatedly discussed and tested with AI; others handed over the entire task to the system and directly adopted the results or only made superficial modifications. Reports are all submitted by people, but the process of forming judgments differs.
A decision doesn’t start with the final choice. Before that, people need to figure out what happened, why, and what the basis for each plan is. If these interpretations and comparisons are mainly done by AI, then it has already undertaken part of the work of forming judgments.
In this sense, AI transforms scattered materials into explanations and understandings that can be used for decision-making, thereby undertaking part of the knowledge production work and participating in shaping people’s understanding of things. Therefore, although the final confirmation is still completed by humans, it is not enough to conclude that the process of forming judgments is still dominated by humans.
When these perceptions become the basis for public decisions, those affected are included. It is often difficult for the public to verify all professional materials on their own; if the AI analysis process is difficult to understand, they may rely more on professional explanations.
At this point, reliance on AI output may also translate into reliance on those who interpret, verify and confirm these outputs. The question of who is producing knowledge is further linked to whose explanations can be trusted.
When AI becomes “cognitive authority”
The interpretation and adoption of AI by professionals may also enhance the authority of experts and AI.
In 2025, at the AI pediatrician pilot signing ceremony at Beijing Beier Doudian Children’s Hospital, doctors demonstrated human-machine collaborative diagnosis and treatment: AI proposed preliminary diagnosis, examination, and treatment suggestions for doctors’ reference. According to relevant introductions, the system integrates the clinical experience of experts from Beijing Children’s Hospital and years of medical record data.
In such a collaborative relationship, expert experience and hospital credibility can become reasons for people to trust the system. In turn, when professionals cite AI analysis to illustrate their recommendations, these recommendations may be considered to be supported by both professional experience and data calculations.
Mutual endorsement may be just the starting point for the relationship. As technology develops, if AI performs reliably in continued use and is recognized by professional institutions and users, its status as “knowledge authority” may be consolidated. Trust in its ability to provide reliable knowledge may lead to a shift away from the requirement that every output be confirmed by experts.
If this trust further turns into a presumption of the correctness of AI, when experts disagree with AI, people may first ask experts to explain why their judgment deviates from AI, instead of examining whether AI makes mistakes or has hallucinations. At this time, AI has become the criterion for measuring expert opinions. It has also moved from “knowledge authority” that provides credible knowledge to “cognitive authority” that evaluates other knowledge.
The consequence of this change is that “technocratic rule” or “expert rule” in post-industrial society has gradually evolved into “algorithmic rule” in the era of digital intelligence. As AI agents participate in people’s daily decision-making throughout the entire process, at least at this stage, the biggest concern about “algorithmic rule” is not that super intelligence begins to rule humans after AI wakes up, but that humans unconsciously entrust the cognitive authority and decision-making power that originally belonged to humans to AI.
AI is influencing agenda setting
If people’s trust in AI’s judgment ability further extends to acceptance of the way it defines problems, AI’s impact may enter agenda setting: what constitutes a problem and which goals should be prioritized will also begin to be understood according to the framework provided by AI. Converting public issues into computational tasks requires setting goals and evaluation criteria, and these settings themselves contain value trade-offs. Once agenda setting is treated as a technical prerequisite that does not need to be discussed due to the authority of AI, public discussion may begin after the goals have been determined and focus on comparing which solution is more effective.
For example, if improving urban travel is simply expressed as shortening motor vehicle travel time, the calculation can compare which solution is faster, but it cannot decide whether pedestrians and public transportation should be given priority based on this indicator alone. Appeals that are not included in the evaluation framework may also lose the opportunity to be seriously considered because they are difficult to reflect as improvements in established indicators.
This change does not necessarily eliminate personal choice. Within a given framework, AI can guide behavior through default options, solution sequencing and presentation methods. An individual can still choose another plan, but this does not mean that he has the opportunity to change the goals that the plan follows.
Maintaining the opportunity to choose and participating in forming a range of choices are two issues at different levels. Everyone wants to choose the optimal solution, but no one wants to be the option ignored or excluded by the “optimization goal”.
The era of “cognitive outsourcing” means that the “principal-agent” problem in political science has been further expanded into a widespread reliance on problems and goals defined by AI. Although “algorithmic rule” has not yet truly arrived, overreliance on AI may gradually make it a reality.
