We invite you to read the opinion piece by our Director of New Technologies and AI, Juan Pablo González, on the challenges posed by the adoption of artificial intelligence and the importance of properly managing its risks.
Today, the use of Artificial Intelligence (AI) in an organization’s day-to-day activities has become widespread in operations that were previously performed manually. However, the incorporation of this type of technology into institutions does not necessarily go hand in hand with a vision for the governance of AI systems.
Therefore, one of the major challenges, without a doubt, is for this advancement to move from an AI system that merely responded to user requests and perhaps generated content, to a system capable of performing actions that have effects on the physical world.
Thus, an AI agent is an AI system that can achieve a specific goal with minimal supervision. Consequently, they range from “learning models that mimic human decision-making to solve problems in real time[1].” Thus, by leveraging large language models (LLMs) alongside other tools, it enables the performance of a specific task—that is, the agent carries out multi-step tasks, such as sending emails, making reservations, or processing payments. It goes beyond merely generating content; rather, it enables the agent to perform actions requested by the user, using information authorized by the user.
Internationally, there has been discussion regarding the legal nature of an AI agent[2], and some have already stated that they are not legal entities; therefore, even though certain tasks are delegated, responsibility remains entirely with the party delegating the tasks. This undoubtedly implies that the user must assume responsibility for the actions performed by this agent, as well as analyze the potential risks associated with these practices.
Some of the risks that can be identified include, for example: a) access to information through excessive privileges; b) vulnerabilities in AI systems; c) the ability to aggregate and recombine information from various sources (“function creep”); and d) the risk associated with multi-agent systems. Regarding the latter, it must be considered that the improper training of one agent can affect another agent, or, more directly, that the assigned subtasks may involve coordinating agents whose behavior is biased.
Consequently, some international authorities have recommended guidelines for the responsible use of AI agents: a) minimizing access to data; b) transparency—that is, disclosing that an agent is being used in certain processes; c) accuracy through constant review of the results obtained with the support of this technology; d) establishing rules to limit data retention; e) setting limits on the purpose for which the user authorized the agent’s use; f) security measures addressing both the IT system and the data it processes; g) a “privacy by design and by default” approach; and h) ongoing risk management, which includes clear governance that extends beyond the organization to relationships with suppliers, with a focus on training[3].
Another key point is traceability, whether through the implementation of logs—that is, the ability to record the operations performed by the agent, for example, using timestamps—which not only provides clarity on AI uses but also ensures the auditability of the systems and the databases accessed to perform those actions.
Why is this an important issue to consider? If one examines the European regulatory landscape regarding AI, it has been characterized by the uses and risks of AI systems, but has not focused on the ability of AI systems with varying levels of autonomy to act on data and systems—especially when a user authorizes them to do so. This scenario not only has implications for the legal profession but also entails a paradigm shift in the operations and processes within the organization.
Therefore, it is important to move from a paradigm centered on a technical approach to one that allows us to understand that tasks such as taking inventory of the organization’s AI systems—as well as clarifying the other applications to which they connect, the databases they access, and, consequently, the early adoption of measures such as human oversight—will be key to governing the use of this technology.
To think that the use of AI will slow down is contrary to how things are currently progressing; the correct approach will be to focus on how proper management can maximize the use of these types of technologies in certain processes, with a clear and determined approach to risk management.
Column written by:
Juan Pablo González | New Technologies and AI Director | jgonzalez@az.cl
