[비즈한국] With the development of generative artificial intelligence (AI), the financial sector is actively incorporating AI technology. Along with this, side effects such as 'new-technology financial fraud' have emerged. As advancements in technology make it increasingly difficult to distinguish between what is fake and what is real, damages from new-technology scams are expected to grow. However, under current laws, prevention and victim restitution remain difficult. As the 'Basic Act on Artificial Intelligence Development and Trust Foundation (AI Basic Act)' passed the National Assembly's Science, ICT, Broadcasting and Communications Committee on November 26, voices are calling for detailed legislation to prevent AI-driven financial crime damages.
As financial fraud exploiting generative AI technology increases, a venue was provided to discuss measures for consumer damage prevention and relief. On November 27, the Consumer Rights Forum, the Korea Financial Consumer Society, Future Consumer Action, and the office of Democratic Party lawmaker Kim Byung-kee held a forum titled 'Seeking Consumer Protection Measures Against Financial Fraud Using New Technology.'
AI-based financial fraud primarily falls into three categories. The first is 'deepfake' fraud, which lures investors using videos impersonating celebrities. It is a method of creating deepfake videos featuring the faces of trusted financial figures or celebrities to induce investment. Last March, 137 celebrities who had been impersonated came forward to urge for a resolution to the problem.
The second is 'AI washing' fraud. Scammers lure consumers by exaggerating or falsely marketing as if they use AI technology when they do not. For example, they may promote that they "generate high returns using an AI-based stock trading program," but in reality, no such program or investment method exists.
The third is 'AI chatbot' fraud. This is also utilized in 'romance scam' crimes, where scammers approach victims via messages, gain their trust, and then demand money.
Experts gathered at the forum agreed that current laws are insufficient to stop AI financial fraud or provide relief for victims. Moon Chan-hyun, an expert committee member at Future Consumer Action (a Certified Financial Planner) who led the presentation, said, "If you look at deepfake investment scam videos, the quality is not actually that high. They are crude enough that one could distinguish them as fake, yet damages continue to occur. This means there are many financially vulnerable people." He added, "Looking at the characteristics of financial fraud victims, contrary to the common belief that they are mostly elderly or have lower education levels, many incidents occur among people in their 40s and 50s who are economically active. The fact that the number one perpetrator is an acquaintance, followed by someone working in the financial sector, is also a reason why the damage is so extensive."
Committee member Moon advised, "As financial fraud types appear in complex patterns—combining quasi-deposits, phishing, smishing, and scams—the amount of damage per person is also increasing. To protect financial consumers, we must reinforce systems such as strengthening punishments related to financial fraud, put effort into promotional activities that educate on various types and prevention methods, and provide financial education to improve consumer competence."
It was pointed out that even if consumers suffer damages using AI services provided by financial institutions, compensation is almost impossible. Jung Shin-dong, a professor at Hankuk University of Foreign Studies Law School, noted, "In the financial sector, a massive amount of data is accumulated, much of which is sensitive. However, when damages occur due to the negligence of financial institutions, consumers cannot prove the issue themselves. This is due to the characteristics of AI: autonomy and opacity."
While AI makes autonomous judgments, it is difficult to hold the AI itself liable for negligence when an accident occurs. It is also ambiguous to hold the AI developer responsible, as they are not the ones who committed the negligence. Regarding opacity, companies often refuse to disclose AI algorithms citing 'trade secrets,' and even developers struggle to prove the causality of negligence since they often do not know what results the AI will produce.
Professor Jung said, "Traditional legal order cannot prevent consumer damage. There is a clear legal vacuum. Making it mandatory to label AI-generated content so that consumers can recognize it is the foundation for increasing transparency. Institutional support is also needed so that AI technology can be used to prevent crimes in advance."
After the presentation, the designated discussion highlighted the need for legislation to recover fraud damages. Sung Jun-ho, a senior researcher at the Sungkyunkwan University Institute of Legal Studies, stated, "Even if the perpetrator is punished, the damage cannot be restored. Moreover, in cases of romance scams or new types of investment fraud, there is no practical relief method because the victims deposited the money themselves, even though they were deceived."
Lee Hyo-seob, a senior researcher at the Korea Capital Market Institute, pointed out, "When financial disputes occur on generative AI-based platforms, consumers often do not even know they are victims, and it takes a long time to get relief. An independent financial dispute mediation organization is necessary. Since recovering damages in fraud crimes is virtually impossible, a separate victim protection act should be enacted."
The problem of current laws placing the burden of proof on AI financial fraud victims was also mentioned. Baek Joo-sun, an attorney at Law Firm Daeyul, explained, "Unlike environmental crimes, medical malpractice, or intellectual property disputes, where related laws and precedents ease the victim's burden of proof by presuming the perpetrator's negligence or causality, large-scale financial fraud cases do not allow for this. We need detailed institutional supplements, referencing laws like the EU AI Liability Directive, to prevent arbitrary judicial operations."
The AI Basic Act, which passed the Science, ICT, Broadcasting and Communications Committee on November 26, was also discussed. Jung Joon-hwa, a legislative researcher at the National Assembly Research Service's Social and Cultural Investigation Office (Science, Broadcasting, and Communications Team), said, "19 AI-related bills were proposed, and the name was finally unified to 'Trust Foundation.' At the core of AI fraud is the fact that it is difficult for consumers to distinguish whether the information is virtual, created by AI, or real. The bill also emphasizes the labeling of AI-generated content. However, the entity responsible for the obligation to label is ambiguous and needs to be specified. Additionally, we need not only an AI Safety Institute but also an organization that acts as a protection center specializing in investigating consumer damages."
Researcher Jung emphasized that strengthening the digital literacy (the ability to understand and acquire information) of financial consumers should be viewed as a minimum measure. He emphasized, "The emphasis on digital literacy is actually a dangerous narrative if looked at closely. Not only consumers with low literacy but also those with high literacy can be scammed, because they strongly trust their own judgment. If we place too much importance on digital literacy, it leads to the problem of shifting the responsibility of the government or public institutions onto the consumer."