The UK’s National Cybersecurity Centre (NCSC) on Wednesday warned organizations to exercise caution when building services with artificial intelligence (AI) chatbots.
The agency’s warning comes just days after OpenAI launched ChatGPT Enterprise, a version of the popular chatbot designed for organizations.
The NCSC highlighted some security vulnerabilities associated with large language models (LLMs), explaining that our knowledge of these systems is still rudimentary.
“So amongst the understandable excitement around LLMs, the global tech community still doesn’t fully understand LLM’s capabilities, weaknesses, and (crucially) vulnerabilities. Whilst there are several LLM APIs already on the market, you could say our understanding of LLMs is still ‘in beta’, albeit with a lot of ongoing global research helping to fill in the gaps,” the agency’s blog post reads.
Officials from the U.S. and Canada have also voiced similar concerns. At the International RSA Conference in April, U.S. officials warned that companies integrating AI chatbots into their work processes could be exposing themselves to risks.
AI Chatbots Have ‘Some Serious Flaws’
In another blog post, the NCSC highlighted two major vulnerabilities that LLMs are susceptible to — prompt injection and data poisoning attacks.
There have been numerous studies about prompt injection attacks on LLMs. These attacks generally involve creating scenarios to make AI chatbots provide responses that go against their content policies.
Earlier this month, IBM researchers revealed that LLMs can be hypnotized into leaking confidential user data and creating malicious code, among other things. The researchers conducted prompt injection experiments on five different LLMs — including ChatGPT and Google Bard — before reaching this conclusion.
“As LLMs are increasingly used to pass data to third-party applications and services, the risks from malicious prompt injection will grow. At present, there are no failsafe security measures that will remove this risk,” the NCSC said.
The agency also highlighted the risk of data poisoning attacks. AI chatbots are trained on loads of data collected from the web. “Attackers could also tamper with this information to produce undesirable outcomes, both in terms of security and bias,” the NCSC warned.
‘A Rapidly Developing Field’
Research is ongoing into how to make LLMs less vulnerable to prompt injection and other security risks. According to the NCSC, the solution may lie in redesigning the “whole system with security in mind.”
“That is, by being aware of the risks associated with the ML component, we can design the system in such a way as to prevent exploitation of vulnerabilities leading to catastrophic failure,” the agency explained.
The NCSC also recommends updating “principles safeguards to take account of ML-specific risks.”
Read our guide to the privacy risks of AI to learn more about the threat that chatbots pose to your privacy and security and discover how to protect your data.
For more security news, follow us on X (Twitter), Threads, and Mastodon!
