AI, Deepfakes and Cyber Security - What Organisations Should Know - Cyber and Fraud Centre

AI - Photo by Sanket Mishra — AI – Photo by Sanket Mishra

Artificial intelligence is making it easier to create convincing emails, images, audio and video; and cyber criminals are taking notice.

During our latest Cyber Byte, our panel looked at the growing use of AI in cyber attacks, from convincing phishing messages and synthetic media, to voice cloning and deepfakes.

But one message came through particularly strongly. While AI may be changing how attacks are created and delivered, many of the ways organisations can protect themselves remain reassuringly familiar.

AI is making attacks easier

AI doesn’t necessarily require cyber criminals to invent completely new types of attack. Instead, it can help them carry out existing attacks more quickly, convincingly and at a greater scale.

Phishing messages can be better written and personalised. Images and documents can be manipulated. Voices can be cloned. Video can be generated. Attackers can also use AI tools to help identify vulnerabilities or automate parts of an attack.

That creates a particular challenge for organisations because some of the warning signs people have traditionally relied upon such as a poorly written email, an unusual tone of voice or something that simply doesn’t look quite right, are all becoming less reliable.

Seeing and hearing isn’t always believing

Deepfakes and synthetic media add another dimension to social engineering.

An urgent request may appear to come from a senior colleague. A voice on a telephone call might sound familiar. Even video shouldn’t automatically be treated as proof that someone is who they claim to be.

The answer isn’t to distrust everything. It’s to make sure important actions don’t depend solely on whether a message, voice or image appears to be genuine.

For example, this could be as simple as calling someone back using a number you already hold, checking a request through another communication channel or agreeing safe words for particularly sensitive transactions.

Don’t forget the cyber security basics

One of the strongest messages from the session was that AI doesn’t make established cyber security controls obsolete. In fact, it makes getting the basics right even more important.

All organisations should continue to:

Keep software and systems patched and up to date.
Use multi-factor authentication and appropriate access controls.
Remove unnecessary administrator privileges.
Understand which systems and services are exposed to the internet.
Maintain secure and tested backups.
Train staff to recognise social engineering and unusual requests.
Have clear processes for independently verifying sensitive or financial requests, and
Maintain and regularly test an incident response plan.

Attackers may increasingly use AI to find or exploit weaknesses faster, which means organisations also need to become quicker at identifying or addressing them.

Understand how AI is being used inside your organisation

The risk isn’t limited to criminals using AI.

Businesses are adopting AI assistants and other tools themselves, sometimes without having a clear picture of what information employees are entering into them.

That makes governance important.

Organisations should understand which AI tools are being used, what information can be shared with them and whether sensitive, personal, customer or commercially confidential information could leave the organisation.

Clear policies can help staff understand what is, and isn’t, appropriate.

The same principle applies when buying AI-enabled products from suppliers. Organisations should understand how their information is processed, where it is stored and what security controls sit around the service.

Plan for an AI-related incident

Incident response plans also need to keep pace with the technology.

What would your organisation do if a convincing deepfake was used to impersonate a senior colleague? What if an AI-enabled service exposed customer information? Or an employee inadvertently shared sensitive information with an AI platform?

The response isn’t purely technical.

Depending on the incident, legal, data protection, communications, insurance and senior leadership teams may all need to be involved. Knowing who needs to make decisions, and how to reach them, before an incident happens can save valuable time when something goes wrong.

The key lesson – AI changes the threat, not every defence

AI will continue to change what cyber attacks look like.

But organisations shouldn’t become so focused on the technology that they overlook the controls that already prevent a large proportion of attacks.

Strong access controls, patching, verification processes, staff awareness, secure backups and a tested incident response plan still provide strong foundations.

The difference is that organisations now need to consider how AI affects those controls, both when it’s being used against them and when it is being used by them.

5 Key Takeaways

Know who is responsible for AI. Make sure there is clear ownership and accountability for how AI is used across your organisation.
Set clear rules for using AI safely. Have an AI policy that explains which tools staff can use, what information they can share and what they should avoid.
Keep AI security on everyone’s radar. Regularly remind staff – from employees through to senior leaders and board members – about the risks and their responsibilities when using AI.
Understand the difference between free and paid AI tools. Don’t assume all AI services handle your information the same way. Check how your data is used, stored and protected before using a tool for business purposes.
Agree a simple way to verify important requests. Deepfake audio and video can make impersonation more convincing. Use a separate check; such as a known phone number, second communication channel or agreed safe word, before approving sensitive requests or financial transactions.

Our next AI discussions

Our next free Cyber Byte webinar takes the discussion from AI as an attack tool, to the security of AI-enabled applications themselves.

Cyber Byte: Are Your AI and Web Apps Secure? Will look at traditional web application vulnerabilities alongside newer AI-specific risks, including what happens when AI is connected to organisational systems and data.

Additionally, we still have a couple of spaces left at our Sword and the Shield: AI and Cyber Security in-person morning workshop taking place on 1^st September from 8:30 – midday. During this practical morning briefing, we will explore how organisations can benefit from AI while protecting themselves from the emerging cyber, fraud, governance and reputational risks that come with technology.

25.08.26