0%
BACK TO OVERVIEW

So Real It's Deceptive: Deepfakes! A Technical Look Behind the Scenes

So Real It's Deceptive: Deepfakes! A Technical Look Behind the Scenes
Deceptively Real with Deepfakes! A Technical Look Behind the Scenes – Access Granted

A few seconds of voice recording. A single photo. That's all it takes today to clone a voice or a face convincingly.

May 2024: Mark Read, CEO of the advertising holding company WPP, becomes the target of an AI-generated voice clone of himself – fraudsters attempt to deceive colleagues using his cloned voice and a fake video call.

January 2024: At the British design and engineering firm Arup, a highly sophisticated video vishing attack leads an employee to transfer funds after being instructed to do so during a fake video conference with supposed executives.

June 2026: At the IT service provider MSG, a vishing attack results in a data leak of 26 million records.

Three completely different companies, three completely different industries – one common denominator: deepfake technology, used as a tool for social engineering. Vishing has therefore long ceased to be a simple phone call and has become, in the age of generative AI, one of the most effective attack surfaces of all – and the numbers confirm just how fast this is developing.

0.1 → 6.5%
Share of deepfakes in fraud attempts, increase over three years
Every 5 min.
A deepfake attack somewhere in the world, on average in 2024
+53%
Growth in deepfake incidents in Germany, 2025
0.1%
People who can reliably detect every type of deepfake

These figures raise two central questions that we explore in this article, both practically and technically: how low is the barrier to entry for attackers really – and how do deepfakes work under the hood? To answer this, we didn't cite a statistic – we created a deepfake ourselves.

What exactly are deepfakes?

The term is a combination of Deep Learning and Fake. It refers to generated video, image, or audio material that uses AI to manipulate human faces or voices so that one person is replaced by another or convincingly simulated – visually and acoustically almost indistinguishable from the original.

The technology itself is neutral. It's the application that determines whether it results in benefit or harm:

Beneficial Use
Benefit
Creative · Medical · Technical
Photo editing and creative post-production
Digitally inserting deceased actors into films after the fact
Perfect dubbing of films into other languages
Voice restoration for people who have lost their voice due to illness
Malicious Use
Harm
Fraud · Manipulation · Loss of Trust
Fraud and cybercrime, e.g. via vishing
Bullying and targeted defamation of individuals
Political manipulation and disinformation
Loss of societal trust, e.g. in evidence presented in court

The same technology that gives a person their own voice back can, in the wrong hands, be used to impersonate someone else. The difference lies not in the tool, but in the intent behind it.

Image, audio, or video – what kind of deepfake should it be?

Before a deepfake is created, a simple decision has to be made: which medium should be manipulated? The three common categories differ significantly in effort and the source material required.

01
Images – Face Swap & Static Image Material
A photo is taken and one's own face is swapped via AI for the face of another person, e.g. a celebrity. All that's required is a picture of yourself and a picture of the person whose face is to be swapped.
02
Audio – Voice Cloning & Text-to-Speech
Any text is entered, a cloned voice is selected, and the model reads out the text in that voice. Voice cloning creates a replica of a real person's voice – 3 to 10 seconds of clean voice recording are now enough, and pre-built voice models can be used as an alternative.
03
Video – Lip-Sync from Image & Audio
A combination of a cloned voice and an image or video whose mouth movements are synchronized to the new audio. Result: a moving, talking likeness of a person who never said what is being heard – technically the most elaborate, but most effective, type of deepfake.

Tried it ourselves: how easy is a deepfake really?

To honestly assess the barrier to entry, we didn't cite a report – we got our hands dirty ourselves. Goal: a short video in which the British actor Tom Holland – known for his lead role as Spider-Man – says something he never said. The platform ElevenLabs was used for this.

01
Clone Voice & Generate Audio
The source material was an interview publicly available on YouTube, trimmed down to roughly one minute of spoken content. ElevenLabs was used to build a voice model of the target person from it, and text-to-speech was used to generate arbitrary, freely written text in that voice.
02
Video Material & Lip-Sync
A high-resolution photo of the target person was additionally sourced from the internet. Using ElevenLabs' "Image & Video" feature, a lip-sync avatar model (HeyGen Avatar 4) was used to merge the photo with the previously created audio track into a talking video.

The generated script for the demo clip was deliberately worded so that the fake reveals itself at the end:

Script // Generated Demo Clip „Hey there! Welcome to New York! ... Well, actually, I'm not really there. And to be honest: I'm not even real! This entire video was generated by the tech team at Access Granted using AI voice cloning and lip-sync tools. Pretty mind-blowing, right!?"
???? Final Audio Result (Voice Clone)
???? Video with Lip-Sync and Sound

The entire process – from voice recording to the finished, talking video – could be carried out using freely accessible tools and without specialist technical knowledge. That's the real takeaway: the barrier today no longer lies in the technology, but at most in the price.

How a classic deepfake is technically created

Behind the simple user interface lies a multi-stage process built on computer vision and neural networks. Using classic face-swapping as an example, this can be broken down into four steps.

01
Extraction and Data Separation
Face detection algorithms such as MTCNN or RetinaFace scan every frame of a video and locate faces. Landmark tracking marks the eyes, nose, and corners of the mouth, and the face is isolated, scaled, and aligned. The result is two datasets: Person A (source, whose face is to be shown) and Person B (target, whose face in the original video is to be replaced).
02
Training: Shared Encoder & Dual Decoder
The core of the classic deepfake architecture consists of three components: a shared encoder that compresses the facial data of both people into an abstract mathematical vector (latent space) – independent of identity, but capturing gaze direction, head tilt, and mouth opening – and two separate decoders, each trained only to reconstruct the face of Person A or Person B from this latent space.
03
The Swap: Generation
For the actual swap, an image of Person B is passed through the shared encoder, which determines the current facial expression and position. This result is passed to Decoder A – which then generates the face of Person A, but with exactly the facial expression, gaze direction, and head pose of Person B.
04
Post-Processing & Compositing
So that the inserted face doesn't look like a mask stuck on top, automated post-processing steps follow: color matching adjusts skin tone and lighting to match the surroundings, and blending creates smooth transitions at the edges of the face.

What attackers actually use deepfakes for today

These technical capabilities translate directly into concrete attack scenarios – from bypassing biometric security systems to targeted reputational damage.

Domain Attack Objective Description
Authentication Defeating Biometric Systems Since fake voices and faces can now sometimes be generated live, deepfakes effortlessly bypass digital security controls such as telephone voice recognition – the system simply lacks any way to look behind the curtain remotely.
Social Engineering Targeted Fraud Using deceptively realistic voices and images, fraudsters trick victims by, for example, perfectly imitating their boss on the phone and thereby forcing blind obedience for costly wire transfers.
Public Discourse Disinformation Fake videos of politicians or celebrities can create a perceived truth online within seconds, deliberately deceiving and manipulating entire societies.
Reputation Defamation Because deepfakes can attribute any scandal or statement to any person, a single fake video is enough to permanently ruin a victim's reputation.

How to spot deepfakes – what to actually look out for

Awareness is the key countermeasure: anyone who knows how such an attack works technically can assess the authenticity of image, video, and audio material far more accurately. Even today, most deepfakes still leave behind typical, recurring artifacts.

Visual Artifacts
Image & Video
Facial Manipulation
Messy transitions at the edges of the inserted face, visible seams, or inconsistent skin texture
The original face showing through in individual frames, e.g. doubled eyebrows visible at the edge
Blurry, indistinct fine details such as teeth or eyes
Empty gaze without blinking, inconsistent lighting, artifacts during strong side profiles
Acoustic Artifacts
Synthetic Voices
Voice Cloning & TTS
A metallic, slightly artificial timbre to the voice
Mispronunciation of individual words, often with foreign-language terms within a differently trained TTS model
Monotone speech output with noticeably consistent intonation

No single characteristic on its own proves a fake. Only the combination of several small inconsistencies – both visual and acoustic – produces a reliable picture. For security-critical decisions, technical material should therefore never be the sole basis for verification.

Conclusion: the barrier has fallen – vigilance must rise

Our own test makes it clear: a convincing deepfake today is no longer a matter of elaborate specialist software, but a matter of a few seconds of source material and a freely accessible tool. What was once reserved for Hollywood studios is now within reach of a few clicks – and that's exactly what makes vishing and other deepfake-powered attacks so dangerously effective.

Technical understanding remains the best protection: anyone who knows how encoders, decoders, and lip-sync models work together also recognizes the limits of the technology – the subtle artifacts that still give fakes away.

Further reading on this topic:

  • Deepfake Vishing – how vishing works in practice through digital channels
  • Pretexting – how cover stories are built for physical attacks
  • MGM Hack – what vishing can do on a large scale

Would your team recognize a deepfake call?

We use realistic vishing and deepfake scenarios to test whether your employees and processes hold up in a real incident – and show you in the report exactly where the gap lies.

Request an initial consultation →
Tags // #Vishing #SocialEngineering #CyberSecurity #Phishing #Deepfake #VoiceCloning #AIFraud #VoiceClone #Desinformation

© AccessGranted X GmbH