Deepfake Defense: An IDS/IPS
for Identity Verification

The Threat Landscape

In 2024, deepfake-related identity fraud increased by over 300%. The tooling has crossed a critical threshold: you no longer need a research lab to generate a photorealistic face. Consumer-grade apps can produce a face swap in real time. Open-source models like Stable Diffusion and ROOP can generate synthetic identities that pass a casual human inspection.

For KYC providers, this changes the game completely. The question is no longer "Can we match two faces?" — it's "Is the face we're looking at even real?"

No single detection technique defeats all five attack levels. A printed photo check won't catch a GAN face. Liveness detection won't catch a real-time face swap. That's why FaceVault doesn't rely on any single signal — it runs five independent detectors and fuses their scores.

Why IDS/IPS, Not a Single Model

Network security doesn't rely on a single firewall rule. An intrusion detection system (IDS) runs multiple detection engines in parallel — signature-based, anomaly-based, protocol analysis — and correlates their alerts to produce a confidence score. If that score exceeds a threshold, the intrusion prevention system (IPS) blocks the traffic.

FaceVault's anti-spoofing pipeline follows the same architecture:

This design gives us two critical properties. Defense in depth: an attacker must beat all five signals simultaneously, not just one. Graceful degradation: if one signal fails or isn't available, the remaining signals redistribute its weight and still produce a meaningful score.

Signal 1: Eye Specular Highlights

When light hits a real eye, the curved surface of the cornea produces small, bright specular reflections. In a normal environment, both eyes receive similar lighting and produce bilateral reflections — roughly symmetric highlights in both corneas. Printed photos, screens, and many GAN-generated images fail this test.

# MediaPipe iris landmarks: left 468-472, right 473-477
for each eye:
    crop iris region from grayscale image
    threshold at max(95th percentile, 200) brightness
    count connected bright spots (specular highlights)
    compute area ratio, mean brightness

# Scoring (4 sub-scores)
presence:  both eyes must have >= 1 highlight     (0.35 weight)
symmetry:  highlight counts should be similar      (0.25 weight)
area:      small bright spots, not uniform glow    (0.20 weight)
brightness: highlights >> surrounding tissue       (0.20 weight)

Screens produce uniform brightness across the entire eye region (area ratio too high). Printed photos produce no highlights at all (area ratio near zero). GAN-generated eyes often show asymmetric or absent reflections because the generator doesn't model specular optics. Real eyes sit in a narrow sweet spot: small, bright, bilateral spots.

Signal 2: Blendshape Micro-Expressions

MediaPipe's Face Landmarker outputs 52 blendshape coefficients — a numerical decomposition of facial expression into components like eyeBlinkLeft, jawOpen, browOuterUpRight. Even when a person tries to hold perfectly still, micro-expressions produce measurable activation across 30-60% of these blendshapes.

A printed photo produces zero activation across all blendshapes. A static GAN image is the same. Even a video replay tends to show either uniformly low or unnaturally high activation because the replay lighting and angle don't match the live capture environment.

# 1. Non-zero ratio (what % of blendshapes are active?)
#    Real face: 15-60%  |  Flat photo: <5%

# 2. Total energy (sum of all activations)
#    Real face: 0.3-8.0  |  Dead flat: <0.1

# 3. Blink activity (eyeBlinkLeft or eyeBlinkRight > 0.01)
#    Real face: nearly always  |  Photo: never

# 4. Micro-expression diversity (variance of activations)
#    Real face: varied  |  Photo/GAN: uniform

Final = nz_score * 0.30 + energy * 0.30 + blink * 0.15 + diversity * 0.25

This signal is particularly valuable as a fast pre-filter. A completely dead blendshape vector (total energy < 0.1) is an almost certain presentation attack, allowing us to flag sessions before running the more expensive depth and rPPG analysis.

Signal 3: GAN Texture Fingerprints

Every GAN architecture — StyleGAN, DDPM, Stable Diffusion — uses learned upsampling to generate images. These upsampling layers imprint periodic artifacts in the frequency domain that are invisible to the human eye but clearly detectable via spectral analysis. Think of it as a watermark baked into the generation process itself.

Real photographs exhibit a smooth 1/f power spectral density falloff — lower frequencies carry more energy, higher frequencies carry less, with a smooth gradient between them. GAN-generated images break this pattern with periodic peaks and irregular high-frequency energy.

# Crop face, resize to 256x256, compute 2D DCT
dct = cv2.dct(face_float)
power = log(1 + |dct|)

# Azimuthal average (radial power spectral density)
for each radius r:
    radial_profile[r] = mean(power at distance r from center)

# Three detection metrics:
smoothness  = 1/(1 + gradient_variance * 10)     # Smooth falloff?
peak_energy = spectral peaks vs. smoothed baseline # Periodic spikes?
hf_ratio    = high_freq_energy / low_freq_energy   # Unusual HF boost?

# GANs show: high gradient variance, periodic peaks, anomalous HF ratio
score = smoothness * 0.40 + peak_score * 0.35 + hf_score * 0.25

This signal carries the highest weight in our fusion engine (0.30) because it targets the most sophisticated attack vector — synthetic face generation — and is extremely difficult to evade. Even post-processing a GAN image (JPEG compression, noise, blur) only partially attenuates the spectral fingerprint.

Signal 4: Monocular Depth Estimation

A real human face is a 3D object. The nose protrudes forward. The ears recede. The forehead curves. When you photograph a printed photo or display a face on a screen, this 3D structure is lost — the depth map becomes flat.

FaceVault uses Depth Anything ViT-S — a compact vision transformer trained on millions of images to predict per-pixel depth from a single photograph. We run it as an ONNX model on CPU, keeping inference fast and dependency-free.

# 1. Run Depth Anything ViT-S ONNX inference
input  = resize(image, 518x518) + ImageNet normalize
output = depth_map (H x W float32, relative depth)

# 2. Crop face region using MediaPipe landmarks
face_depth = depth_map[face_bbox]

# 3. Three metrics:
depth_range:    max - min across face region
                Real: > 0.15  |  Flat: < 0.05

nose_ear_diff:  mean(center strip) - mean(edge strips)
                Real: > 0.05 (nose closer)  |  Flat: ~0

gradient_std:   variation in depth gradient magnitude
                Real: smooth gradients  |  Screen: too uniform

score = range * 0.40 + protrusion * 0.35 + smoothness * 0.25

Signal 5: Remote Photoplethysmography (rPPG)

This is the signal that deepfakes fundamentally cannot replicate. When your heart beats, blood pulses through the capillaries in your face, causing sub-pixel colour oscillations invisible to the naked eye but measurable from video. No GAN, no face swap, no screen replay produces this signal — because none of them have a cardiovascular system.

During the selfie capture, the webapp records 90 frames (~3 seconds at 30fps) from the video feed and uploads them as the rPPG sample. On the server, we extract the mean RGB value from the face ROI in each frame, then apply the POS method (Plane Orthogonal to Skin, Wang et al. 2017) to isolate the blood volume pulse from motion artifacts and illumination changes.

# Extract mean RGB per frame from face ROI
rgb_signals = [[mean_R, mean_G, mean_B] for frame in frames]

# Normalize each channel by its temporal mean
normalized = rgb / mean(rgb, axis=0)

# POS projection: isolate pulse from noise
S1 = G - B
S2 = G + B - 2R
pulse = S1 + (std(S1)/std(S2)) * S2

# Bandpass filter: 0.7-4.0 Hz (42-240 BPM)
filtered = butterworth_bandpass(pulse, order=3)

# FFT: find dominant frequency
peak_freq = argmax(|FFT(filtered)|^2) in [0.7, 4.0] Hz
bpm = peak_freq * 60
snr = peak_power / mean_noise_power

# Real face: SNR > 5 dB, BPM 50-120
score = snr_score * 0.70 + bpm_plausibility * 0.30

A real face produces a clear spectral peak with an SNR above 5 dB at a physiologically plausible heart rate (50-120 BPM resting). A printed photo produces flat noise. A screen replay may transmit some color variation from the original video, but the SNR is attenuated by the double-sampling (screen pixels → camera sensor) and the heart rate frequency shifts due to temporal aliasing.

The Fusion Engine

Five signals, each producing a score from 0.0 (definitely fake) to 1.0 (definitely real). How do we combine them?

SIGNAL_WEIGHTS = {
    "eye_specular": 0.25,   # Physical optics
    "blendshapes":  0.20,   # Facial biomechanics
    "gan_texture":  0.30,   # Frequency-domain forensics
    "depth":        0.20,   # 3D geometry
    "rppg":         0.20,   # Cardiovascular physiology
}
# Sum = 1.15 (intentionally > 1.0)
# Normalized at runtime to available signals only

The weights deliberately sum to more than 1.0. This is intentional: they represent relative importance, not probabilities. At runtime, the fusion engine filters out any signals that errored or were skipped, collects the weights for the remaining signals, normalizes them to sum to 1.0, and computes the weighted average.

This means a session where rPPG was skipped (no frames uploaded) still gets a meaningful score from the four remaining signals. The weights redistribute proportionally — just like a network IDS that still functions when one sensor is offline.

Document Fraud Detection

The selfie isn't the only attack surface. A fraudster with a real face and a forged document can pass face matching and liveness detection. FaceVault runs a parallel fraud detection pipeline on every ID document upload with four independent checks.

# Independent signals, weighted fusion
weights = {"moire": 0.40, "ela": 0.35, "exif": 0.25}

# Barcode is informational (cross-reference), not scored
# Stored alongside OCR data for developer review

combined = weighted_average(signals, weights)
# Stored as document_fraud_data JSON on the session

Graduated Enforcement

Running an IDS in monitor-only mode before switching to inline prevention is standard practice. You don't flip an IPS to blocking mode on day one — you observe, tune thresholds, measure false positive rates, then gradually enforce.

FaceVault follows the same playbook with three configuration knobs:

Every session stores the full anti-spoofing breakdown as JSON: the fused score, the pass/fail verdict, whether enforcement was active, the threshold used, per-signal scores, and the normalized weights. This gives developers complete forensic visibility — exactly what happened, why it was flagged, and which signal contributed most to the decision.

By the Numbers