> For the complete documentation index, see [llms.txt](https://neuraldefend.gitbook.io/neural-defend/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://neuraldefend.gitbook.io/neural-defend/face-swap-and-deepfake-detection-api-guardrails-and-scenarios.md). # Face swap & Deepfake Detection API - Guardrails & Scenarios **Request:** Please list your ideal real-world case scenarios so that Neural Defend can tailor the solution for your specific needs. Some cases mentioned earlier will be resolved in the upcoming update scheduled before 24th Feb.\ \ Category 1: Image Quality & Lighting Issues > **Note:** Image quality guardrails (1.1-1.6) will be rolled out by **24th Feb** (may release earlier for testing). Quality error guardrails (1.8, 1.9) will be rolled out by **28th Feb** - API will return **quality error** with defined failure scenarios. Updated API document will be shared along with rollout. | # | Failure Scenario | Why API Fails | Action Steps | | ---- | --------------------------------------- | ----------------------------------------------------------------------------------- | -------------------------------------------------------- | | 1.1 | Overexposed image | Facial features washed out, model misclassifies as AI-generated | Planned for rollout (by 24th Feb, may release earlier) | | 1.2 | Underexposed image | Facial details lost in shadows | Planned for rollout (by 24th Feb, may release earlier) | | 1.3 | High contrast | Unnatural shadows confuse model | Planned for rollout (by 24th Feb, may release earlier) | | 1.4 | Low contrast | Flat image, features indistinguishable | Planned for rollout (by 24th Feb, may release earlier) | | 1.5 | Motion blur | Blurred edges mimic deepfake artifacts | Planned for rollout (by 24th Feb, may release earlier) | | 1.6 | Out of focus | Soft edges similar to AI generation | Planned for rollout (by 24th Feb, may release earlier) | | 1.7 | Low resolution | Insufficient detail for analysis | Already implemented | | 1.8 | Heavy compression (JPEG artifacts) | Block artifacts mimic deepfake traces | Quality error rollout (by 28th Feb) | | 1.9 | Noise/grain | Random noise interferes with detection | Quality error rollout (by 28th Feb) | | 1.10 | Backlighting/silhouette | Face in shadow, background bright | Covered by 1.2 (Underexposed) | | 1.11 | Uneven lighting | Half face lit, half dark | Covered by 1.3/1.4 (Contrast checks) | | 1.12 | Colored lighting | Unnatural skin tones | Planned for rollout (by 24th Feb, may release earlier) | | 1.13 | Glare/reflection on skin | Specular highlights distort features | Covered by 1.1 (Overexposed) | | 1.14 | Strong shadows on face | Shadows create false edges | Covered by 1.3 (High contrast) | | 1.15 | Window/sunlight backlight | Bright window behind subject, face appears darker | Covered by 1.1/1.2 (Exposure checks) | | 1.16 | Mixed indoor/outdoor lighting | Inconsistent color temperature | Covered by 1.12 (Color checks) | | 1.17 | Overhead ceiling light only | Creates shadows under eyes, nose, chin | Covered by 1.3 (Contrast checks) | | 1.18 | Fluorescent light flicker | Banding artifacts in video frames | Quality error rollout (by 28th Feb) | | 1.19 | Pixelated/blurry image (no sharpening) | Image lacks sharpness, pixelated appearance | API may fail - Covered by 1.5/1.6 (blur/focus checks) | | 1.20 | Low quality webcam capture | Poor webcam quality produces low detail image | API may fail - Covered by 1.7 (resolution check) | | 1.21 | Dirty webcam lens | Smudges/dirt on lens cause haze or spots | API may fail - Covered by 1.6 (focus) and 1.1 (exposure) | | 1.22 | Low image clarity | Overall poor clarity, unclear facial details | API may fail - Covered by quality checks | | 1.23 | Polished/smoothed face (camera quality) | Face lacks texture details, appears overly smooth due to poor webcam/camera quality | API may fail - No facial information to analyze | *** ### Category 2: Presentation Attacks (Liveness/Spoof) > **Note:** Current API focuses on **Face Swap Detection** and **AI Generated Detection** only. Liveness check is **out of scope** for this API. If liveness detection is required, please use our **Unified API** (Liveness + AI Face Swap + AI Generated Deepfake Detection). | # | Failure Scenario | Why API Fails | Action Steps | | --- | ---------------------------- | ------------------------------------------------------ | ----------------------------------------------------- | | 2.1 | Photo of printed photo | Camera captures "real" photo, no digital artifacts | Out of scope - Use Unified API for liveness detection | | 2.2 | Photo of screen/monitor | Screen removes AI artifacts, looks like camera capture | Out of scope - Use Unified API for liveness detection | | 2.3 | Photo of phone screen | Same as above | Out of scope - Use Unified API for liveness detection | | 2.4 | Video replay on screen | Playing deepfake video on screen | Out of scope - Use Unified API for liveness detection | | 2.5 | 3D printed mask | Physical mask bypasses 2D checks | Out of scope - Use Unified API for liveness detection | | 2.6 | Silicone/latex mask | Realistic masks with skin texture | Out of scope - Use Unified API for liveness detection | | 2.7 | Cutout photo with eye holes | Basic attack with real eye movement | Out of scope - Use Unified API for liveness detection | | 2.8 | Projected face on 3D surface | Projection on mannequin | Out of scope - Use Unified API for liveness detection | *** ### Category 3: Face Detection Issues > **Note:** Guardrails for no face, multiple faces, and face too small are already implemented. User should avoid sunglasses, masks, cloth, or hair covering face for better results. | # | Failure Scenario | Why API Fails | Action Steps | | --- | -------------------------- | ----------------------------------- | ---------------------------------------------------------------------- | | 3.1 | Partial face | Incomplete features for analysis | Planned for update (before 24th Feb) - Partial faces will be rejected | | 3.2 | Face too small | Insufficient pixels for detection | Already implemented - Face too small rejection | | 3.3 | Face too large/close | Features cropped out | Planned for update (before 24th Feb) | | 3.4 | Extreme head angle | Profile view lacks frontal features | Planned for update (before 24th Feb) | | 3.5 | Multiple faces | Confusion on which face to analyze | Already implemented - Multiple face rejection | | 3.6 | No face detected | Background or non-face image | Already implemented - No face rejection | | 3.7 | Occluded face (sunglasses) | Eyes hidden, key features missing | Out of scope - User should avoid sunglasses for better results | | 3.8 | Occluded face (mask/cloth) | Mouth/nose hidden | Out of scope - User should avoid mask/cloth for better results | | 3.9 | Hair covering face | Features partially hidden | Out of scope - User should avoid hair covering face for better results | *** ### Category 4: Face Swap Detection > **Note:** Low quality face swap or images with lower quality facial pixels - model may not detect. This case has been discussed and we are working on improving accuracy on low quality images and real-world scenarios with high brightness, background contrast images. Continuous improvements are ongoing. > > **In case of any specific scenarios, please let us know.** | # | Failure Scenario | Why API Fails | Action Steps | | ---- | --------------------------------------------- | --------------------------------------------------------------------------------------- | ---------------------------------------------------------- | | 4.1 | High-quality face swap | Advanced swaps leave no visible artifacts | Already fixed | | 4.2 | Low quality face swap | Lower quality facial pixels, model may miss detection | Working on improving accuracy - discussed case | | 4.3 | Face reenactment (expression transfer) | Expression/pose transfer on face | Already fixed | | 4.4 | Morphed face (two identities) | Blended passport-style photo | Already fixed | | 4.5 | Post-processed face swap | Filters/edits remove artifacts | Under evaluation - case identified, continuous improvement | | 4.6 | Same gender swap (Male→Male, Female→Female) | Similar facial structure makes swap seamless | Already fixed | | 4.7 | Cross gender swap (Male→Female or vice versa) | Gender mismatch in features | Already fixed | | 4.8 | Same ethnicity swap | Similar skin tone/features | Already fixed | | 4.9 | Cross ethnicity swap | Different skin tone/features | Already fixed | | 4.10 | Age difference swap (Young→Old or vice versa) | Age mismatch in skin texture | Already fixed | | 4.11 | Celebrity face swap | Famous face on regular person | Already fixed | | 4.12 | Face swap with glasses | Glasses on swapped face | Already fixed | | 4.13 | Face swap with beard/facial hair | Hair texture mismatch | Already fixed | | 4.14 | Face swap with makeup | Heavy makeup hides boundaries | Already fixed | | 4.15 | Face swap with different lighting | Lighting inconsistency between face and background | Under improvement - case discussed, Feb 24 rollout | | 4.16 | Face swap on tilted/angled face | Non-frontal pose swap | Already fixed | | 4.17 | Model agnostic detection | Detection works regardless of which tool/model used (DeepFaceLab, Roop, FaceSwap, etc.) | Already fixed | | 4.18 | Real-time/live face swap | Face swap during live video call | Some cases fixed, edge cases being worked on | | 4.19 | Latest face swap models support | Detection of face swaps from latest tools and models | Already fixed | *** ### Category 5: AI Generated Face Detection > **Note:** Model is updated with AI models launched till January. For newly launched AI models (till 10th Feb), update will be rolled out **before 24th Feb**. > > **In case of any specific scenarios, please let us know.** | # | Failure Scenario | Why API Fails | Action Steps | | ---- | ---------------------------------------------- | ------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | | 5.1 | AI-generated face (StyleGAN, Midjourney, etc.) | Perfect synthetic faces from generative models | Already fixed | | 5.2 | AI-generated face (DALL-E, Stable Diffusion) | Text-to-image generated faces | Already fixed | | 5.3 | AI-generated face (new Feb 2025 models) | Newly launched AI models | Already fixed | | 5.4 | Lip-sync deepfake | Only mouth region manipulated | Out of scope | | 5.5 | AI face with unusual lighting | AI generated face with extreme lighting conditions | Already fixed - Working perfectly | | 5.6 | AI face with accessories (glasses, hat) | AI generated face with added accessories | Already fixed - Working perfectly | | 5.7 | AI face with partial occlusion | AI generated face with hair/hand covering | Already fixed - Working perfectly | | 5.8 | AI face at unusual angles | AI generated face in non-frontal pose | Already fixed - Working perfectly | | 5.9 | Low quality AI generated face | Compressed or low resolution AI face | Already fixed - Working perfectly | | 5.10 | Model agnostic detection (200+ AI models) | Supports 200+ AI generation models from different companies and open source | Already fixed - Working well | | 5.11 | Slight changes in real images using AI | Minor AI edits like eye color change, mustache/beard addition, hairstyle modification | Out of scope - Not supported for video verification (relevant to forensic tools) | *** ### Category 6: Technical/Input Issues | # | Failure Scenario | Why API Fails | Action Steps | | ---- | --------------------------- | -------------------------- | ----------------------------------- | | 6.1 | Corrupted image file | Cannot decode | Already handled - API returns error | | 6.2 | Unsupported format | API expects JPEG/PNG | Already handled - API returns error | | 6.3 | Image too large | Timeout or memory issues | Already handled - API returns error | | 6.4 | Image too small | Insufficient data | Already handled - API returns error | | 6.5 | Wrong color space (CMYK) | Color processing error | Planned for rollout (by 28th Feb) | | 6.6 | Animated GIF/APNG | Multiple frames confusion | Already handled - API returns error | | 6.7 | Embedded ICC profile issues | Color rendering mismatch | Planned for rollout (by 28th Feb) | | 6.8 | EXIF orientation mismatch | Face appears rotated | Already handled | | 6.9 | Truncated/incomplete file | Partial upload | Already handled - API returns error | | 6.10 | Screenshot with UI elements | Non-face elements in frame | Out of scope | *** ### Category 7: Video KYC Specific Issues | # | Failure Scenario | Why API Fails | Action Steps | | --- | --------------------------------- | ------------------------------------- | --------------------------------------- | | 7.1 | Frame extracted during transition | Motion blur, partial face | Covered by Category 1 & 3 guardrails | | 7.2 | Video codec artifacts | Compression blocks mimic deepfake | Covered by Category 1 (1.8 compression) | | 7.3 | Low bitrate video source | Detail loss | Covered by Category 1 (1.7 resolution) | | 7.4 | Screen recording of video call | Double compression | Covered by Category 1 (1.8 compression) | | 7.5 | Video filter active (beauty mode) | Smoothing mimics AI generation | Under evaluation | | 7.6 | Agent captures at wrong moment | Eyes closed, mouth open, looking away | Out of scope - SOP recommendation | | 7.7 | Network lag causing frame drop | Corrupted/incomplete frames | Covered by Category 6 (6.9 truncated) | *** ### Category 8: Edge Cases & Unusual Inputs | # | Failure Scenario | Why API Fails | Action Steps | | ---- | ----------------------------- | ------------------------------------- | ------------------------------------------------------------ | | 8.1 | Twins/lookalikes | Similar faces confuse matching | Not a deepfake issue (ID verification scope) | | 8.2 | Heavy makeup/prosthetics | Altered appearance | Under evaluation | | 8.3 | Plastic surgery changes | Face structure altered | Under evaluation | | 8.4 | Elderly with many wrinkles | Texture patterns differ from training | Already handled | | 8.5 | Very young children | Different facial proportions | Out of scope | | 8.6 | Unusual skin conditions | Texture anomalies | Already handled | | 8.7 | Tattoos on face | Ink patterns detected as manipulation | Already fixed - Working well | | 8.8 | Non-human image submitted | Animal, cartoon, object | Covered by Category 3 (no face rejection) | | 8.9 | Celebrity/public figure photo | Stock photo submission | Out of scope | | 8.10 | ID card photo submitted | Document photo, not selfie | Also comes under spoof attack - Use Unified API for liveness | *** ### Category 9: Document + Face Verification (Video KYC) | # | Failure Scenario | Why API Fails | Action Steps | | ---- | ------------------------------------ | -------------------------------------------------- | ----------------------------------------------------- | | 9.1 | ID card held in frame with face | Multiple faces detected (live + document photo) | Covered by Category 3 (multiple face handling) | | 9.2 | Document photo confused as live face | API analyzes wrong face | Covered by Category 3 (face size - largest face used) | | 9.3 | Document photo flagged as spoof | ID photo detected as "photo attack" false positive | Out of scope - Use Unified API | | 9.4 | Hand/fingers occluding face | User holds document near face | Covered by Category 3 (partial face) | | 9.5 | Document glare/reflection | Shiny ID card surface creates hotspots | Out of scope - Document verification | | 9.6 | Document out of focus | User focuses on face, document blurry | Out of scope - Document verification | | 9.7 | Wrong document shown | User shows incorrect ID type | Out of scope - Document verification | | 9.8 | Document tilted/angled | Cannot read document details | Out of scope - Document verification | | 9.9 | Fake/edited document | Tampered ID card | Out of scope - Document verification | | 9.10 | Face-document mismatch | Different person on ID vs live face | Out of scope - Face matching API | *** ### Summary | Category | Status | | --------------------------------------- | ---------------------------------------------------------- | | Category 1: Image Quality & Lighting | 1.1-1.6 by 24th Feb; Quality errors (1.8, 1.9) by 28th Feb | | Category 2: Liveness/Spoof | Out of scope - Use Unified API | | Category 3: Face Detection | Partially implemented, update before 24th Feb | | Category 4: Face Swap Detection | Mostly fixed, continuous improvement | | Category 5: AI Generated Face Detection | Already fixed - 200+ models supported | | Category 6: Technical/Input | Mostly implemented | | Category 7: Video KYC | Covered by other categories | | Category 8: Edge Cases | Under evaluation | | Category 9: Document + Face | Covered by other categories |