> For the complete documentation index, see [llms.txt](https://neuraldefend.gitbook.io/neural-defend/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://neuraldefend.gitbook.io/neural-defend/unified-api/face-swap-and-deepfake-detection-api-guardrails-and-scenarios.md). # Face swap & Deepfake Detection API - Guardrails & Scenarios **Request:** Please list your ideal real-world case scenarios so that Neural Defend can tailor the solution for your specific needs. Some cases mentioned earlier will be resolved in the upcoming update scheduled before 24th Feb.\ \ Category 1: Image Quality & Lighting Issues > **Note:** Image quality guardrails (1.1-1.6) will be rolled out by **24th Feb** (may release earlier for testing). Quality error guardrails (1.8, 1.9) will be rolled out by **28th Feb** - API will return **quality error** with defined failure scenarios. Updated API document will be shared along with rollout. | # | Failure Scenario | Why API Fails | Action Steps | | ---- | --------------------------------------- | ----------------------------------------------------------------------------------- | -------------------------------------------------------- | | 1.1 | Overexposed image | Facial features washed out, model misclassifies as AI-generated | Planned for rollout (by 24th Feb, may release earlier) | | 1.2 | Underexposed image | Facial details lost in shadows | Planned for rollout (by 24th Feb, may release earlier) | | 1.3 | High contrast | Unnatural shadows confuse model | Planned for rollout (by 24th Feb, may release earlier) | | 1.4 | Low contrast | Flat image, features indistinguishable | Planned for rollout (by 24th Feb, may release earlier) | | 1.5 | Motion blur | Blurred edges mimic deepfake artifacts | Planned for rollout (by 24th Feb, may release earlier) | | 1.6 | Out of focus | Soft edges similar to AI generation | Planned for rollout (by 24th Feb, may release earlier) | | 1.7 | Low resolution | Insufficient detail for analysis | Already implemented | | 1.8 | Heavy compression (JPEG artifacts) | Block artifacts mimic deepfake traces | Quality error rollout (by 28th Feb) | | 1.9 | Noise/grain | Random noise interferes with detection | Quality error rollout (by 28th Feb) | | 1.10 | Backlighting/silhouette | Face in shadow, background bright | Covered by 1.2 (Underexposed) | | 1.11 | Uneven lighting | Half face lit, half dark | Covered by 1.3/1.4 (Contrast checks) | | 1.12 | Colored lighting | Unnatural skin tones | Planned for rollout (by 24th Feb, may release earlier) | | 1.13 | Glare/reflection on skin | Specular highlights distort features | Covered by 1.1 (Overexposed) | | 1.14 | Strong shadows on face | Shadows create false edges | Covered by 1.3 (High contrast) | | 1.15 | Window/sunlight backlight | Bright window behind subject, face appears darker | Covered by 1.1/1.2 (Exposure checks) | | 1.16 | Mixed indoor/outdoor lighting | Inconsistent color temperature | Covered by 1.12 (Color checks) | | 1.17 | Overhead ceiling light only | Creates shadows under eyes, nose, chin | Covered by 1.3 (Contrast checks) | | 1.18 | Fluorescent light flicker | Banding artifacts in video frames | Quality error rollout (by 28th Feb) | | 1.19 | Pixelated/blurry image (no sharpening) | Image lacks sharpness, pixelated appearance | API may fail - Covered by 1.5/1.6 (blur/focus checks) | | 1.20 | Low quality webcam capture | Poor webcam quality produces low detail image | API may fail - Covered by 1.7 (resolution check) | | 1.21 | Dirty webcam lens | Smudges/dirt on lens cause haze or spots | API may fail - Covered by 1.6 (focus) and 1.1 (exposure) | | 1.22 | Low image clarity | Overall poor clarity, unclear facial details | API may fail - Covered by quality checks | | 1.23 | Polished/smoothed face (camera quality) | Face lacks texture details, appears overly smooth due to poor webcam/camera quality | API may fail - No facial information to analyze | *** ### Category 2: Presentation Attacks (Liveness/Spoof) > **Note:** Current API focuses on **Face Swap Detection** and **AI Generated Detection** only. Liveness check is **out of scope** for this API. If liveness detection is required, please use our **Unified API** (Liveness + AI Face Swap + AI Generated Deepfake Detection). | # | Failure Scenario | Why API Fails | Action Steps | | --- | ---------------------------- | ------------------------------------------------------ | ----------------------------------------------------- | | 2.1 | Photo of printed photo | Camera captures "real" photo, no digital artifacts | Out of scope - Use Unified API for liveness detection | | 2.2 | Photo of screen/monitor | Screen removes AI artifacts, looks like camera capture | Out of scope - Use Unified API for liveness detection | | 2.3 | Photo of phone screen | Same as above | Out of scope - Use Unified API for liveness detection | | 2.4 | Video replay on screen | Playing deepfake video on screen | Out of scope - Use Unified API for liveness detection | | 2.5 | 3D printed mask | Physical mask bypasses 2D checks | Out of scope - Use Unified API for liveness detection | | 2.6 | Silicone/latex mask | Realistic masks with skin texture | Out of scope - Use Unified API for liveness detection | | 2.7 | Cutout photo with eye holes | Basic attack with real eye movement | Out of scope - Use Unified API for liveness detection | | 2.8 | Projected face on 3D surface | Projection on mannequin | Out of scope - Use Unified API for liveness detection | *** ### Category 3: Face Detection Issues > **Note:** Guardrails for no face, multiple faces, and face too small are already implemented. User should avoid sunglasses, masks, cloth, or hair covering face for better results. | # | Failure Scenario | Why API Fails | Action Steps | | --- | -------------------------- | ----------------------------------- | ---------------------------------------------------------------------- | | 3.1 | Partial face | Incomplete features for analysis | Planned for update (before 24th Feb) - Partial faces will be rejected | | 3.2 | Face too small | Insufficient pixels for detection | Already implemented - Face too small rejection | | 3.3 | Face too large/close | Features cropped out | Planned for update (before 24th Feb) | | 3.4 | Extreme head angle | Profile view lacks frontal features | Planned for update (before 24th Feb) | | 3.5 | Multiple faces | Confusion on which face to analyze | Already implemented - Multiple face rejection | | 3.6 | No face detected | Background or non-face image | Already implemented - No face rejection | | 3.7 | Occluded face (sunglasses) | Eyes hidden, key features missing | Out of scope - User should avoid sunglasses for better results | | 3.8 | Occluded face (mask/cloth) | Mouth/nose hidden | Out of scope - User should avoid mask/cloth for better results | | 3.9 | Hair covering face | Features partially hidden | Out of scope - User should avoid hair covering face for better results | *** ### Category 4: Face Swap Detection > **Note:** Low quality face swap or images with lower quality facial pixels - model may not detect. This case has been discussed and we are working on improving accuracy on low quality images and real-world scenarios with high brightness, background contrast images. Continuous improvements are ongoing. > > **In case of any specific scenarios, please let us know.** | # | Failure Scenario | Why API Fails | Action Steps | | ---- | --------------------------------------------- | --------------------------------------------------------------------------------------- | ---------------------------------------------------------- | | 4.1 | High-quality face swap | Advanced swaps leave no visible artifacts | Already fixed | | 4.2 | Low quality face swap | Lower quality facial pixels, model may miss detection | Working on improving accuracy - discussed case | | 4.3 | Face reenactment (expression transfer) | Expression/pose transfer on face | Already fixed | | 4.4 | Morphed face (two identities) | Blended passport-style photo | Already fixed | | 4.5 | Post-processed face swap | Filters/edits remove artifacts | Under evaluation - case identified, continuous improvement | | 4.6 | Same gender swap (Male→Male, Female→Female) | Similar facial structure makes swap seamless | Already fixed | | 4.7 | Cross gender swap (Male→Female or vice versa) | Gender mismatch in features | Already fixed | | 4.8 | Same ethnicity swap | Similar skin tone/features | Already fixed | | 4.9 | Cross ethnicity swap | Different skin tone/features | Already fixed | | 4.10 | Age difference swap (Young→Old or vice versa) | Age mismatch in skin texture | Already fixed | | 4.11 | Celebrity face swap | Famous face on regular person | Already fixed | | 4.12 | Face swap with glasses | Glasses on swapped face | Already fixed | | 4.13 | Face swap with beard/facial hair | Hair texture mismatch | Already fixed | | 4.14 | Face swap with makeup | Heavy makeup hides boundaries | Already fixed | | 4.15 | Face swap with different lighting | Lighting inconsistency between face and background | Under improvement - case discussed, Feb 24 rollout | | 4.16 | Face swap on tilted/angled face | Non-frontal pose swap | Already fixed | | 4.17 | Model agnostic detection | Detection works regardless of which tool/model used (DeepFaceLab, Roop, FaceSwap, etc.) | Already fixed | | 4.18 | Real-time/live face swap | Face swap during live video call | Some cases fixed, edge cases being worked on | | 4.19 | Latest face swap models support | Detection of face swaps from latest tools and models | Already fixed | *** ### Category 5: AI Generated Face Detection > **Note:** Model is updated with AI models launched till January. For newly launched AI models (till 10th Feb), update will be rolled out **before 24th Feb**. > > **In case of any specific scenarios, please let us know.** | # | Failure Scenario | Why API Fails | Action Steps | | ---- | ---------------------------------------------- | ------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | | 5.1 | AI-generated face (StyleGAN, Midjourney, etc.) | Perfect synthetic faces from generative models | Already fixed | | 5.2 | AI-generated face (DALL-E, Stable Diffusion) | Text-to-image generated faces | Already fixed | | 5.3 | AI-generated face (new Feb 2025 models) | Newly launched AI models | Already fixed | | 5.4 | Lip-sync deepfake | Only mouth region manipulated | Out of scope | | 5.5 | AI face with unusual lighting | AI generated face with extreme lighting conditions | Already fixed - Working perfectly | | 5.6 | AI face with accessories (glasses, hat) | AI generated face with added accessories | Already fixed - Working perfectly | | 5.7 | AI face with partial occlusion | AI generated face with hair/hand covering | Already fixed - Working perfectly | | 5.8 | AI face at unusual angles | AI generated face in non-frontal pose | Already fixed - Working perfectly | | 5.9 | Low quality AI generated face | Compressed or low resolution AI face | Already fixed - Working perfectly | | 5.10 | Model agnostic detection (200+ AI models) | Supports 200+ AI generation models from different companies and open source | Already fixed - Working well | | 5.11 | Slight changes in real images using AI | Minor AI edits like eye color change, mustache/beard addition, hairstyle modification | Out of scope - Not supported for video verification (relevant to forensic tools) | *** ### Category 6: Technical/Input Issues | # | Failure Scenario | Why API Fails | Action Steps | | ---- | --------------------------- | -------------------------- | ----------------------------------- | | 6.1 | Corrupted image file | Cannot decode | Already handled - API returns error | | 6.2 | Unsupported format | API expects JPEG/PNG | Already handled - API returns error | | 6.3 | Image too large | Timeout or memory issues | Already handled - API returns error | | 6.4 | Image too small | Insufficient data | Already handled - API returns error | | 6.5 | Wrong color space (CMYK) | Color processing error | Planned for rollout (by 28th Feb) | | 6.6 | Animated GIF/APNG | Multiple frames confusion | Already handled - API returns error | | 6.7 | Embedded ICC profile issues | Color rendering mismatch | Planned for rollout (by 28th Feb) | | 6.8 | EXIF orientation mismatch | Face appears rotated | Already handled | | 6.9 | Truncated/incomplete file | Partial upload | Already handled - API returns error | | 6.10 | Screenshot with UI elements | Non-face elements in frame | Out of scope | *** ### Category 7: Video KYC Specific Issues | # | Failure Scenario | Why API Fails | Action Steps | | --- | --------------------------------- | ------------------------------------- | --------------------------------------- | | 7.1 | Frame extracted during transition | Motion blur, partial face | Covered by Category 1 & 3 guardrails | | 7.2 | Video codec artifacts | Compression blocks mimic deepfake | Covered by Category 1 (1.8 compression) | | 7.3 | Low bitrate video source | Detail loss | Covered by Category 1 (1.7 resolution) | | 7.4 | Screen recording of video call | Double compression | Covered by Category 1 (1.8 compression) | | 7.5 | Video filter active (beauty mode) | Smoothing mimics AI generation | Under evaluation | | 7.6 | Agent captures at wrong moment | Eyes closed, mouth open, looking away | Out of scope - SOP recommendation | | 7.7 | Network lag causing frame drop | Corrupted/incomplete frames | Covered by Category 6 (6.9 truncated) | *** ### Category 8: Edge Cases & Unusual Inputs | # | Failure Scenario | Why API Fails | Action Steps | | ---- | ----------------------------- | ------------------------------------- | ------------------------------------------------------------ | | 8.1 | Twins/lookalikes | Similar faces confuse matching | Not a deepfake issue (ID verification scope) | | 8.2 | Heavy makeup/prosthetics | Altered appearance | Under evaluation | | 8.3 | Plastic surgery changes | Face structure altered | Under evaluation | | 8.4 | Elderly with many wrinkles | Texture patterns differ from training | Already handled | | 8.5 | Very young children | Different facial proportions | Out of scope | | 8.6 | Unusual skin conditions | Texture anomalies | Already handled | | 8.7 | Tattoos on face | Ink patterns detected as manipulation | Already fixed - Working well | | 8.8 | Non-human image submitted | Animal, cartoon, object | Covered by Category 3 (no face rejection) | | 8.9 | Celebrity/public figure photo | Stock photo submission | Out of scope | | 8.10 | ID card photo submitted | Document photo, not selfie | Also comes under spoof attack - Use Unified API for liveness | *** ### Category 9: Document + Face Verification (Video KYC) | # | Failure Scenario | Why API Fails | Action Steps | | ---- | ------------------------------------ | -------------------------------------------------- | ----------------------------------------------------- | | 9.1 | ID card held in frame with face | Multiple faces detected (live + document photo) | Covered by Category 3 (multiple face handling) | | 9.2 | Document photo confused as live face | API analyzes wrong face | Covered by Category 3 (face size - largest face used) | | 9.3 | Document photo flagged as spoof | ID photo detected as "photo attack" false positive | Out of scope - Use Unified API | | 9.4 | Hand/fingers occluding face | User holds document near face | Covered by Category 3 (partial face) | | 9.5 | Document glare/reflection | Shiny ID card surface creates hotspots | Out of scope - Document verification | | 9.6 | Document out of focus | User focuses on face, document blurry | Out of scope - Document verification | | 9.7 | Wrong document shown | User shows incorrect ID type | Out of scope - Document verification | | 9.8 | Document tilted/angled | Cannot read document details | Out of scope - Document verification | | 9.9 | Fake/edited document | Tampered ID card | Out of scope - Document verification | | 9.10 | Face-document mismatch | Different person on ID vs live face | Out of scope - Face matching API | *** ### Summary | Category | Status | | --------------------------------------- | ---------------------------------------------------------- | | Category 1: Image Quality & Lighting | 1.1-1.6 by 24th Feb; Quality errors (1.8, 1.9) by 28th Feb | | Category 2: Liveness/Spoof | Out of scope - Use Unified API | | Category 3: Face Detection | Partially implemented, update before 24th Feb | | Category 4: Face Swap Detection | Mostly fixed, continuous improvement | | Category 5: AI Generated Face Detection | Already fixed - 200+ models supported | | Category 6: Technical/Input | Mostly implemented | | Category 7: Video KYC | Covered by other categories | | Category 8: Edge Cases | Under evaluation | | Category 9: Document + Face | Covered by other categories | --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://neuraldefend.gitbook.io/neural-defend/unified-api/face-swap-and-deepfake-detection-api-guardrails-and-scenarios.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.