This page describes an old version of the Face Detection API, which was part of ML Kit for Firebase. Development of this API has been moved to the standalone ML Kit SDK, which you can use with or without Firebase.Learn more.

See Detect faces with ML Kit on iOS for the latest documentation.

You can use ML Kit to detect faces in images and video.

Before you begin

1. If you have not already added Firebase to your app, do so by following the steps in thegetting started guide.
2. Include the ML Kit libraries in your Podfile:

   pod 'Firebase/MLVision', '6.25.0'# If you want to detect face contours (landmark detection and classification# don't require this additional model):pod 'Firebase/MLVisionFaceModel', '6.25.0'

   After you install or update your project's Pods, be sure to open your Xcode project using its.xcworkspace.
3. In your app, import Firebase:

   Swift

   importFirebase

   Objective-C

   @importFirebase;

Input image guidelines

For ML Kit to accurately detect faces, input images must contain faces that are represented by sufficient pixel data. In general, each face you want to detect in an image should be at least 100x100 pixels. If you want to detect the contours of faces, ML Kit requires higher resolution input: each face should be at least 200x200 pixels.

If you are detecting faces in a real-time application, you might also want to consider the overall dimensions of the input images. Smaller images can be processed faster, so to reduce latency, capture images at lower resolutions (keeping in mind the above accuracy requirements) and ensure that the subject's face occupies as much of the image as possible. Also seeTips to improve real-time performance.

Poor image focus can hurt accuracy. If you aren't getting acceptable results, try asking the user to recapture the image.

The orientation of a face relative to the camera can also affect what facial features ML Kit detects. SeeFace Detection Concepts.

1. Configure the face detector

For example, build aVisionFaceDetectorOptionsobject like one of the following examples:

// High-accuracy landmark detection and face classificationletoptions=VisionFaceDetectorOptions()options.performanceMode=.accurateoptions.landmarkMode=.alloptions.classificationMode=.all// Real-time contour detection of multiple facesletoptions=VisionFaceDetectorOptions()options.contourMode=.all

// High-accuracy landmark detection and face classificationFIRVisionFaceDetectorOptions*options=[[FIRVisionFaceDetectorOptionsalloc]init];options.performanceMode=FIRVisionFaceDetectorPerformanceModeAccurate;options.landmarkMode=FIRVisionFaceDetectorLandmarkModeAll;options.classificationMode=FIRVisionFaceDetectorClassificationModeAll;// Real-time contour detection of multiple facesFIRVisionFaceDetectorOptions*options=[[FIRVisionFaceDetectorOptionsalloc]init];options.contourMode=FIRVisionFaceDetectorContourModeAll;

2. Run the face detector

1. Get an instance ofVisionFaceDetector:

   Swift

   lazyvarvision=Vision.vision()letfaceDetector=vision.faceDetector(options:options)

   Objective-C

   FIRVision*vision=[FIRVisionvision];FIRVisionFaceDetector*faceDetector=[visionfaceDetector];// Or, to change the default settings:// FIRVisionFaceDetector *faceDetector =//     [vision faceDetectorWithOptions:options];

2. Create aVisionImage object using aUIImage or aCMSampleBufferRef.

   To use aUIImage:

   1. If necessary, rotate the image so that itsimageOrientation property is.up.
   2. Create aVisionImage object using the correctly-rotatedUIImage. Do not specify any rotation metadata—the default value,.topLeft, must be used.

      Swift

      letimage=VisionImage(image:uiImage)

      Objective-C

      FIRVisionImage*image=[[FIRVisionImagealloc]initWithImage:uiImage];

   To use aCMSampleBufferRef:

   1. Create aVisionImageMetadata object that specifies the orientation of the image data contained in theCMSampleBufferRef buffer.

      To get the image orientation:

      Swift

      funcimageOrientation(deviceOrientation:UIDeviceOrientation,cameraPosition:AVCaptureDevice.Position)->VisionDetectorImageOrientation{switchdeviceOrientation{case.portrait:returncameraPosition==.front?.leftTop:.rightTopcase.landscapeLeft:returncameraPosition==.front?.bottomLeft:.topLeftcase.portraitUpsideDown:returncameraPosition==.front?.rightBottom:.leftBottomcase.landscapeRight:returncameraPosition==.front?.topRight:.bottomRightcase.faceDown,.faceUp,.unknown:return.leftTop}}

      Objective-C

      -(FIRVisionDetectorImageOrientation)imageOrientationFromDeviceOrientation:(UIDeviceOrientation)deviceOrientationcameraPosition:(AVCaptureDevicePosition)cameraPosition{switch(deviceOrientation){caseUIDeviceOrientationPortrait:if(cameraPosition==AVCaptureDevicePositionFront){returnFIRVisionDetectorImageOrientationLeftTop;}else{returnFIRVisionDetectorImageOrientationRightTop;}caseUIDeviceOrientationLandscapeLeft:if(cameraPosition==AVCaptureDevicePositionFront){returnFIRVisionDetectorImageOrientationBottomLeft;}else{returnFIRVisionDetectorImageOrientationTopLeft;}caseUIDeviceOrientationPortraitUpsideDown:if(cameraPosition==AVCaptureDevicePositionFront){returnFIRVisionDetectorImageOrientationRightBottom;}else{returnFIRVisionDetectorImageOrientationLeftBottom;}caseUIDeviceOrientationLandscapeRight:if(cameraPosition==AVCaptureDevicePositionFront){returnFIRVisionDetectorImageOrientationTopRight;}else{returnFIRVisionDetectorImageOrientationBottomRight;}default:returnFIRVisionDetectorImageOrientationTopLeft;}}

      Then, create the metadata object:

      Swift

      letcameraPosition=AVCaptureDevice.Position.back// Set to the capture device you used.letmetadata=VisionImageMetadata()metadata.orientation=imageOrientation(deviceOrientation:UIDevice.current.orientation,cameraPosition:cameraPosition)

      Objective-C

      FIRVisionImageMetadata*metadata=[[FIRVisionImageMetadataalloc]init];AVCaptureDevicePositioncameraPosition=AVCaptureDevicePositionBack;// Set to the capture device you used.metadata.orientation=[selfimageOrientationFromDeviceOrientation:UIDevice.currentDevice.orientationcameraPosition:cameraPosition];

   2. Create aVisionImage object using theCMSampleBufferRef object and the rotation metadata:

      Swift

      letimage=VisionImage(buffer:sampleBuffer)image.metadata=metadata

      Objective-C

      FIRVisionImage*image=[[FIRVisionImagealloc]initWithBuffer:sampleBuffer];image.metadata=metadata;

3. Then, pass the image to thedetect(in:) method:

   Swift

   faceDetector.process(visionImage){faces,erroringuarderror==nil,letfaces=faces,!faces.isEmptyelse{// ...return}// Faces detected// ...}

   Objective-C

   [faceDetectordetectInImage:imagecompletion:^(NSArray<FIRVisionFace*>*faces,NSError*error){if(error!=nil){return;}elseif(faces!=nil){// Recognized faces}}];

3. Get information about detected faces

forfaceinfaces{letframe=face.frameifface.hasHeadEulerAngleY{letrotY=face.headEulerAngleY// Head is rotated to the right rotY degrees}ifface.hasHeadEulerAngleZ{letrotZ=face.headEulerAngleZ// Head is rotated upward rotZ degrees}// If landmark detection was enabled (mouth, ears, eyes, cheeks, and// nose available):ifletleftEye=face.landmark(ofType:.leftEye){letleftEyePosition=leftEye.position}// If contour detection was enabled:ifletleftEyeContour=face.contour(ofType:.leftEye){letleftEyePoints=leftEyeContour.points}ifletupperLipBottomContour=face.contour(ofType:.upperLipBottom){letupperLipBottomPoints=upperLipBottomContour.points}// If classification was enabled:ifface.hasSmilingProbability{letsmileProb=face.smilingProbability}ifface.hasRightEyeOpenProbability{letrightEyeOpenProb=face.rightEyeOpenProbability}// If face tracking was enabled:ifface.hasTrackingID{lettrackingId=face.trackingID}}

for(FIRVisionFace*faceinfaces){// Boundaries of face in imageCGRectframe=face.frame;if(face.hasHeadEulerAngleY){CGFloatrotY=face.headEulerAngleY;// Head is rotated to the right rotY degrees}if(face.hasHeadEulerAngleZ){CGFloatrotZ=face.headEulerAngleZ;// Head is tilted sideways rotZ degrees}// If landmark detection was enabled (mouth, ears, eyes, cheeks, and// nose available):FIRVisionFaceLandmark*leftEar=[facelandmarkOfType:FIRFaceLandmarkTypeLeftEar];if(leftEar!=nil){FIRVisionPoint*leftEarPosition=leftEar.position;}// If contour detection was enabled:FIRVisionFaceContour*upperLipBottomContour=[facecontourOfType:FIRFaceContourTypeUpperLipBottom];if(upperLipBottomContour!=nil){NSArray<FIRVisionPoint*>*upperLipBottomPoints=upperLipBottomContour.points;if(upperLipBottomPoints.count>0){NSLog("Detected the bottom contour of the subject's upper lip.")}}// If classification was enabled:if(face.hasSmilingProbability){CGFloatsmileProb=face.smilingProbability;}if(face.hasRightEyeOpenProbability){CGFloatrightEyeOpenProb=face.rightEyeOpenProbability;}// If face tracking was enabled:if(face.hasTrackingID){NSIntegertrackingID=face.trackingID;}}

Example of face contours

When you have face contour detection enabled, you get a list of points for each facial feature that was detected. These points represent the shape of the feature. See theFace Detection Concepts Overview for details about how contours are represented.

The following image illustrates how these points map to a face (click the image to enlarge):