Way-1 Face Recognition with alignment + clip

使用的是dlib的shape_predictor_68_face_landmarks.dat, 在http://dlib.net/files/下载。

Untitled

predictor_model = 'shape_predictor_68_face_landmarks.dat'
detector = dlib.get_frontal_face_detector()# dlib人脸检测器
predictor = dlib.shape_predictor(predictor_model)

关于人脸识别和截取 Face Recognition and Crop

# get the image list based on the directory path
img_hasface_dirpath =  "/Users/zhaonan/UAL-CCI/07-MScAdvancedProject/pythonproject/faceOutput/original-hasface-1"
img_hasface_path=os.path.join(img_hasface_dirpath) # 使用os.path模块的join方法生成路径
delete_dot_ds_store(img_hasface_dirpath) # delete the .DS_Store file
img_hasface_list=os.listdir(img_hasface_path)

# iterate through the image list
for i in img_hasface_list: 
    img_hasface_name = os.path.splitext(os.path.basename(os.path.join(img_hasface_path,i)))[0] # get the name(number)
    img_hasface = cv2.imread(os.path.join(img_hasface_path,i),cv2.IMREAD_COLOR) # 调用cv2.imread读入图片，读入格式为IMREAD_COLOR
    img_hasface = cv2.cvtColor(img_hasface, cv2.COLOR_BGR2RGB) # convert color
    rects = detector(img_hasface, 0) # using detector get face count / 人脸数 rects
    faces = dlib.full_object_detections() # faces存储full_object_detection对象
    for i in range(len(rects)):
        faces.append(predictor(img,rects[i]))
    face_images = dlib.get_face_chips(img_hasface, faces, size=512) # clip the face. 进行裁剪
		# 将裁减过的face储存到一文件夹中，并且根据命名，后续可以整理出没有被截出人脸的图片
    ind3 = 0
    for f in face_images:
        cv_bgr_img = cv2.cvtColor(f, cv2.COLOR_RGB2BGR)
        img_face_name= img_hasface_name+'_' + str(ind3)+'_clip.jpg'
        ind3 = ind3 + 1
        cv2.imwrite('faceOutput/clip-1/'+img_face_name, cv_bgr_img)

关于人脸对齐 Facial Alignment

计算两眼连线与水平线的夹角，然后构建仿射矩阵-通过角度得到对应的旋转矩阵。对图片进行相应的变换。

def single_face_alignment(face, landmarks):
    eye_center = ((landmarks[36, 0] + landmarks[45, 0]) * 1. / 2,  # 计算两眼的中心坐标
                  (landmarks[36, 1] + landmarks[45, 1]) * 1. / 2)
    dx = (landmarks[45, 0] - landmarks[36, 0])  # note: right - right
    dy = (landmarks[45, 1] - landmarks[36, 1])

    angle = math.atan2(dy, dx) * 180. / math.pi  # 计算角度
    RotateMatrix = cv2.getRotationMatrix2D(eye_center, angle, scale=1)  # 计算仿射矩阵
    align_face = cv2.warpAffine(face, RotateMatrix, (face.shape[0], face.shape[1]))  # 进行放射变换，即旋转
    return align_face

因为针对已经截取出的人脸，再次执行脸部识别时，也有部分人脸没有被识别出，所以需要将没有识别对齐的人脸单独修改命名。

landmarks = get_landmarks(face)
if type(landmarks) is np.matrix:
    print("get landmarks!")
    aligned_img = single_face_alignment(face, landmarks)
    img_align_name= clipname+'_align.jpg'
    cv2.imwrite('faceOutput/clip-align-1/'+img_align_name, aligned_img) # 写入图片
else :
    os.rename(CLIP_DATADIR+'/'+filename, CLIP_DATADIR+'/'+"notdetected_"+filename) # 执行一边即可
    print("didn't detect!")

Problem

感觉用dlib的不大准，不全。The face detection is not reliable with dlib. It failed to detect some faces, leading to the incompletion for preparing the dataset.

1-检测到没有脸的图片里其实也有脸。The images that are not detected faces, they have faces.

2-检测到有脸的，截取也不全。The dnnlib clip function failed to clip all the faces from one image.