iOS 使用GPUImage简单实现绿幕功能

一.前言

本文在上一篇文章的基础上，简单实现一下绿幕功能。原理是将原视频中的绿色部分纹理替换成图片或视频帧的纹理。

二.滤镜链

滤镜链示意图

三.关键代码

1.图片和视频作为滤镜链的输入源。

- (void)showGreenScreenWithImage:(UIImage *)image{
    self.inputPicture = [[GPUImagePicture alloc] initWithImage:image smoothlyScaleOutput:YES];
    self.greenScreenType = HYRendererGreenScreenTypePicture;
}

- (void)showGreenScreenWithVideoUrl:(NSURL *)videoUrl{
    self.gpuMovie = [[GPUImageMovie alloc] initWithURL:videoUrl];
    self.gpuMovie.playAtActualSpeed = YES;
    self.gpuMovie.shouldRepeat = YES;
    self.greenScreenType = HYRendererGreenScreenTypeVideo;
}

2.设置三种状态的滤镜链。

- (void)setGreenScreenType:(HYRendererGreenScreenType)greenScreenType
{
    _greenScreenType = greenScreenType;
    
    [self.dataInput removeAllTargets];
    [self.filter removeAllTargets];
    [self.inputPicture removeAllTargets];
    [self.gpuMovie removeAllTargets];
    
    switch (greenScreenType) {
        case HYRendererGreenScreenTypeNone:{
            [self.dataInput addTarget:self.filter];
            [self.filter addTarget:self.textureOutput];
        }break;
        case HYRendererGreenScreenTypePicture:{
            [self.dataInput addTarget:self.blendFilter];
            [self.inputPicture addTarget:self.blendFilter];
            [self.blendFilter addTarget:self.filter];
            [self.filter addTarget:self.textureOutput];
            runSynchronouslyOnVideoProcessingQueue(^{
                [self.gpuMovie cancelProcessing];
                [self.inputPicture processImage];
            });

        }break;
        case HYRendererGreenScreenTypeVideo:{
            [self.dataInput addTarget:self.blendFilter];
            [self.gpuMovie addTarget:self.blendFilter];
            [self.blendFilter addTarget:self.filter];
            [self.filter addTarget:self.textureOutput];
            runSynchronouslyOnVideoProcessingQueue(^{
                [self.gpuMovie startProcessing];
            });
        }break;
            
        default:
            break;
    }
}

3.清空绿幕。

- (void)cleanGreenScreen{
    self.greenScreenType = HYRendererGreenScreenTypeNone;
    if (self.inputPicture) {
        [self.inputPicture removeAllTargets];
        self.inputPicture = nil;
    }
    if (self.gpuMovie) {
        [self.gpuMovie removeAllTargets];
        [self.gpuMovie cancelProcessing];
        self.gpuMovie = nil;
    }
}

4.在self.textureOutput中的代理方法里解锁firstInputFramebuffer，防止内存泄露。放在代理方法里解锁的原因是FBO的操作在异步线程，其他地方不能保证lock和unlock一一对应，出现firstInputFramebuffer被过度释放导致的crash。

- (void)newFrameReadyFromTextureOutput:(GPUImageTextureOutput *)callbackTextureOutput{
    [self.textureOutput doneWithTexture];
}

5.GPUImageGreenScreenFilter继承自GPUImageTwoInputFilter，可接收两个输入源。片元着色器里包含着替换绿色背景的glsl代码，原理是计算rgb中绿色和红蓝色平均强度的差值，再送入smoothstep函数处理，再判断哪些使用原始纹理，哪些使用背景纹理。

NSString *const kGPUImageGreenScreenFragmentShaderString = SHADER_STRING
(
 varying highp vec2 textureCoordinate;
 varying highp vec2 textureCoordinate2;

 uniform sampler2D inputImageTexture;
 uniform sampler2D inputImageTexture2;
 
 void main()
 {
    lowp vec4 textureColor = texture2D(inputImageTexture, textureCoordinate);
    lowp vec4 textureColor2 = texture2D(inputImageTexture2, textureCoordinate2);
    //举例：视频中绿色纹素的 rgb = （0.294，0.976，0.173）
    lowp float rbAverage = textureColor.r * 0.5 + textureColor.b * 0.5;
    // rbAverage = 0.234
    lowp float gDelta = textureColor.g - rbAverage;
    //gDelta = 0.742
    textureColor.a = 1.0 - smoothstep(0.0, 0.25, gDelta);
    //根据平滑阶梯函数，smoothstep(0.0, 0.25, 0.742) = 1.0
    // textureColor.a = 0.0 
    if(textureColor.a < 0.5){
        //显示背景纹理
        gl_FragColor = textureColor2;
    }else{
        //显示原始纹理
        gl_FragColor = textureColor;
    }
 }
);

四.效果展示

原视频画面 添加图片背景
添加视频背景

五.存在的问题

1.反复切换图片背景和视频背景时，会出现FBO size为0的情况，待解决。

2.绿幕抠图的算法待优化（实际上把RGB转换成HSV来处理更合理)。无法应对有缺陷的绿色背景，比如褶皱，闪光等。视频中人物胸口位置的衣服也被抠掉了。

源码

Github:Demo地址
欢迎留言或私信探讨问题及Star，谢谢~