Metal与图形渲染十：动态滤镜
2021-10-18 本文已影响0人
肠粉白粥_Hoben
零. 前言
提起图形渲染技术，大家的第一时间应该是各种各样的滤镜，而几年前抖音能得以迅速扩张，能整出各种花活的滤镜自然是功臣之一，今天来用Metal做几个滤镜玩玩~
一. 缩放
缩放滤镜是通过顶点着色器实现的，原理是随着时间推移，将顶点坐标放大/缩小，使用sin函数得到平滑的效果，我们将时间戳等分为1000小份，用sin函数将放大系数限制在[1, 1.1]区间：
constant int smooth = 1000;
vertex SingleInputVertexIO
zoomVertex(const device float2 *position [[ buffer(0) ]],
const device float ¤tTime [[ buffer(1) ]],
const device float2 *textureCoord [[ buffer(2) ]],
uint vid [[ vertex_id ]]) {
SingleInputVertexIO out;
long tick = ((long)(currentTime * smooth)) % smooth;
float freq = 1.0 / smooth * tick;
float maxAmplitude = 0.1;
// 将顶点坐标放大
float amplitude = 1.0 + maxAmplitude * max(sin(freq * M_PI_F * 2), 0.0);
float2 currentPos = amplitude * position[vid];
out.position = float4(currentPos, 0, 1);
out.textureCoordinate = textureCoord[vid];
return out;
}
二. 旋转
旋转效果同样也修改到了顶点着色器，原理同样是根据时间，修改顶点x、y坐标，只不过这次x用sin函数，y用cos函数了。
vertex SingleInputVertexIO
rotateVertex(const device float2 *position [[ buffer(0) ]],
const device float ¤tTime [[ buffer(1) ]],
const device float2 *textureCoord [[ buffer(2) ]],
uint vid [[ vertex_id ]]) {
SingleInputVertexIO out;
float speed = 0.5;
long tick = ((long)((currentTime * speed) * smooth)) % smooth;
float freq = 1.0 / smooth * tick;
float maxAmplitude = 0.1;
float amplitudeX = maxAmplitude * sin(freq * M_PI_F * 2);
float amplitudeY = maxAmplitude * cos(freq * M_PI_F * 2);
float2 currentPos = position[vid] + float2(amplitudeX, amplitudeY);
out.position = float4(currentPos, 0, 1);
out.textureCoordinate = textureCoord[vid];
return out;
}
三. 灵魂出窍
灵魂出窍效果的原理是将放大后的纹理和原纹理进行颜色的混合，随着时间推移，纹理会越来越大，但是透明度会越来越小，通过修改片段着色器实现：
fragment float4
soulOutFragment(SingleInputVertexIO input [[ stage_in ]],
texture2d <float> texture [[ texture(0) ]],
constant float ¤tTime [[ buffer(0) ]]) {
long tick = ((long)(currentTime * smooth)) % smooth;
// [0, 1], progress越大，图片越大，但越透明
float progress = 1.0 / smooth * tick;
float maxAlpha = 0.4;
float maxScale = 1.8;
float alpha = maxAlpha * (1.0 - progress);
float scale = 1.0 + (maxScale - 1.0) * progress;
float2 textureCoor = input.textureCoordinate;
// scale越大，纹理采样坐标越靠近中心点，达到放大图像的效果
float2 maskCoor = float2(0.5, 0.5) + (textureCoor - 0.5) / scale;
constexpr sampler textureSampler;
float4 maskColor = texture.sample(textureSampler, maskCoor);
float4 originColor = texture.sample(textureSampler, textureCoor);
return mix(originColor, maskColor, alpha);
}
四. 颜色抖动
颜色抖动的原理是将原纹理的r值和g值分别向左上、右下偏移，而b值、a值则取原纹理的值，通过片段着色器实现：
fragment float4
colorShakeFragment(SingleInputVertexIO input [[ stage_in ]],
texture2d <float> texture [[ texture(0) ]],
constant float ¤tTime [[ buffer(0) ]]) {
long tick = ((long)(currentTime * smooth)) % smooth;
float freq = 1.0 / smooth * tick;
float maxScale = 0.01;
float2 textureCoor = input.textureCoordinate;
float2 offset = maxScale * max(sin(freq * M_PI_F * 2), 0.0);
constexpr sampler textureSampler;
float maskColorR = texture.sample(textureSampler, textureCoor - offset).r;
float maskColorG = texture.sample(textureSampler, textureCoor + offset).g;
float4 originColor = texture.sample(textureSampler, textureCoor);
return float4(maskColorR, maskColorG, originColor.ba);
}
五. 毛刺效果
毛刺效果的原理是，对于纹理的同一行像素，让其随机向左右偏移，但是为了让原图像能被辨别出来，采取了少量像素大偏移、大量像素小偏移的策略，为了有色差的感觉，再将r、g、b分开偏移计算。
值得一提的是，为了获取随机数，使用了fract(sin(x) * 43758.5453123)这个奇怪的公式，但这串奇怪的数字看起来好经典的样子，能得到电视雪花屏的随机效果。
// 获得一个[0, 1]的随机数，https://xiaoiver.github.io/coding/2018/08/01/%E5%99%AA%E5%A3%B0%E7%9A%84%E8%89%BA%E6%9C%AF.html
float rand(float x) {
return fract(sin(x) * 43758.5453123);
}
fragment float4
jitterFragment(SingleInputVertexIO input [[ stage_in ]],
texture2d<float> texture [[ texture(0) ]],
constant float ¤tTime [[ buffer(0) ]]) {
long tick = ((long)(currentTime * smooth)) % smooth;
float freq = 1.0 / smooth * tick;
// [0, 1], 振幅
float amplitude = max(sin(freq * M_PI_F * 2), 0.0);
// 最大抖动
float maxJitter = 0.06;
float3 rgbOffset = float3(0.01, 0.02, -0.03) * amplitude;
float2 textureCoor = input.textureCoordinate;
// [-1, 1]的随机像素偏移
float jitter = rand(textureCoor.y) * 2 - 1.0;
bool needOffset = abs(jitter) < maxJitter * amplitude;
// 根据x坐标和needOffset计算x撕裂
// 当needOffset为true，撕裂大
// 当needOffset为false，撕裂小
// 绝大部分的行会撕裂小，少量的会撕裂大
float textureX = textureCoor.x + (needOffset ? jitter : jitter * amplitude * 0.006);
float2 maskCoorR = float2(textureX + rgbOffset.r, textureCoor.y);
float2 maskCoorG = float2(textureX + rgbOffset.g, textureCoor.y);
float2 maskCoorB = float2(textureX + rgbOffset.b, textureCoor.y);
constexpr sampler textureSampler;
float maskR = texture.sample(textureSampler, maskCoorR).r;
float maskG = texture.sample(textureSampler, maskCoorG).g;
float maskB = texture.sample(textureSampler, maskCoorB).b;
float4 originColor = texture.sample(textureSampler, textureCoor);
return float4(maskR, maskG, maskB, originColor.a);
}
六. 闪白效果
闪白效果的原理是随着时间推移，将白色色值和纹理色值进行不同程度的混合，比较简单：
fragment float4
flashFragment(SingleInputVertexIO input [[ stage_in ]],
texture2d <float> texture [[ texture(0) ]],
constant float ¤tTime [[ buffer(0) ]]) {
long tick = ((long)(currentTime * smooth)) % smooth;
float freq = 1.0 / smooth * tick;
float progress = max(sin(freq * M_PI_F * 2), 0.0);
constexpr sampler textureSampler;
float4 originColor = texture.sample(textureSampler, input.textureCoordinate);
float4 whiteColor = float4(1.0, 1.0, 1.0, 1.0);
return mix(originColor, whiteColor, progress);
}
七. 滤镜的叠加和删除
滤镜链的初始状态为Picture => RenderView
当需要添加链时，修改结构为Picture => 原有Filter =>... => newFilter => RenderView，将需要添加的Filter指向RenderView，再将原指向RenderView的结构指向新增的Filter。
当需要删除链时，修改结构为Picture => 原有Filter => ... => RenderView，将原指向Filter的结构指向Filter的下一级。
其实就是链表的插入和删除，哈哈
- (void)changeTargetStatus:(HobenMetalFilter *)target insert:(BOOL)insert {
if (![self.picture.targets[0] isKindOfClass:[HobenMetalFilter class]]) {
if (insert) {
[self.picture removeTarget:self.renderView];
[self.picture addTarget:target];
[target addTarget:self.renderView];
}
return;
}
HobenMetalFilter *cur = (HobenMetalFilter *)self.picture.targets[0];
HobenMetalOutput *pre = self.picture;
while ([cur.targets[0] isKindOfClass:[HobenMetalFilter class]] && ![cur isEqual:target]) {
pre = cur;
cur = (HobenMetalFilter *)cur.targets[0];
}
if (insert) {
// 插入
[pre removeTarget:cur];
[pre addTarget:target];
[target addTarget:cur];
} else {
// 删除
[pre removeTarget:target];
[pre addTarget:cur.targets[0]];
[target removeAllTargets];
}
}
最后在每一帧调用的时候都给Filter们传入当前的时间戳就可以啦~
- (void)drawInMTKView:(MTKView *)view {
float currentTime = [[NSDate date] timeIntervalSince1970] - self.startTime;
[[self filters] makeObjectsPerformSelector:@selector(setCurrentTime:) withObject:@(currentTime)];
[self.picture processImage];
}
八. 总结
酷炫的滤镜的产生，离不开顶点着色器和片段着色器，当我们需要对纹理本身进行一些位置的变化时，一般修改的是顶点着色器（当然，将片段着色器的纹理坐标放缩也能达到效果）；当我们需要对纹理一些颜色进行处理、或者是基于原有纹理进行一些颜色的混合时，一般修改的是片段着色器。
如果需要颜色分离，不妨考虑下将R、G、B三个通道的值赋予不同的offset，会有一些惊艳的效果产生。
当前已经实现的Metal滤镜链支持不同滤镜的叠加和删除，后续的优化方向可以考虑加入某些时间点使用哪些滤镜的操作，用户可以自行对滤镜进行时间上和效果上的调整，对原素材进行剪辑，会使得滤镜可玩性更高。
