音频编码
2018-05-22 本文已影响0人
Style_mao
音频基础知识
PCM格式
pcm是经过话筒录音后直接得到的未经压缩的数据流
数据大小=采样频率采样位数声道*秒数/8
采样频率一般是44k，位数一般是8位或者16位，声道一般是单声道或者双声道
pcm属于编码格式，就是一串由多个样本值组成的数据流，本身没有任何头信息或者帧的概念。如果不是音频的录制者，光凭一段PCM数据，是没有办法知道它的采样率等信息的。
AAC格式
初步了解，AAC文件可以没有文件头，全部由帧序列组成，每个帧由帧头和数据部分组成。帧头包含采样率、声道数、帧长度等，有点类似MP3格式。
AAC编码
初始化编码转换器
-(BOOL)createAudioConvert{
if(m_converter != nil){
return TRUE;
}
AudioStreamBasicDescription inputFormat = {0};
inputFormat.mSampleRate = _configuration.audioSampleRate;
inputFormat.mFormatID = kAudioFormatLinearPCM;
inputFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagsNativeEndian | kAudioFormatFlagIsPacked;
inputFormat.mChannelsPerFrame = (UInt32)_configuration.numberOfChannels;
inputFormat.mFramesPerPacket = 1;
inputFormat.mBitsPerChannel = 16;
inputFormat.mBytesPerFrame = inputFormat.mBitsPerChannel / 8 * inputFormat.mChannelsPerFrame;
inputFormat.mBytesPerPacket = inputFormat.mBytesPerFrame * inputFormat.mFramesPerPacket;
AudioStreamBasicDescription outputFormat; // 这里开始是输出音频格式
memset(&outputFormat, 0, sizeof(outputFormat));
outputFormat.mSampleRate = inputFormat.mSampleRate; // 采样率保持一致
outputFormat.mFormatID = kAudioFormatMPEG4AAC; // AAC编码 kAudioFormatMPEG4AAC kAudioFormatMPEG4AAC_HE_V2
outputFormat.mChannelsPerFrame = (UInt32)_configuration.numberOfChannels;;
outputFormat.mFramesPerPacket = 1024; // AAC一帧是1024个字节
const OSType subtype = kAudioFormatMPEG4AAC;
AudioClassDescription requestedCodecs[2] = {
{
kAudioEncoderComponentType,
subtype,
kAppleSoftwareAudioCodecManufacturer
},
{
kAudioEncoderComponentType,
subtype,
kAppleHardwareAudioCodecManufacturer
}
};
OSStatus result = AudioConverterNewSpecific(&inputFormat, &outputFormat, 2, requestedCodecs, &m_converter);
if(result != noErr) return NO;
return YES;
}
编码转换
char *aacBuf;
if(!aacBuf){
aacBuf = malloc(inBufferList.mBuffers[0].mDataByteSize);
}
// 初始化一个输出缓冲列表
AudioBufferList outBufferList;
outBufferList.mNumberBuffers = 1;
outBufferList.mBuffers[0].mNumberChannels = inBufferList.mBuffers[0].mNumberChannels;
outBufferList.mBuffers[0].mDataByteSize = inBufferList.mBuffers[0].mDataByteSize; // 设置缓冲区大小
outBufferList.mBuffers[0].mData = aacBuf; // 设置AAC缓冲区 UInt32
outputDataPacketSize = 1;
if (AudioConverterFillComplexBuffer(m_converter, inputDataProc, &inBufferList, &outputDataPacketSize, &outBufferList, NULL) != noErr){
return;
}
AudioFrame *audioFrame = [AudioFrame new];
audioFrame.timestamp = timeStamp;
audioFrame.data = [NSData dataWithBytes:aacBuf length:outBufferList.mBuffers[0].mDataByteSize];
char exeData[2];
exeData[0] = _configuration.asc[0];
exeData[1] = _configuration.asc[1];
audioFrame.audioInfo =[NSData dataWithBytes:exeData length:2];
在Ios中，实现打开和捕获麦克风大多是用的AVCaptureSession这个组件来实现的，它可以不仅可以实现音频捕获，还可以实现视频的捕获。
针对打开麦克风和捕获音频的代码，简单的整理了一下：
首先，我们需要定义一个AVCaptureSession类型的变量，它是架起在麦克风设备和数据输出上的一座桥，通过它可以方便的得到麦克风的实时原始数据。
AVCaptureSession *m_capture;
同时，定义一组函数，用来打开和关闭麦克风；为了能使数据顺利的导出，你还需要实现AVCaptureAudioDataOutputSampleBufferDelegate这个协议
-(void)open;
-(void)close;
-(BOOL)isOpen;
下面我们将分别实现上述参数函数，来完成数据的捕获
-(void)open {
NSError *error;
m_capture = [[AVCaptureSession alloc]init];
AVCaptureDevice *audioDev = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeAudio];
if (audioDev == nil)
{
CKPrint("Couldn't create audio capture device");
return ;
}
// create mic device
AVCaptureDeviceInput *audioIn = [AVCaptureDeviceInput deviceInputWithDevice:audioDev error:&error];
if (error != nil)
{
CKPrint("Couldn't create audio input");
return ;
}
// add mic device in capture object
if ([m_capture canAddInput:audioIn] == NO)
{
CKPrint("Couldn't add audio input")
return ;
}
[m_capture addInput:audioIn];
// export audio data
AVCaptureAudioDataOutput *audioOutput = [[AVCaptureAudioDataOutput alloc] init];
[audioOutput setSampleBufferDelegate:self queue:dispatch_get_main_queue()];
if ([m_capture canAddOutput:audioOutput] == NO)
{
CKPrint("Couldn't add audio output");
return ;
}
[m_capture addOutput:audioOutput];
[audioOutput connectionWithMediaType:AVMediaTypeAudio];
[m_capture startRunning];
return ;
}
-(void)close {
if (m_capture != nil && [m_capture isRunning])
{
[m_capture stopRunning];
}
return;
}
-(BOOL)isOpen {
if (m_capture == nil)
{
return NO;
}
return [m_capture isRunning];
}
通过上面三个函数，即可完成所有麦克风捕获的准备工作，现在我们就等着数据主动送上门了。要想数据主动送上门，我们还需要实现一个协议接口：
- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {
char szBuf[4096];
int nSize = sizeof(szBuf);
#if SUPPORT_AAC_ENCODER
if ([self encoderAAC:sampleBuffer aacData:szBuf aacLen:&nSize] == YES)
{
[g_pViewController sendAudioData:szBuf len:nSize channel:0];
}
#else //#if SUPPORT_AAC_ENCODER
AudioStreamBasicDescription outputFormat = *(CMAudioFormatDescriptionGetStreamBasicDescription(CMSampleBufferGetFormatDescription(sampleBuffer)));
nSize = CMSampleBufferGetTotalSampleSize(sampleBuffer);
CMBlockBufferRef databuf = CMSampleBufferGetDataBuffer(sampleBuffer);
if (CMBlockBufferCopyDataBytes(databuf, 0, nSize, szBuf) == kCMBlockBufferNoErr)
{
[g_pViewController sendAudioData:szBuf len:nSize channel:outputFormat.mChannelsPerFrame];
}
#endif
}
到这里，我们的工作也就差不多做完了，所捕获出来的数据是原始的PCM数据。
当然，由于PCM数据本身比较大，不利于网络传输，所以如果需要进行网络传输时，就需要对数据进行编码；Ios系统本身支持多种音频编码格式，这里我们就以AAC为例来实现一个PCM编码AAC的函数。
在Ios系统中，PCM编码AAC的例子，在网上也是一找一大片，但是大多都是不太完整的，而且相当一部分都是E文的，对于某些童鞋而言，这些都是深恶痛绝的。我这里就做做好人，把它们整理了一下，写成了一个函数，方便使用。
在编码前，需要先创建一个编码转换对象
AVAudioConverterRef m_converter;
#if SUPPORT_AAC_ENCODER
-(BOOL)createAudioConvert:(CMSampleBufferRef)sampleBuffer { //根据输入样本初始化一个编码转换器
if (m_converter != nil)
{
return TRUE;
}
AudioStreamBasicDescription inputFormat = *(CMAudioFormatDescriptionGetStreamBasicDescription(CMSampleBufferGetFormatDescription(sampleBuffer))); // 输入音频格式
AudioStreamBasicDescription outputFormat; // 这里开始是输出音频格式
memset(&outputFormat, 0, sizeof(outputFormat));
outputFormat.mSampleRate = inputFormat.mSampleRate; // 采样率保持一致
outputFormat.mFormatID = kAudioFormatMPEG4AAC; // AAC编码
outputFormat.mChannelsPerFrame = 2;
outputFormat.mFramesPerPacket = 1024; // AAC一帧是1024个字节
AudioClassDescription *desc = [self getAudioClassDescriptionWithType:kAudioFormatMPEG4AAC fromManufacturer:kAppleSoftwareAudioCodecManufacturer];
if (AudioConverterNewSpecific(&inputFormat, &outputFormat, 1, desc, &m_converter) != noErr)
{
CKPrint(@"AudioConverterNewSpecific failed");
return NO;
}
return YES;
}
-(BOOL)encoderAAC:(CMSampleBufferRef)sampleBuffer aacData:(char*)aacData aacLen:(int*)aacLen { // 编码PCM成AAC
if ([self createAudioConvert:sampleBuffer] != YES)
{
return NO;
}
CMBlockBufferRef blockBuffer = nil;
AudioBufferList inBufferList;
if (CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer, NULL, &inBufferList, sizeof(inBufferList), NULL, NULL, 0, &blockBuffer) != noErr)
{
CKPrint(@"CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer failed");
return NO;
}
// 初始化一个输出缓冲列表
AudioBufferList outBufferList;
outBufferList.mNumberBuffers = 1;
outBufferList.mBuffers[0].mNumberChannels = 2;
outBufferList.mBuffers[0].mDataByteSize = *aacLen; // 设置缓冲区大小
outBufferList.mBuffers[0].mData = aacData; // 设置AAC缓冲区
UInt32 outputDataPacketSize = 1;
if (AudioConverterFillComplexBuffer(m_converter, inputDataProc, &inBufferList, &outputDataPacketSize, &outBufferList, NULL) != noErr)
{
CKPrint(@"AudioConverterFillComplexBuffer failed");
return NO;
}
*aacLen = outBufferList.mBuffers[0].mDataByteSize; //设置编码后的AAC大小
CFRelease(blockBuffer);
return YES;
}
-(AudioClassDescription*)getAudioClassDescriptionWithType:(UInt32)type fromManufacturer:(UInt32)manufacturer { // 获得相应的编码器
static AudioClassDescription audioDesc;
UInt32 encoderSpecifier = type, size = 0;
OSStatus status;
memset(&audioDesc, 0, sizeof(audioDesc));
status = AudioFormatGetPropertyInfo(kAudioFormatProperty_Encoders, sizeof(encoderSpecifier), &encoderSpecifier, &size);
if (status)
{
return nil;
}
uint32_t count = size / sizeof(AudioClassDescription);
AudioClassDescription descs[count];
status = AudioFormatGetProperty(kAudioFormatProperty_Encoders, sizeof(encoderSpecifier), &encoderSpecifier, &size, descs);
for (uint32_t i = 0; i < count; i++)
{
if ((type == descs[i].mSubType) && (manufacturer == descs[i].mManufacturer))
{
memcpy(&audioDesc, &descs[i], sizeof(audioDesc));
break;
}
}
return &audioDesc;
}
OSStatus inputDataProc(AudioConverterRef inConverter, UInt32 *ioNumberDataPackets, AudioBufferList *ioData,AudioStreamPacketDescription **outDataPacketDescription, voidvoid *inUserData) { //<span style="font-family: Arial, Helvetica, sans-serif;">AudioConverterFillComplexBuffer 编码过程中，会要求这个函数来填充输入数据，也就是原始PCM数据</span>
AudioBufferList bufferList = *(AudioBufferList*)inUserData;
ioData->mBuffers[0].mNumberChannels = 1;
ioData->mBuffers[0].mData = bufferList.mBuffers[0].mData;
ioData->mBuffers[0].mDataByteSize = bufferList.mBuffers[0].mDataByteSize;
return noErr;
}
#endif
好了，世界是那么美好，一个函数即可所有的事情搞定了。当你需要进行AAC编码时，调用encoderAAC这个函数就可以了（在上面有完整的代码）
char szBuf[4096];
int nSize = sizeof(szBuf);
if ([self encoderAAC:sampleBuffer aacData:szBuf aacLen:&nSize] == YES)
{
// do something
}
