版本记录

版本号	时间
V1.0	2017.12.29

前言

Core Audio使用专门的数据类型与音频流、复杂的缓冲区和audiovisual时间戳交互。接下来这几篇就对该框架进行详细解析。感兴趣的可以参考上面几篇文章。
1. Core Audio框架详细解析（一） —— 基本概要

Core Audio框架的地位

Core Audio 是iOS和MAC系统中的关于数字音频处理的基础，它是应用程序用来处理音频的一组软件框架，所有关于iOS音频开发的接口都是由Core Audio来提供或者经过它提供的接口来进行封装的。

其实一句话，它是任何iOS或者MAC系统音频处理框架的基础。

具体可以用官方文档的一张图表示。

接下来我们就一起分析一下。

---

High-Level 高级服务

这里的高级别服务，更加接近于顶层，基本上我们很多关于音频开发的工作在这一层就可以完成。

1. Audio Queue Services

它位于框架AudioToolbox中。

提供录制、播放、暂停、循环、和同步音频它自动采用必要的编解码器处理压缩的音频格式。

要在iOS设备上播放和录制音频，苹果推荐我们使用AVFoundation框架中的AVAudioPlayer和AVAudioRecorder类。虽然用法比较简单，但是不支持流式；这就意味着：在播放音频前，必须等到整个音频加载完成后，才能开始播放音频；录音时，也必须等到录音结束后，才能获取到录音数据。这给应用造成了很大的局限性。为了解决这个问题，我们就需要使用Audio Queue Services来播放和录制音频。感兴趣的可以看我前面写的几篇关于Audio Queue Services的文章。这里只是简单的给出录音和播放的原理图，具体原理和流程，看我前面写的那几篇，都有详细的介绍。

录音 播放

2. AVAudioPlayer

它位于框架AVFoundation中。

是专为IOS平台提供的基于Objective-C接口的音频播放类，可以支持iOS所支持的所有音频的播放，它主要支持以下音频格式。

AAC
AMR (Adaptive multi-Rate，一种语音格式)
ALAC (Apple lossless Audio Codec)
iLBC (internet Low Bitrate Codec，另一种语音格式)
IMA4 (IMA/ADPCM)
linearPCM (uncompressed)
u-law 和 a-law
MP3 (MPEG-Laudio Layer 3)

这个是纯OC的实现，特点就是调用简单，下面简单的看一下他的API。

#import <AVFoundation/AVBase.h>
#import <AVFoundation/AVAudioFormat.h>
#import <Foundation/Foundation.h>
#import <AVFAudio/AVAudioSettings.h>

#if (TARGET_OS_IPHONE && __has_include(<AVFoundation/AVAudioSession.h>))
#import <AVFAudio/AVAudioSession.h>
#endif // #if TARGET_OS_EMBEDDED

#import <Availability.h>

NS_ASSUME_NONNULL_BEGIN

@class NSData, NSURL, NSError;
#if (TARGET_OS_IPHONE && __has_include(<AVFoundation/AVAudioSession.h>))
@class AVAudioSessionChannelDescription;
#endif
@protocol AVAudioPlayerDelegate;

NS_CLASS_AVAILABLE(10_7, 2_2) __WATCHOS_AVAILABLE(3_0)
@interface AVAudioPlayer : NSObject {
@private
    id _impl;
}

/* For all of these init calls, if a return value of nil is given you can check outError to see what the problem was.
 If not nil, then the object is usable for playing
*/

/* all data must be in the form of an audio file understood by CoreAudio */
- (nullable instancetype)initWithContentsOfURL:(NSURL *)url error:(NSError **)outError;
- (nullable instancetype)initWithData:(NSData *)data error:(NSError **)outError;

/* The file type hint is a constant defined in AVMediaFormat.h whose value is a UTI for a file format. e.g. AVFileTypeAIFF. */
/* Sometimes the type of a file cannot be determined from the data, or it is actually corrupt. The file type hint tells the parser what kind of data to look for so that files which are not self identifying or possibly even corrupt can be successfully parsed. */
- (nullable instancetype)initWithContentsOfURL:(NSURL *)url fileTypeHint:(NSString * __nullable)utiString error:(NSError **)outError NS_AVAILABLE(10_9, 7_0);
- (nullable instancetype)initWithData:(NSData *)data fileTypeHint:(NSString * __nullable)utiString error:(NSError **)outError NS_AVAILABLE(10_9, 7_0);

/* transport control */
/* methods that return BOOL return YES on success and NO on failure. */
- (BOOL)prepareToPlay;  /* get ready to play the sound. happens automatically on play. */
- (BOOL)play;           /* sound is played asynchronously. */
- (BOOL)playAtTime:(NSTimeInterval)time NS_AVAILABLE(10_7, 4_0); /* play a sound some time in the future. time is an absolute time based on and greater than deviceCurrentTime. */
- (void)pause;          /* pauses playback, but remains ready to play. */
- (void)stop;           /* stops playback. no longer ready to play. */

/* properties */

@property(readonly, getter=isPlaying) BOOL playing; /* is it playing or not? */

@property(readonly) NSUInteger numberOfChannels;
@property(readonly) NSTimeInterval duration; /* the duration of the sound. */

#if !TARGET_OS_IPHONE
/* the UID of the current audio device (as a string) */
@property(copy, nullable) NSString *currentDevice API_AVAILABLE(macos(10.13));
#endif

/* the delegate will be sent messages from the AVAudioPlayerDelegate protocol */ 
@property(assign, nullable) id<AVAudioPlayerDelegate> delegate;

/* one of these properties will be non-nil based on the init... method used */
@property(readonly, nullable) NSURL *url; /* returns nil if object was not created with a URL */
@property(readonly, nullable) NSData *data; /* returns nil if object was not created with a data object */

@property float pan NS_AVAILABLE(10_7, 4_0); /* set panning. -1.0 is left, 0.0 is center, 1.0 is right. */
@property float volume; /* The volume for the sound. The nominal range is from 0.0 to 1.0. */
- (void)setVolume:(float)volume fadeDuration:(NSTimeInterval)duration API_AVAILABLE(macos(10.12), ios(10.0), watchos(3.0), tvos(10.0)); /* fade to a new volume over a duration */

@property BOOL enableRate NS_AVAILABLE(10_8, 5_0); /* You must set enableRate to YES for the rate property to take effect. You must set this before calling prepareToPlay. */
@property float rate NS_AVAILABLE(10_8, 5_0); /* See enableRate. The playback rate for the sound. 1.0 is normal, 0.5 is half speed, 2.0 is double speed. */


/*  If the sound is playing, currentTime is the offset into the sound of the current playback position.  
If the sound is not playing, currentTime is the offset into the sound where playing would start. */
@property NSTimeInterval currentTime;

/* returns the current time associated with the output device */
@property(readonly) NSTimeInterval deviceCurrentTime NS_AVAILABLE(10_7, 4_0);

/* "numberOfLoops" is the number of times that the sound will return to the beginning upon reaching the end. 
A value of zero means to play the sound just once.
A value of one will result in playing the sound twice, and so on..
Any negative number will loop indefinitely until stopped.
*/
@property NSInteger numberOfLoops;

/* settings */
@property(readonly) NSDictionary<NSString *, id> *settings NS_AVAILABLE(10_7, 4_0); /* returns a settings dictionary with keys as described in AVAudioSettings.h */

/* returns the format of the audio data */
@property(readonly) AVAudioFormat *format API_AVAILABLE(macos(10.12), ios(10.0), watchos(3.0), tvos(10.0));

/* metering */

@property(getter=isMeteringEnabled) BOOL meteringEnabled; /* turns level metering on or off. default is off. */

- (void)updateMeters; /* call to refresh meter values */

- (float)peakPowerForChannel:(NSUInteger)channelNumber; /* returns peak power in decibels for a given channel */
- (float)averagePowerForChannel:(NSUInteger)channelNumber; /* returns average power in decibels for a given channel */

#if (TARGET_OS_IPHONE && __has_include(<AVFoundation/AVAudioSession.h>))
/* The channels property lets you assign the output to play to specific channels as described by AVAudioSession's channels property */
/* This property is nil valued until set. */
/* The array must have the same number of channels as returned by the numberOfChannels property. */
@property(nonatomic, copy, nullable) NSArray<AVAudioSessionChannelDescription *> *channelAssignments NS_AVAILABLE(10_9, 7_0); /* Array of AVAudioSessionChannelDescription objects */
#endif

@end

/* A protocol for delegates of AVAudioPlayer */
__WATCHOS_AVAILABLE(3_0)
@protocol AVAudioPlayerDelegate <NSObject>
@optional 
/* audioPlayerDidFinishPlaying:successfully: is called when a sound has finished playing. This method is NOT called if the player is stopped due to an interruption. */
- (void)audioPlayerDidFinishPlaying:(AVAudioPlayer *)player successfully:(BOOL)flag;

/* if an error occurs while decoding it will be reported to the delegate. */
- (void)audioPlayerDecodeErrorDidOccur:(AVAudioPlayer *)player error:(NSError * __nullable)error;

#if TARGET_OS_IPHONE

/* AVAudioPlayer INTERRUPTION NOTIFICATIONS ARE DEPRECATED - Use AVAudioSession instead. */

/* audioPlayerBeginInterruption: is called when the audio session has been interrupted while the player was playing. The player will have been paused. */
- (void)audioPlayerBeginInterruption:(AVAudioPlayer *)player NS_DEPRECATED_IOS(2_2, 8_0);

/* audioPlayerEndInterruption:withOptions: is called when the audio session interruption has ended and this player had been interrupted while playing. */
/* Currently the only flag is AVAudioSessionInterruptionFlags_ShouldResume. */
- (void)audioPlayerEndInterruption:(AVAudioPlayer *)player withOptions:(NSUInteger)flags NS_DEPRECATED_IOS(6_0, 8_0);

- (void)audioPlayerEndInterruption:(AVAudioPlayer *)player withFlags:(NSUInteger)flags NS_DEPRECATED_IOS(4_0, 6_0);

/* audioPlayerEndInterruption: is called when the preferred method, audioPlayerEndInterruption:withFlags:, is not implemented. */
- (void)audioPlayerEndInterruption:(AVAudioPlayer *)player NS_DEPRECATED_IOS(2_2, 6_0);

#endif // TARGET_OS_IPHONE

@end

NS_ASSUME_NONNULL_END

3. Extended Audio File Services

由Audio File与Audio Converter组合而成，提供压缩及无压缩音频文件的读写能力。

它与Audio File Services、Audio File Stream Services和Audio Queue Services等同时存在AudioToolbox框架中。ExtendedAudioFile相对Audio File Services 和 Audio Converter Services ，API调用非常简单和明确，并且不需要去处理AudioStreamPacketDescription，在实际开发中逻辑更为清晰。

4. OpenAL

它就是存在框架OpenAL中。

是CoreAudio对OpenAL标准的实现，可以播放3D混音效果。

OpenAL 主要的功能是在来源物体、音效缓冲和收听者中编码。来源物体包含一个指向缓冲区的指标、声音的速度、位置和方向，以及声音强度。收听者物体包含收听者的速度、位置和方向，以及全部声音的整体增益。缓冲里包含 8 或 16 位元、单声道或立体声 PCM 格式的音效资料，表现引擎进行所有必要的计算，如距离衰减、多普勒效应等。

不同于 OpenGL 规格，OpenAL 规格包含两个API分支；以实际 OpenAL 函式组成的核心，和 ALC API，ALC 用于管理表现内容、资源使用情况，并将跨平台风格封在其中。还有“ALUT”程式库，提供高阶“易用”的函式，其定位相当于 OpenGL 的 GLUT。

---

Mid-Level 中级服务

该层功能比较齐全，包括音频数据格式转换，音频文件读写，音频流解析，插件工作支持等。

1. Audio Convert Services

它位于框架AudioToolbox中。

负责音频数据格式的转换

2. Audio File Services

它位于框架AudioToolbox中。

负责音频数据的读写。

3. Audio Unit Services 和 Audio Processing Graph Services

它位于框架AudioToolbox中。

支持均衡器和混音器等数字信号处理的插件。

4. Audio File Scream Services

它位于框架AudioToolbox中。

负责流解析。

5. Core Audio Clock Services

它位于框架Core Audio中。

负责音频音频时钟同步。

---

Low-Level 低级服务

该主要在MAC上的音频APP实现中并且需要最大限度的实时性能的情况下使用，大部分音频APP不需要使用该层的服务。而且，在iOS上也提供了具备较高实时性能的高层API达到你的需求。例如OpenAL，在游戏中具备与I/O直接调用的实时音频处理能力。

1. I/O Kit

它在IOKit框架中，与硬件驱动交互。

获得用户空间访问硬件设备和驱动程序。I / O Kit框架通过设备接口机制实现对I / O Kit对象（驱动程序和结点）的非内核访问。

2. Audio HAL

音频硬件抽象层，使API调用与实际硬件相分离，保持独立。

3. Core MIDI

它位于Core MIDI框架中，与MIDI设备（如硬件键盘和合成器）进行通信。

Core MIDI框架提供了用于与MIDI（乐器数字接口）设备（包括硬件键盘和合成器）进行通信的API。 使用基座连接器或网络从iOS设备进行连接。 有关使用基座连接器的更多信息，请参阅Apple的MFi program。

4. Host Time Services

访问电脑硬件时钟。

---

音频框架不同场景使用分析

1. condition One

只实现音频的播放，没有其他需求，AVAudioPlayer就可以满足需求。它的接口使用简单，不用关心其中的细节，通常只提供给它一个播放源的URL地址，并且调用其play、pause、stop等方法进行控制，observer其播放状态更新UI即可。

2. condition Two

APP需要对音频进行流播放，就需要AudioFileStreamer加Audio Queue，将网络或者本地的流读取到内存，提交给AudioFileStreamer解析分离音频帧，分离出来的音频帧可以送给AudioQueue进行解码和播放，可参考下面。

AudioStreamer

FreeStreamer

AFSoundManager

3. condition Three

APP需要需要对音频施加音效（均衡器、混响器），就是除了数据的读取和解析以外还需要用到AudioConverter或者Codec来把音频数据转换成PCM数据，再由AudioUnit+AUGraph来进行音效处理和播放，可参考下面。

DouAudioStreamer

TheAmazingAudioEngine

AudioKit

参考文章

1. iOS Audio Unit(一)

后记

未完，待续~~~