一、簡介
Audio是多媒體子系統(tǒng)中的一個重要模塊,其涉及的內(nèi)容比較多,有音頻的渲染、音頻的采集、音頻的策略管理等。本文主要針對音頻渲染功能進(jìn)行詳細(xì)地分析,并通過源碼中提供的例子,對音頻渲染進(jìn)行流程的梳理。
二、目錄
foundation/multimedia/audio_framework
audio_framework
├── frameworks
│ ├── js
│ │ └── napi
│ │ └── audio_renderer #audio_renderer NAPI接口
│ │ ├── include
│ │ │ ├── audio_renderer_callback_napi.h
│ │ │ ├── renderer_data_request_callback_napi.h
│ │ │ ├── renderer_period_position_callback_napi.h
│ │ │ └── renderer_position_callback_napi.h
│ │ └── src
│ │ ├── audio_renderer_callback_napi.cpp
│ │ ├── audio_renderer_napi.cpp
│ │ ├── renderer_data_request_callback_napi.cpp
│ │ ├── renderer_period_position_callback_napi.cpp
│ │ └── renderer_position_callback_napi.cpp
│ └── native
│ └── audiorenderer
│ ├── BUILD.gn
│ ├── include
│ │ ├── audio_renderer_private.h
│ │ └── audio_renderer_proxy_obj.h
│ ├── src
│ │ ├── audio_renderer.cpp
│ │ └── audio_renderer_proxy_obj.cpp
│ └── test
│ └── example
│ └── audio_renderer_test.cpp
├── interfaces
│ ├── inner_api
│ │ └── native
│ │ └── audiorenderer
│ │ └── include
│ │ └── audio_renderer.h
│ └── kits
│ └── js
│ └── audio_renderer
│ └── include
│ └── audio_renderer_napi.h
└── services #服務(wù)端
└── audio_service
├── BUILD.gn
├── client #IPC調(diào)用中的proxy端
│ ├── include
│ │ ├── audio_manager_proxy.h
│ │ ├── audio_service_client.h
│ └── src
│ ├── audio_manager_proxy.cpp
│ ├── audio_service_client.cpp
└── server #IPC調(diào)用中的server端
├── include
│ └── audio_server.h
└── src
├── audio_manager_stub.cpp
└── audio_server.cpp
(左右移動查看全部內(nèi)容)
三、音頻渲染總體流程
四、Native接口使用
在OpenAtom OpenHarmony(以下簡稱“OpenHarmony”)系統(tǒng)中,音頻模塊提供了功能測試代碼,本文選取了其中的音頻渲染例子作為切入點(diǎn)來進(jìn)行介紹,例子采用的是對wav格式的音頻文件進(jìn)行渲染。wav格式的音頻文件是wav頭文件和音頻的原始數(shù)據(jù),不需要進(jìn)行數(shù)據(jù)解碼,所以音頻渲染直接對原始數(shù)據(jù)進(jìn)行操作,文件路徑為:foundation/multimedia/audio_framework/frameworks/native/audiorenderer/test/example/audio_renderer_test.cpp
bool TestPlayback(int argc, char *argv[]) const
{
FILE* wavFile = fopen(path, "rb");
//讀取wav文件頭信息
size_t bytesRead = fread(&wavHeader, 1, headerSize, wavFile);
//設(shè)置AudioRenderer參數(shù)
AudioRendererOptions rendererOptions = {};
rendererOptions.streamInfo.encoding = AudioEncodingType::ENCODING_PCM;
rendererOptions.streamInfo.samplingRate = static_cast(wavHeader.SamplesPerSec);
rendererOptions.streamInfo.format = GetSampleFormat(wavHeader.bitsPerSample);
rendererOptions.streamInfo.channels = static_cast(wavHeader.NumOfChan);
rendererOptions.rendererInfo.contentType = contentType;
rendererOptions.rendererInfo.streamUsage = streamUsage;
rendererOptions.rendererInfo.rendererFlags = 0;
//創(chuàng)建AudioRender實(shí)例
unique_ptr audioRenderer = AudioRenderer::Create(rendererOptions);
shared_ptr cb1 = make_shared();
//設(shè)置音頻渲染回調(diào)
ret = audioRenderer->SetRendererCallback(cb1);
//InitRender方法主要調(diào)用了audioRenderer實(shí)例的Start方法,啟動音頻渲染
if(!InitRender(audioRenderer)) {
AUDIO_ERR_LOG("AudioRendererTest: Init render failed");
fclose(wavFile);
return false;
}
//StartRender方法主要是讀取wavFile文件的數(shù)據(jù),然后通過調(diào)用audioRenderer實(shí)例的Write方法進(jìn)行播放
if(!StartRender(audioRenderer, wavFile)) {
AUDIO_ERR_LOG("AudioRendererTest: Start render failed");
fclose(wavFile);
return false;
}
//停止渲染
if(!audioRenderer->Stop()) {
AUDIO_ERR_LOG("AudioRendererTest: Stop failed");
}
//釋放渲染
if(!audioRenderer->Release()) {
AUDIO_ERR_LOG("AudioRendererTest: Release failed");
}
//關(guān)閉wavFile
fclose(wavFile);
return true;
}
(左右移動查看全部內(nèi)容)
首先讀取wav文件,通過讀取到wav文件的頭信息對AudioRendererOptions相關(guān)的參數(shù)進(jìn)行設(shè)置,包括編碼格式、采樣率、采樣格式、通道數(shù)等。根據(jù)AudioRendererOptions設(shè)置的參數(shù)來創(chuàng)建AudioRenderer實(shí)例(實(shí)際上是AudioRendererPrivate),后續(xù)的音頻渲染主要是通過AudioRenderer實(shí)例進(jìn)行。創(chuàng)建完成后,調(diào)用AudioRenderer的Start方法,啟動音頻渲染。啟動后,通過AudioRenderer實(shí)例的Write方法,將數(shù)據(jù)寫入,音頻數(shù)據(jù)會被播放。
五、調(diào)用流程
1、創(chuàng)建AudioRenderer
std::unique_ptrAudioRenderer::Create(const std::string cachePath,
const AudioRendererOptions &rendererOptions, const AppInfo &appInfo)
{
ContentType contentType = rendererOptions.rendererInfo.contentType;
StreamUsage streamUsage = rendererOptions.rendererInfo.streamUsage;
AudioStreamType audioStreamType = AudioStream::GetStreamType(contentType, streamUsage);
auto audioRenderer = std::make_unique(audioStreamType, appInfo);
if (!cachePath.empty()) {
AUDIO_DEBUG_LOG("Set application cache path");
audioRenderer->SetApplicationCachePath(cachePath);
}
audioRenderer->rendererInfo_.contentType = contentType;
audioRenderer->rendererInfo_.streamUsage = streamUsage;
audioRenderer->rendererInfo_.rendererFlags = rendererOptions.rendererInfo.rendererFlags;
AudioRendererParams params;
params.sampleFormat = rendererOptions.streamInfo.format;
params.sampleRate = rendererOptions.streamInfo.samplingRate;
params.channelCount = rendererOptions.streamInfo.channels;
params.encodingType = rendererOptions.streamInfo.encoding;
if (audioRenderer->SetParams(params) != SUCCESS) {
AUDIO_ERR_LOG("SetParams failed in renderer");
audioRenderer = nullptr;
return nullptr;
}
return audioRenderer;
}
(左右移動查看全部內(nèi)容)
首先通過AudioStream的GetStreamType方法獲取音頻流的類型,根據(jù)音頻流類型創(chuàng)建AudioRendererPrivate對象,AudioRendererPrivate是AudioRenderer的子類。緊接著對audioRenderer進(jìn)行參數(shù)設(shè)置,其中包括采樣格式、采樣率、通道數(shù)、編碼格式。設(shè)置完成后返回創(chuàng)建的AudioRendererPrivate實(shí)例。
2、設(shè)置回調(diào)
int32_t AudioRendererPrivate::SetRendererCallback(const std::shared_ptr &callback)
{
RendererState state = GetStatus();
if (state == RENDERER_NEW || state == RENDERER_RELEASED) {
return ERR_ILLEGAL_STATE;
}
if (callback == nullptr) {
return ERR_INVALID_PARAM;
}
// Save reference for interrupt callback
if (audioInterruptCallback_ == nullptr) {
return ERROR;
}
std::shared_ptr cbInterrupt =
std::static_pointer_cast(audioInterruptCallback_);
cbInterrupt->SaveCallback(callback);
// Save and Set reference for stream callback. Order is important here.
if (audioStreamCallback_ == nullptr) {
audioStreamCallback_ = std::make_shared();
if (audioStreamCallback_ == nullptr) {
return ERROR;
}
}
std::shared_ptr cbStream =
std::static_pointer_cast(audioStreamCallback_);
cbStream->SaveCallback(callback);
(void)audioStream_->SetStreamCallback(audioStreamCallback_);
return SUCCESS;
}
(左右移動查看全部內(nèi)容)
參數(shù)傳入的回調(diào)主要涉及到兩個方面:一方面是AudioInterruptCallbackImpl中設(shè)置了我們傳入的渲染回調(diào),另一方面是AudioStreamCallbackRenderer中也設(shè)置了渲染回調(diào)。
3、啟動渲染
bool AudioRendererPrivate::Start(StateChangeCmdType cmdType) const
{
AUDIO_INFO_LOG("AudioRenderer::Start");
RendererState state = GetStatus();
AudioInterrupt audioInterrupt;
switch (mode_) {
case InterruptMode:
audioInterrupt = sharedInterrupt_;
break;
case InterruptMode:
audioInterrupt = audioInterrupt_;
break;
default:
break;
}
AUDIO_INFO_LOG("AudioRenderer: %{public}d, streamType: %{public}d, sessionID: %{public}d",
mode_, audioInterrupt.streamType, audioInterrupt.sessionID);
if (audioInterrupt.streamType == STREAM_DEFAULT || audioInterrupt.sessionID == INVALID_SESSION_ID) {
return false;
}
int32_t ret = AudioPolicyManager::GetInstance().ActivateAudioInterrupt(audioInterrupt);
if (ret != 0) {
AUDIO_ERR_LOG("AudioRendererPrivate::ActivateAudioInterrupt Failed");
return false;
}
return audioStream_->StartAudioStream(cmdType);
}
(左右移動查看全部內(nèi)容)
AudioPolicyManager::GetInstance().ActivateAudioInterrupt這個操作主要是根據(jù)AudioInterrupt來進(jìn)行音頻中斷的激活,這里涉及了音頻策略相關(guān)的內(nèi)容,后續(xù)會專門出關(guān)于音頻策略的文章進(jìn)行分析。這個方法的核心是通過調(diào)用AudioStream的StartAudioStream方法來啟動音頻流。
bool AudioStream::StartAudioStream(StateChangeCmdType cmdType)
{
int32_t ret = StartStream(cmdType);
resetTime_ = true;
int32_t retCode = clock_gettime(CLOCK_MONOTONIC, &baseTimestamp_);
if (renderMode_ == RENDER_MODE_CALLBACK) {
isReadyToWrite_ = true;
writeThread_ = std::make_unique<std::thread>(&AudioStream::WriteCbTheadLoop, this);
} else if (captureMode_ == CAPTURE_MODE_CALLBACK) {
isReadyToRead_ = true;
readThread_ = std::make_unique<std::thread>(&AudioStream::ReadCbThreadLoop, this);
}
isFirstRead_ = true;
isFirstWrite_ = true;
state_ = RUNNING;
AUDIO_INFO_LOG("StartAudioStream SUCCESS");
if (audioStreamTracker_) {
AUDIO_DEBUG_LOG("AudioStream:Calling Update tracker for Running");
audioStreamTracker_->UpdateTracker(sessionId_, state_, rendererInfo_, capturerInfo_);
}
return true;
}
(左右移動查看全部內(nèi)容)
AudioStream的StartAudioStream主要的工作是調(diào)用StartStream方法,StartStream方法是AudioServiceClient類中的方法。AudioServiceClient類是AudioStream的父類。接下來看一下AudioServiceClient的StartStream方法。
int32_t AudioServiceClient::StartStream(StateChangeCmdType cmdType)
{
int error;
lock_guardlockdata(dataMutex);
pa_operation *operation = nullptr;
pa_threaded_mainloop_lock(mainLoop);
pa_stream_state_t state = pa_stream_get_state(paStream);
streamCmdStatus = 0;
stateChangeCmdType_ = cmdType;
operation = pa_stream_cork(paStream, 0, PAStreamStartSuccessCb, (void *)this);
while (pa_operation_get_state(operation) == PA_OPERATION_RUNNING) {
pa_threaded_mainloop_wait(mainLoop);
}
pa_operation_unref(operation);
pa_threaded_mainloop_unlock(mainLoop);
if (!streamCmdStatus) {
AUDIO_ERR_LOG("Stream Start Failed");
ResetPAAudioClient();
return AUDIO_CLIENT_START_STREAM_ERR;
} else {
AUDIO_INFO_LOG("Stream Started Successfully");
return AUDIO_CLIENT_SUCCESS;
}
}
(左右移動查看全部內(nèi)容)
StartStream方法中主要是調(diào)用了pulseaudio庫的pa_stream_cork方法進(jìn)行流啟動,后續(xù)就調(diào)用到了pulseaudio庫中了。pulseaudio庫我們暫且不分析。
4、寫入數(shù)據(jù)
int32_t AudioRendererPrivate::Write(uint8_t *buffer, size_t bufferSize)
{
return audioStream_->Write(buffer, bufferSize);
}
(左右移動查看全部內(nèi)容)
通過調(diào)用AudioStream的Write方式實(shí)現(xiàn)功能,接下來看一下AudioStream的Write方法。
size_t AudioStream::Write(uint8_t *buffer, size_t buffer_size)
{
int32_t writeError;
StreamBuffer stream;
stream.buffer = buffer;
stream.bufferLen = buffer_size;
isWriteInProgress_ = true;
if (isFirstWrite_) {
if (RenderPrebuf(stream.bufferLen)) {
return ERR_WRITE_FAILED;
}
isFirstWrite_ = false;
}
size_t bytesWritten = WriteStream(stream, writeError);
isWriteInProgress_ = false;
if (writeError != 0) {
AUDIO_ERR_LOG("WriteStream fail,writeError:%{public}d", writeError);
return ERR_WRITE_FAILED;
}
return bytesWritten;
}
(左右移動查看全部內(nèi)容)
Write方法中分成兩個階段,首次寫數(shù)據(jù),先調(diào)用RenderPrebuf方法,將preBuf_的數(shù)據(jù)寫入后再調(diào)用WriteStream進(jìn)行音頻數(shù)據(jù)的寫入。
size_t AudioServiceClient::WriteStream(const StreamBuffer &stream, int32_t &pError)
{
size_t cachedLen = WriteToAudioCache(stream);
if (!acache.isFull) {
pError = error;
return cachedLen;
}
pa_threaded_mainloop_lock(mainLoop);
const uint8_t *buffer = acache.buffer.get();
size_t length = acache.totalCacheSize;
error = PaWriteStream(buffer, length);
acache.readIndex += acache.totalCacheSize;
acache.isFull = false;
if (!error && (length >= 0) && !acache.isFull) {
uint8_t *cacheBuffer = acache.buffer.get();
uint32_t offset = acache.readIndex;
uint32_t size = (acache.writeIndex - acache.readIndex);
if (size > 0) {
if (memcpy_s(cacheBuffer, acache.totalCacheSize, cacheBuffer + offset, size)) {
AUDIO_ERR_LOG("Update cache failed");
pa_threaded_mainloop_unlock(mainLoop);
pError = AUDIO_CLIENT_WRITE_STREAM_ERR;
return cachedLen;
}
AUDIO_INFO_LOG("rearranging the audio cache");
}
acache.readIndex = 0;
acache.writeIndex = 0;
if (cachedLen < stream.bufferLen) {
StreamBuffer str;
str.buffer = stream.buffer + cachedLen;
str.bufferLen = stream.bufferLen - cachedLen;
AUDIO_DEBUG_LOG("writing pending data to audio cache: %{public}d", str.bufferLen);
cachedLen += WriteToAudioCache(str);
}
}
pa_threaded_mainloop_unlock(mainLoop);
pError = error;
return cachedLen;
}
(左右移動查看全部內(nèi)容)
WriteStream方法不是直接調(diào)用pulseaudio庫的寫入方法,而是通過WriteToAudioCache方法將數(shù)據(jù)寫入緩存中,如果緩存沒有寫滿則直接返回,不會進(jìn)入下面的流程,只有當(dāng)緩存寫滿后,才會調(diào)用下面的PaWriteStream方法。該方法涉及對pulseaudio庫寫入操作的調(diào)用,所以緩存的目的是避免對pulseaudio庫頻繁地做IO操作,提高了效率。
六、總結(jié)
本文主要對OpenHarmony 3.2 Beta多媒體子系統(tǒng)的音頻渲染模塊進(jìn)行介紹,首先梳理了Audio Render的整體流程,然后對幾個核心的方法進(jìn)行代碼的分析。整體的流程主要通過pulseaudio庫啟動流,然后通過pulseaudio庫的pa_stream_write方法進(jìn)行數(shù)據(jù)的寫入,最后播放出音頻數(shù)據(jù)。
音頻渲染主要分為以下幾個層次:
-
AudioRenderer的創(chuàng)建,實(shí)際創(chuàng)建的是它的子類AudioRendererPrivate實(shí)例。
-
通過AudioRendererPrivate設(shè)置渲染的回調(diào)。
-
啟動渲染,這一部分代碼最終會調(diào)用到pulseaudio庫中,相當(dāng)于啟動了pulseaudio的流。
-
通過pulseaudio庫的pa_stream_write方法將數(shù)據(jù)寫入設(shè)備,進(jìn)行播放。
更多熱點(diǎn)文章閱讀
- OS內(nèi)核及視窗分論壇詳解之OpenHarmony 3D顯示支持
- 應(yīng)用模型開發(fā)指南上新介紹
- 技術(shù)構(gòu)筑萬物智聯(lián),第一屆OpenHarmony技術(shù)峰會圓滿舉行
- OpenHarmony L1(3.0)串口功能開發(fā)
-
小白指南:手把手教你用低代碼開發(fā)一個應(yīng)用頁面
提示:本文由電子發(fā)燒友論壇發(fā)布,轉(zhuǎn)載請注明來源。如需社區(qū)合作及入群交流,請?zhí)砑游⑿臙EFans0806,或者發(fā)郵箱liuyong@huaqiu.com。
原文標(biāo)題:OpenHarmony 3.2 Beta Audio:音頻渲染
文章出處:【微信公眾號:電子發(fā)燒友開源社區(qū)】歡迎添加關(guān)注!文章轉(zhuǎn)載請注明出處。
-
電子發(fā)燒友
+關(guān)注
關(guān)注
33文章
548瀏覽量
32831 -
開源社區(qū)
+關(guān)注
關(guān)注
0文章
93瀏覽量
396
原文標(biāo)題:OpenHarmony 3.2 Beta Audio:音頻渲染
文章出處:【微信號:HarmonyOS_Community,微信公眾號:電子發(fā)燒友開源社區(qū)】歡迎添加關(guān)注!文章轉(zhuǎn)載請注明出處。
發(fā)布評論請先 登錄
相關(guān)推薦
評論