Audio File Reading and Writing

I have implemented audioinfo and audioread and they should work exactly as in MATLAB. Here is an example session.

octave:1> audioinfo('technical_difficulties.wav')
ans =

  scalar structure containing the fields:

    Filename = technical_difficulties.wav
    CompressionMethod =
    NumChannels =  2
    SampleRate =  44100
    TotalSamples =  305576
    Duration =  6.9292
    BitsPerSample =  16
    BitRate = -1
    Title =
    Artist =
    Comment =

octave:2> [y, Fs] = audioread('technical_difficulties.wav');
octave:3> player = audioplayer(y, Fs);
octave:4> play(player);

For further information about this functionality refer to MATLAB manual pages on these functions: here and here.

Further Development

Since we already have a working version of the audioplayer and audiorecorder functionality that allows one to write callbacks in Octave here is my plan for further development. There are some issues to be ironed out but I think the interface and functionality will stay more or less the same. We have the firm "pencils down" date set at September 23 by Google.

1. October 19-25 implement audioread and audioinfo using libsndfile.
2. October 26-September 1 implement audiowrite.
3. Use the remainder of the time to integrate everything into Octave proper and fix any issues, write documentation, tests, etc.

Changes to Octave Callback Interface

The interface to PortAudio that allows you to write callback functions in Octave underwent slight changes. From now on sound is returned in a single Matrix with one or two columns (depending on number of channels) with as many rows as there are audio frames. An example playing back sine waves of different frequencies on the two channels is given below.

function [ sound, status ] = callback_sine (frames)
  global lphase = 0.0;
  global rphase = 0.0;
  incl = 440.0 / 44100.0;
  incr = 443.0 / 44100.0;
  nl = incl * frames;
  nr = incr * frames;
  left = sin (2.0 * pi * [lphase:incl:lphase+nl]);
  right = sin (2.0 * pi * [rphase:incr:rphase+nr]);
  sound = [left', right'];
  status = 0;
  lphase = lphase + nl;
  rphase = rphase + nr;
endfunction

An example of it's use is given below.

player = audioplayer(@callback_sine);
play(player);
# play as long as you want
stop(player);

It is also possible to use Octave functions to process recorded data. An example that writes recorded data in to a text file is given below.

function status = callback_record (sound)
  fid = fopen('record.txt', 'at');
  for index = 1:rows(sound)
    fprintf(fid, "%.4f, %.4f\n", sound(index, 1), sound(index, 2));
  endfor
  fclose(fid);
  status = 0;
endfunction

Here is a plot of the resulting file with me clapping three times. Only one channel is plotted.

Writing Portaudio Callbacks in Octave

It is now possible to have Octave functions supply audio data to be played by audioplayer objects. Stereo and mono playback is supported, the function must supply a specified number of audio frames and make sure to return a valid PortAudio status value (0 for continue, 1 for completed and 2 for abort). For example, put the following in a file called callback_noise.m:

function [ left, right, status ] = callback_noise (frames)
left = randn (1, frames) - 0.5;
right = randn (1, frames) - 0.5;
status = 0;
endfunction

You can now use this as a callback as shown below.

player = audioplayer ('callback_noise', 44100);
play (player);
# play for as much you like
stop (player);

This will play white noise on both channels, which is not really interesting. To make matters more interesting we can use global variables to store state between callback calls. The following will play sine waves of different frequencies in both channels, place it in a file called callback_sine.m:

function [ left, right, status ] = callback_sine (frames)
global lphase = 0.0;
global rphase = 0.0;
incl = 440.0 / 44100.0;
incr = 443.0 / 44100.0;
nl = incl * frames;
nr = incr * frames;
left = sin (2.0 * pi * [lphase:incl:lphase+nl]);
right = sin (2.0 * pi * [rphase:incr:rphase+nr]);
status = 0;
lphase = lphase + nl;
rphase = rphase + nr;
endfunction

Both of these examples will play for as long as you don't use the stop method on the player object you created. This shows an advantage of this mode of operation over having to store all data in memory at one time. This way only small portion of audio has to be stored at any given time.

During audioplayer object construction all the parameters work the same way as when passing audio data. You can supply bit depth and device ID in addition to sampling frequency if you want to.

The Status of the Project

All of the major goals we set for the program were achieved before midterm with small exceptions. That means that audiodevinfo, audioplayer and audiorecorder (supposedly) work as they are described in MATLAB documentation here. There are some small exceptions. Notable properties that depend on the timer class don't work. These are used to issue command during recording and playback periodically. There still does not seem to be a clear agreement on what to do after the midterm, however there are several good possibilities.

Audiorecorder Class is Working

All of the functionality of the audiorecorder is implemented except for the properties dependent on the timer class.

octave:1> recorder = audiorecorder (44100, 16, 2);
octave:2> record (recorder);
# record something using the default recording device on your system
octave:3> stop (recorder);
# now you have several options, for example you get get the data for manipulation:
octave:4> data = getaudiodata (recorder);
# play the recorded data directly
octave:5> play (recorder);
# or get an audioplayer object containing the recorded audio
octave:6> player = audioplayer (recorder);
# or alternatively
octave:7> player = getplayer (recorder);

All the other examples from MATLAB documentation should be working as well.

Directions for Further Development

Since both audioplayer and audiorecorder classes are going to be completed before the midterm, thus effectively completing the goals outlined in the original GSoC proposal we need to come up with things to do after the midterm. I have the following suggestions:

1. Implement audioinfo, audioread and audiowrite functionality that is present in MATLAB. I can do this using libsndfile library for cross-platform audio file reading and writing. I have worked with this library before and I expect this would not be as big a task as it might seem at first.
2. Implement the timer class from MATLAB. Possibly using Boost::Thread for threading? It is needed to fully implement all the functionality in audioplayer and audiorecorder.
3. Callback based playback and recording. That is allow the user to register an Octave function to be called during the processing of each audio buffer and get it to provide the data for playback or to do something with the recorded audio.
4. Anything else you might think is a good idea to have in Octave and falls roughly within it's audio/DSP functionality.

New Functionality

Most of the functionality of MATLAB's audioplayer class is working now. The only two missing pieces of functionality is the ability to specify an audiorecorder object to use. I am not even fully sure what that is for yet. The other piece of missing functionality are the StartFcn, StopFcn, TimerFcn and TimerPeriod properties that require a working timer class. Other than that any code using audioplayer that works in MATLAB should work in Octave too if you compile my code from https://bitbucket.org/bucket_brigade/octave-sound. Here are some examples to try out.

octave:1> sound = randn(2, 441000) - 0.5;
octave:2> player = audioplayer(sound, 44100);
octave:3> play(player);
# wait a little bit
octave:4> pause(player);
# will pause the playback
octave:5> resume(player);
# will resume the playback where it was paused

octave:1> sound = randn(2, 441000) - 0.5;
octave:2> player = audioplayer(sound, 44100, 8);
octave:3> player.BitsPerSample
ans = 8
octave:4> player.Tag = "testing";
octave:5> player.Tag
ans = testing

You can obviously load an audio file if you have one in suitable format using octave's loadaudio function, apply some signal processing on it and play that back.

Non-blocking Playback

Non-blocking playback is now working as well. See example below.

signal1 = sin(linspace(0.0, 440.0 * 10.0 * 2.0 * pi, 441000)) * 0.2;
signal2 = sin(linspace(0.0, 442.0 * 10.0 * 2.0 * pi, 441000)) * 0.2;
signal = [signal1; signal2];
player = audioplayer(signal, 44100);
player_play(player);
sleep(5);
player_pause(player);
sleep(5);
player_resume(player);

This example will play a 440Hz sine wave in the left speaker and 442Hz sine wave in the right speaker for 5 seconds, pause, wait 5 seconds and then resume playing again. The difference between blocking and non-blocking modes is that when using non-blocking mode, after issuing the play command the control returns to the octave interpreter.

Basic Blocking Playback

The new audio system now supports basic blocking playback.

player = audioplayer(rand(2, 44100), 44100);
player_playblocking(player);

The code above will play one second of white noise on both speakers at 44100 sampling rate. The signals in the different speakers will be different. Mono playback is also supported.

player = audioplayer(rand(1, 44100), 44100);
player_playblocking(player);

Will play the same signal on both channels.

Different audioplayer parameters will result in different player settings. For example:

player = audioplayer(signal, 44100, 16);

Will result in a player that expects signals to be 16 bit integer values.

player = audioplayer(signal, 44100, 16, id);

Will result in a player that tries to write to audio output device with the specified id. If no id is specified then the player will use the default audio device on that system.       

Welcome

This blog is dedicated to my Google Summer of Code 2013 project of implementing MATLAB like audio playback and recording classes in Octave. The source repository for the project is here. Stay tuned.