ALTERNATIVE CSOUND PLAYERS AND COMPOSING ENVIRONMENTS FOR ANDROID
Art Hunkins
abhunkin@uncg.edu
http://www.arthunkins.com

INTRODUCTION

The Android apps presented in this article describe expanded user interfaces as compared to the those posted on Sourceforge.[1] Otherwise largely identical to the "canonical" apps, these are primarily intended for 7" Android tablets and larger. Csound5a and Csound6a GUIs include a fixed arrangement of 9 sliders, 12 buttons and no trackpad. Csound6b includes up to 16 sliders and buttons, but no trackpad or console output.  Csound6c is identical to 6b, except that it adds back a single-line Csound console output. These apps can all coexist with the original Android Csound apps. The article places these enhanced apps within the general context of the development of Android Csound.



In mid-to-late 2012, Csound took a major step forward as it was ported to mobile devices.[2] For the first time, Csound compositions could be performed in the palm of your hand, at a total cost of less than $100US (for Android devices). Until then, a desktop or laptop computer, often tethered to a wall socket - and in the case of live performance, an external MIDI controller - were normally required.

In March 2012, Victor Lazzarini introduced the first "CSD Player" app in his Audio Programming Blog: CsoundApp.apk.[3] This app included a simple performance GUI consisting of 5 sliders and buttons, plus an x/y controller or trackpad (essentially two sliders in one). It was the model for the Android apps that followed.[4] Its sliders and buttons conveniently substituted for a MIDI controller in interactive performance.

Lazzarini and Steven Yi subsequently presented their ideas for Csound on mobile devices in a paper, "Digital Audio Effects on Mobile Platforms", in September 2012.[5] It described another app, CsoundAndroid.apk, which, though not the same kind of player, demoed 9 simple examples that included sliders, buttons, a complete Csound composition[6], waveform display and a multitouch performance controller! The app remains a persuasive demonstration of the power of Csound ported to the Android OS.

Two refinements of CsoundApp.apk led to CsoundApp-5.19.apk, built with the final incarnation of Csound5 (5.19). These versions included limited console output, and did a modest job of reporting compile-time errors - but gave no performance-time information. If you wished, you could tweak your composition in an external text editor on your device.[7] Nonetheless, these apps were all appropriately identified as "CSD Players."[8]
 
Eager to try my hand at porting one of my own MIDI compositions to Android, I set about converting my composition, Chimeplay[9], to ChimePad (ChimePad.csd), for performance with CsoundApp.apk. It was relatively easy to adapt a minimal version for CsoundApp's GUI. I purposefully used all 5 sliders and buttons, and the x/y controller just to see what was possible. As the computational demands were fairly minimal, the result proved highly workable.

Most of my MIDI compositions, however, require 8-9 or more sliders, and I wanted to see how adaptable they might be to CsoundApp's  modest set of widgets. My compositions Spiritus Sanctus and Spiritus Sanctus2 - likewise "sonic environments" - required anywhere from 7 to 16 MIDI sliders. Could CsoundApp handle greater complexity? I created fairly minimal versions for CsoundApp's 5 sliders, with two additional "sliders" handled by the x/y trackpad. It turned out that the Android OS could perform Spiritus Sanctus and Spiritus Sanctus2 with little difficulty.   

Converting MIDI code from hardware controllers to Android widgets, has not proven a major undertaking. It involves changing ctrl7 and midiin opcodes to chnget calls. There are adjustments to make in slider ranges (the Android is 0 to 1), and one must remember that Android buttons are momentary contact (but thankfully much more simply programmed than midiin events).

The only major conversion issue was in making the x/y trackpad act as separate "sliders."[10] The two problems were: 1) unlike MIDI sliders, there was no visual indication of the last touched value, and so it was difficult to return to the last "spot"; 2) when lifting your finger, the "slider" returned to zero instead of holding at its last "touched" value. I found a way to code around the second issue, and dealt with the first by either ending on min or max "slider" values (0 or 1), or assigning slider parameters whose precise value was not significant.

For me, however, additional sliders would clearly be preferable to an x/y controller. The trackpad took up a lot of screen space - space that I thought would be better populated with additional sliders and buttons. I wanted to see how many separate controllers the Android, with its limited screen-space and computing power, might handle. And so the idea for my first alternative/enhanced CSD Player was born.

To realize this vision, I enlisted the talents of my son Dave, an independent Android developer with a significant interest in sound and music[11]. Together we modified CsoundApp to create Csound5a.apk, with 9 sliders and 12 buttons, but no trackpad. It retained the quite limited console window. We hoped Android could cope with this additional level of musical complexity.[12]

To test this premise, I converted two recent MIDI pieces for performance by Csound5a: Spiritus Sanctus and Spiritus Sanctus 2. The former, which required 9 sliders and 12 buttons, became Spiritus Sanctus alt. The latter, with 8 sliders only, became Spiritus Sanctus 2alt. They worked well.[13]


A quantum leap forward for Csound took place in mid-2013, when Michael Gogins ported the new Csound6 to the Android OS. His port transformed Csound into a complete development system and composing environment for Android. Meanwhile, the app had expanded from under 3MB to 10MB and more: the distribution now included a 4+MB GM soundfont bank, 1+MB of HRTF files, and librairies for fluidsynth, Lua, and signalflow graph opcodes. Also present were a basic Csound tutorial, a built-in link to your choice of text editor and to on-line Csound documentation. The editor link was a real compositional boon; it was now far more practical to make minor changes to a .csd on the device itself.

Again eager to take advantage of these expanded resources, my son and I collaborated on Csound6a.apk[14], which duplicated the GUI of our Csound5a.apk. All the works I had ported to 5a now could be worked on and performed in this new development environment. The composer was now no longer necessarily tethered to a traditional computer.

Encouraged by the results of my work with a 9-slider GUI, I wondered whether the more advanced versions of my compositions - which required up to 16 or more sliders - could be accommodated by the Android platform. I decided on a maximum of 16 sliders, as I wanted these works to be performable on smartphones - though larger tablets would clearly be more practical.[15] In addition, given display limitations, I wanted the ability to specify only the number of sliders (and buttons) actually required, and thus for the GUI to be customizable to a certain degree for the specific composition. Thinking too of compositions that employed only buttons (as rudimentary "drum pads") an option for "buttons only" was included. So, Csound6b was born.[16]

With Csound6b.apk, we reached the limit of what at least the current generation of Android devices could handle. My compositions, fortunately, require neither great computing power nor low latency. They all perform quite well on Android with these alternative apps. It seems clear to me, however, that more complex and computationally demanding works would not - at least at this time.

My most recent compositions, Peace be with You and Peace be with You2, presented an entirely different "Android challenge", however.[17] Besides 9-16 sliders and 9 buttons, they required at least minimal Csound console feedback - both during setup and actual performance.[18] With the screen maximally chock-full of widgets, there was precious little space remaining for Console output.  In fact, there was only room for a single line!

Getting my printed feedback on a single line turned out not to be so difficult[19]; the greater challenge was to get the "correct" line to appear. The complicating factor was the fact that Android doesn't recognize "\r" (carriage return alone) - just "\n" (newline)! Fortunately, console output of performance data is always the *last* line of output, and a one-line console window constantly displays this last line! So normally, no scrolling is evident; the new line simply appears to rewrite the old. This result turns out to be more user-friendly than the multi-line Console output of both the Csound6 and Csound6a apps. It has the added advantage of Font size being able to be set in Settings | Display ("Normal" is a good size - larger and more readable than what both CS6 and CS6a put out).[20]

CONCLUSION

The additional Csound players and composing environments discussed here allow for the performance and composing of realtime Csound works with a greater number of both sliders and buttons (up to 16 each). In some cases the specific number of controls can be chosen by the user. Though generic and limited in nature, they are simple, practical, and user-friendly alternatives to creating complete Android apps, or using HTML5 within Csound to create custom GUIs.[21] Like the "canonical" versions, they require only an Android device, a single Android app, no internet connection or wifi router - and, as a bonus, only minimal modification of .csds.[22]
 
A NOTE ABOUT LATENCY AND DROPOUTS	

In my informal button tests for latency and dropouts, I got results comparable to Victor Lazzarini.[23] With my Samsung Galaxy Tab2 7.0 (Android 4.2.2), I found maximal settings with a mono SR=44100 to be: -b128 and -B2048. (Victor cited -B1024, but I got some dropout at that number.) As he states, values lower that -b128 don't get better latency, and values higher than -B2048 result in greater latency. I agree that, for the most part, the latency of these maximal settings is workable, if hardly ideal.

If, due to textural complexity, these settings do result in dropouts, the best idea is to lower Sample Rate (to 32000, 22050, 11025 or even 8000 - all valid values). Be aware, however, that these lower rates are accompanied by higher latency. Finally, SR=48000 at the recommended settings, is not possible; it will produce dropouts.


FOOTNOTES

[1] http://sourceforge.net/projects/csound/files/ (in Csound6 and Csound5 directories)

[2] Over the same time period, Victor Lazzarini and Steven Yi ported Csound both to iOS and Android. Their seminal presentation, "Csound for Android", is available at: http://lac.linuxaudio.org/2012/papers/20.pdf . Csound5 was then in the latter days of its active life, soon to be superceded by Csound6.

[3] https://audioprograming.wordpress.com/2012/03/16/a-general-purpose-ui-for-csound-on-android/

Subsequently, a basic tutorial on CSD Player, "Introducing the Android CSD Player", was written by Brian Redfern for the Csound Journal: http://www.csounds.com/journal/issue17/android_csd_player.html 

[4] A major limitation was that its only .csd feedback was a general, and somewhat ambiguous, indication of any compile-time error.

[5] http://eprints.maynoothuniversity.ie/4120/1/dafx12_submission_2.pdf

The third coauthor was Joseph Timoney.

[6] Ian McCurdy's Haiku IV.

[7] Serious text editing/.csd development is difficult on mobile devices, especially on smartphones. I've done all my development work on a desktop, with my Android tablets tethered to it via USB cable. Here the device simply appears as an external drive to the computer.

[8] A nagging problem with the CSD Players is that they could not return sliders all the way to zero - at least on my devices.

[9] My live-performance Csound compositions are all listed and available at: http://www.arthunkins.com . They were originally conceived for desktop/laptop computer plus hardware MIDI controller, usually consisting of 8 or more sliders. (Alternate versions then incorporate different GUIs [for presets], as well as diverse platforms and control devices.) The hardware slider/fader - especially the long-throw variety - is my preferred controller; I find it infinitely more intuitive than the less user-friendly rotary knob.

I had previously created a version of Chimeplay, a "sonic environment" that incorporated a variable # of controllers (and samples), for the equally underpowered XO computer. It is a creative activity for children.

ChimePad is included in the diverse collection Michael Gogins assembled as "Csound6AndroidExamples." It is available at: http://www.soundforge.net/projects/csound/files/csound6/Csound6.03/Csound6AndroidExamples.zip . The sample rate is set to 8000 to accommodate even the lowest-powered devices. Before attempting to perform this work, one should study the accompanying ChimePadReadMe.txt. One drawback of generic Android GUIs is that they don't display any specific labels or performance guidelines.

[10] The single advantage to the x/y controller is that it can adjust two parameters simultaneously. This is not possible with other sliders (or buttons) - at least not within the current series of Csound5 and 6 Android apps. It's "one at a time" strictly. This, however, was not an issue for me, as I rarely need to change several parameters simultaneously.

[11] Dave and I had been working on several traditional Android apps involving sound, with my furnishing .csds that did the sound generation. Several of these "consumer apps" of his can be found at: https://play.google.com/store/search?q=zatchu&c=apps .

[12] The 12 buttons were arranged in 3 rows of 4, similar to a typical modestly-priced MIDI control surface. 8 or 9 sliders or rotary knobs are also the norm for such hardware controllers.

The Csound5a GUI also fixed the "slider doesn't return to zero" problem.

[13] Any possible dropout issues could be handled by simply reducing Sample Rate; further see "A Note about Latency and Dropouts" at the end of this article.

[14] Csound6a.apk was based on the Csound6.01 version of Gogins' Csound6.apk. 6.01 had dealt with several bugs of the original 6.00 and its Android port. Our subsequent alternative .apk's continue to incorporate Csound 6.01.

[15] The display of 16 horizontal sliders is indeed possible on smartphones, if only barely. Performance requires great care, and preferably tiny fingers! It is far from ideal.

[16] In "buttons only" mode, the up to 16 buttons could be much larger, and be played more like a rudimentary 4-pads-per-row MIDI drum machine.

The challenge for Android buttons is one of latency of response - a problem discussed under "A Note about Latency and Dropouts" at the end of this article. 

[17] I additionally put out these two works in a version for Mac or PC that uses an Android device solely as wireless MIDI controller. This was accomplished via the TouchOSC app - with custom templates (layouts) as performance GUIs. (See further my article, "Android Devices as MIDI Controllers": http://csounds.com/journal/issue20/android_midi.html .) These versions are included in my list of recent compositions as "Peace be with You and Peace be with You2 for Windows and Mac platforms - alternate editions that enable Android devices as MIDI controller."

[18] During setup, the buttons select performance options that are clearly disclosed on-screen when other than default values are desired. Also, during performance, specific frequencies must be specified via slider prior to these pitches sounding. Both cases necessitate Console feedback.

[19] Care needs to be taken for the single-line console output not to overflow the line. This is particularly critical for smartphones. I have found that with a Font size of "Normal" (default Font style), approximately 33 characters can be displayed on these devices. Seven-inch tablets, on the other hand, accommodate 53 characters.

The print-formatting challenges I faced are well demonstrated in PeacealtAndroid.csd - part of the zip archive, PeacealtAndroid  ( http://arthunkins.com/PeacealtAndroid.zip ). All the printing to screen is done in instrument 1. (For those interested in printing to the Android console, this instrument is worthy of study.)

Since printing is to the console, it is handled much the same way as with traditional platforms. However, given lower-powered processors, efficiency of printing is crucial, as printing interruptions can cause audio glitches. It is important to observe the following principles:
a) Print only when there is an actual change of value;
b) Print an entire line at a time, assembling all values into a single print statement. (Note that only the actual printing interrupts audio, not the calculation and assembly of what is printed);
c) Print at most once per k-cycle. (Global values can be aggregated from multiple instruments.)

[20] This is with Font style set to "Default font."

[21] Those interested in exploring Gogins' implementation of HTML5 within .csds should study his composition Drone-HTML5.csd. It too can be found in his "Csound6AndroidExamples" collection: http://www.soundforge.net/projects/csound/files/csound6/Csound6.03/Csound6AndroidExamples.zip . This HTML5 GUI option is only available in Csound6.apks based on Csound6.03 and later, Though it is currently (5/15) unique to Android Csound, Gogins is in the process of making it available to the mainline platforms.

On a related note, Justin Smith has recently found that custom GUIs (Layouts) created by the TouchOSC app can drive Android Csound6; the apps run simultaneously on a single device. (For TouchOSC, see: http://hexler.net/software/touchosc and https://play.google.com/store/apps/details?id=net.hexler.touchosc_a .) Doing so requires the latest build of Csound6 incorporating Csound 6.05 - the first to include the OSC opcodes. An outstanding feature of this arrangement is that, since only one device is involved, there is no need for any network or USB connection! The primary communication requirement is to specify an OSC Host address (in TouchOSC) of 127.0.0.1, and appropriate outgoing and (sometimes) outgoing Ports (usually 8000 and 9000). Changes to .csds are anywhere from minimal to more complex - basically exchanging chngets for more elaborate implementations of OSClisten.  The formatting and handling of variables passed to Csound via chnget (or MIDI) and OSClisten are significantly different. Two very different conversion challenges - from alternative Android GUIs directly to TouchOSC - can be seen in versions of my compositions "SPIRITUS SANCTUS alt" and "PEACE BE WITH YOU alt" (http://www.arthunkins.com). The differing GUI implementations of each work are presented side by side in explanatory .txt files and their associated .zip archives (which include the TouchOSC Layouts). My conclusion: since I only require sliders and buttons, it seems a lot of additional work in the form of code modification and extra setup just for a fancier GUI.

The above capability requires a fairly recent OS: it works with Android 4.4.2, but not with 4.2.2, for example.

[22] The primary changes involve swapping out MIDI button and slider code for chnget commands - a process already familiar to many Csound users.

[23] https://audioprograming.wordpress.com/2012/12/02/an-update-on-the-latency-issue/

This is an excellent report. The situation doesn't seem to have changed much since Victor wrote in late 2012. Latency remains a significant issue in circumstances where precise triggering is required.

.