Stay awhile and listen

A couple of days ago, I pushed to Github the first implementation for a new audio subsystem with the intention of providing a set of tools for audio playback, both in 2D and 3D.

Although Crimild does currently provide a simple 3D audio tool, it is far from useful in productive environments, specially because it lacks of any means to play audio streams, as in music.

Decided to work from scratch in the new system, I managed to complete a working version in just a couple of days and it has already proven to be much more useful than the previous one.

Here’s a brief introduction of what I’ve accomplished so far.

Core abstraction

As with many other systems in Crimild, the new new audio subsystem is split into two parts: the core abstraction and a platform dependent implementation.

A handful of new classes have been added to the core library, describing abstract interfaces for audio-related concepts, like audio sources, listeners and the audio manager itself, which also acts as an abstract factory for those objects (something that I’m planning to replicate for images too in the future).

In addition, two new components are now available: one for audio sources and the other for the audio listener (usually attached to a camera). These components will handle audio spatialization if needed (for 3D audio), using the parent node’s transformation.

But how useful is an audio system that doesn’t play audio, right?

SFML

As I was looking for a simple, portable, library that will allow me to implement the actual audio layer in a simple way, I came upon SFML. SFML is an abstraction layer for most of the platform specific functionalities, like window handling, networking and, of course, audio.

SFML provides an API to handle both 2D and 3D sounds and streams, so the implementation was pretty straightforward. Much as with other third-party dependencies (like GLFW), this code is organized in a separate package.

Integration

The unique instance for the Audio Manager is stored by the Simulation class itself, in a similar way as with the graphics renderer or other manager. The GLSimulation class, the one used in desktop projects, is already creating a valid instance upon construction, which means client projects should be able to use the new audio facilities right out of the box. No additional setup is needed.

Finally, In order to avoid confusion, I removed the old OpenAL implementation completely.

Examples

There are two examples that make use of the new audio system.

AudioRoom presents you with three music instruments in the floor. As you approach them, a music clip is played and it is possible to have more than one instrument at the same time if you move the camera quick enough.

Screen Shot 2017-11-05 at 6.43.45 PM

The Drone example has been enhanced with multiple actors. This example uses several audio sources and a listener attached to the camera to showcase how 3D sound spatialization works.

Screen Shot 2017-11-05 at 6.43.33 PM

Both of the examples are already available in the demo project.

Final comments

The new audio system is already available in the development branch, in Github. I’m still working on this code, so expect changes in the near future.

Now, I have to admit that I’m not 100% comfortable with SFML. Don’t get me wrong, the library is great and extremely useful, but it feels a bit overkill in my case. SFML provides so much functionality, not only for audio, yet I’m only using a minimal part of it, mainly because Crimild already depends on GLFW for window and input management. On the other hand, I haven’t check how to use SFML on iOS yet and that might cause some headaches in the future. Also, there’s no support for MP3 at the moment.

But then again, SFML is extremely simple to use and I’m short of time at the moment, so I guess it’s a good compromise. I’m confident that I would be able to easily change the implementation in the future if needed, since the level of abstraction of the core classes is quite good.

Time will tell.

 

Advertisements

Progress Update – November 2016

This post summarizes all the things I’m currently working on, without any specific priority order (as usual).

Job System

Probably the biggest feature to be included in the next release (whenever that happens) is the new Job System.

Last year I implemented a Task Scheduler for asynchronous work and while it is (was?) useful, it does lack the mechanisms required for programs to properly work concurrently. In particular, there’s no way to wait for tasks to complete or group them together as one unit of work.

Enters the new Job System, in which we’re going to be able to schedule jobs (obviously), that may or may not be linked with other jobs (as in parent/child jobs) and wait for them to complete before continuing. A “work stealing” approach is being used internally and we can spawn as many worker threads as we need.

The new Job System will allow us to move to a more parallel architecture by implementing things like parallel visitors, multi-threaded render queues and more.

Surprisingly, the current state of this feature is quite advanced and looking good. I just pushed the main classes to the development branch and I expect to refactor the Simulation flow pretty soon.

Ray Tracer Improvements

I needed a way to test the new Job System before having to refactor the entire Simulation flow and the Ray Tracer seemed to be the perfect candidate.

Initial tests are promising, showing that the rendering time has been reduced by 60% (depending on the number of worker threads, of course). The image below took “only” 80 minutes to render in my Macbook using 8 worker threads.

output_800_600_100

I’ve included several fixes in the latest code and I’m going to move to phase two (actual geometry, lighting, textures, etc) probably before the end of the year.

Scripting

As it is right now, the scripting system is mostly used for scene building, specifying objects and components in script files using Lua. But I want more. I need more. I want to be able to create components from Lua files and interact between them and first class ones written in C++. I want to be able to create a whole new simulation or game without enforcing the developer to write and compile C++ code.

The improved scripting tools are still in a design phase and even when I would like to include them in the next release, I know they might not make it.

Improving Audio Support

This is something that I had in my list for a while and I’m giving it a try as a side project. At the moment, Crimild supports audio clips using OpenAL (including positional audio, of course) only in WAV format and there’s no way to play background music.

My initial goal is to work with OGG files. By the end of phase one I should be able not only to play music files but also to support some simple mixing mechanisms.

I don’t expect this feature to be included in the next release, but it should be done early next year because it’s something that I need for a different, super secret, project.

Scene Streaming

Yes, this is back. You know, that super useful thing let you save and load entire scenes to disk, using a binary file format (which was there five years ago but then disappeared one day without reason).

A working implementation for scene streaming was included in the last version (I’m not kidding, it’s there). I’m constantly revisiting it by adding support for more components and entities. The new binary file format has proven quite good, and loading times have been reduced in a great amount.

That’s it. Well, not really. There might be one or two more things in my TODO list right now, but the ones above are the most important ones. I’m planning on having a new release before the year’s done, including most of these features. So, stay tuned.