Progress Update – November 2016

This post summarizes all the things I’m currently working on, without any specific priority order (as usual).

Job System

Probably the biggest feature to be included in the next release (whenever that happens) is the new Job System.

Last year I implemented a Task Scheduler for asynchronous work and while it is (was?) useful, it does lack the mechanisms required for programs to properly work concurrently. In particular, there’s no way to wait for tasks to complete or group them together as one unit of work.

Enters the new Job System, in which we’re going to be able to schedule jobs (obviously), that may or may not be linked with other jobs (as in parent/child jobs) and wait for them to complete before continuing. A “work stealing” approach is being used internally and we can spawn as many worker threads as we need.

The new Job System will allow us to move to a more parallel architecture by implementing things like parallel visitors, multi-threaded render queues and more.

Surprisingly, the current state of this feature is quite advanced and looking good. I just pushed the main classes to the development branch and I expect to refactor the Simulation flow pretty soon.

Ray Tracer Improvements

I needed a way to test the new Job System before having to refactor the entire Simulation flow and the Ray Tracer seemed to be the perfect candidate.

Initial tests are promising, showing that the rendering time has been reduced by 60% (depending on the number of worker threads, of course). The image below took “only” 80 minutes to render in my Macbook using 8 worker threads.

output_800_600_100

I’ve included several fixes in the latest code and I’m going to move to phase two (actual geometry, lighting, textures, etc) probably before the end of the year.

Scripting

As it is right now, the scripting system is mostly used for scene building, specifying objects and components in script files using Lua. But I want more. I need more. I want to be able to create components from Lua files and interact between them and first class ones written in C++. I want to be able to create a whole new simulation or game without enforcing the developer to write and compile C++ code.

The improved scripting tools are still in a design phase and even when I would like to include them in the next release, I know they might not make it.

Improving Audio Support

This is something that I had in my list for a while and I’m giving it a try as a side project. At the moment, Crimild supports audio clips using OpenAL (including positional audio, of course) only in WAV format and there’s no way to play background music.

My initial goal is to work with OGG files. By the end of phase one I should be able not only to play music files but also to support some simple mixing mechanisms.

I don’t expect this feature to be included in the next release, but it should be done early next year because it’s something that I need for a different, super secret, project.

Scene Streaming

Yes, this is back. You know, that super useful thing let you save and load entire scenes to disk, using a binary file format (which was there five years ago but then disappeared one day without reason).

A working implementation for scene streaming was included in the last version (I’m not kidding, it’s there). I’m constantly revisiting it by adding support for more components and entities. The new binary file format has proven quite good, and loading times have been reduced in a great amount.

That’s it. Well, not really. There might be one or two more things in my TODO list right now, but the ones above are the most important ones. I’m planning on having a new release before the year’s done, including most of these features. So, stay tuned.

 

Crimild v4.2 is Live!

Finally! Crimild v4.2 has just being published

This marks the second release this year which means I earned some sort of achievement, right?

This new version includes a lot of new features and changes like Metal support in iOS/tvOS, skeletal animation, math improvements and streaming (yes, that’s back too), just to name a few. As usual, the API changed quite a bit to reflect these new features and some other ones have been deprecated or removed.

Download the new version here or clone the master branch.

Check out the full release notes here

  • Math
    • Return best match in intersection test
    • Implemented pow function in numeric
    • Implemented times function in Matrix and Vector to perform element-wise multipliciation
    • Added method in Ray class to compute interpolated point in ray
    • Fixed polynomial root solver bug
    • Added helper constants in Vector class for known vectors
    • Improved numerical methods
    • Fixed Quaternion SLERP interpolation
    • Allow transformations to be created form a 4×4 matrix
  • Core
    • Implemented NonCopyable classes (replacing CRIMILD_DISALLOW_COPY_AND_ASSIGN macro)
    • Implemented simple low-overhead RTTI facilities
    • Improved SkinnedMesh animation
    • Implemented streaming for objects in a scene
    • Fixed concurrency problems with render queues.
    • Fixed cloning for skinned mesh
    • Fixed FileSystem base directory bug
    • Check if message dispatcher instances are valid before invoking them
    • Implemented ArcPrimitive
    • Save skinned mesh (if any) when updating render state for a scene
    • Avoid discarding groups on culling pass.
    • New particle system
    • Implemented progress callback for animations. Fixed time for animation reset
  • Rendering
    • Implemented StandardRenderPass
    • Deprecated ForwardRenderPass
    • Deprecated BasicRenderPass
    • Implemented PostRenderPass for post-processing
    • Moved render passes to their own folder
    • Implemented ColorMaskState for occluders
    • Create and configure the screen buffer as early as possible
    • ForwardRenderPass will compute shadow maps in all platforms
    • Fixed shadow map shaders
    • Generate auxiliary FBOs on Renderer setup and store them in asset cache
    • Use GLEW from sources instead of as a submodule
    • Fixed directional lighting rendering issue
    • Improved debug render tools
    • (Experimental) Metal support on iOS/tvOS
  • Simulation
    • Clear AssetManager when stopping a simulation
    • Improved deinitialization for simulation and subsystems
    • Implemented axis state in input facilities
  • Scripting
    • Enable/Disable verbose logging for scripting from CMake
    • Improved light support on lua scene builder
    • Build new particle system from Lua script
    • Enhanced Lua builder to support physical entities
  • Audio
    • No changes
  • Physics
    • Fixed ground check for rigid bodies
    • Implemented function to detect ground collisions in rigid bodies
    • Implemented standard colliders for physical objects
    • Improved rigid body configuration for physical objects
    • Allowed to apply linear velocity to bodies during simulation
  • Testing
    • Fixed existing tests
    • Implement new tests for streaming
    • Modified CMake scripts to include GTest cases

 

Color masks and occluders

One of the newest features that will be included in the next major release for Crimild (coming soon) is the support for color masks and invisible occluders for our scenes.

Occluders are objects that block the visibility path, fully or partially hiding whatever is beyond them. An invisible occluder behaves in the same way, and while the object itself is not drawn it still prevents objects behind it to be rendered.

For example, in the following scene the teapot (in yellow) is an occluder. Other objects are orbiting around it and the scene is rendered normally.

screen-shot-2016-09-28-at-12-34-10-pm

Original scene with color mask enabled for all channels

By playing around with the color mask and turning it off for all channels we can avoid the teapot itself for being drawn, yet it still blocks the objects that are passing behind it. The green plane is not being affected by this behavior (that’s on purpose).

The effect is a pretty cool one and has a lot of applications in games and simulations. It’s specially useful in augmented reality to mix real-life objects with virtual ones.