Apr 4, 2017

Progress Update April 2017

Big announcement time! I'm abandoning my self-made game engine and starting again in Unity3D. BAM!

It was a very tough decision to make. I've grown quite attached to my codebase in 7 years of tinkering in my spare time. As a learning journey, it is how I learnt C# and simultaneously contains the best and the worst C# code I've ever written. As an idea, abandoning that is bittersweet to say the least.

I actually explored the idea of switching to Unity quite a few years ago, but I found the learning curve to be too much and went running back to my comfort zone with my tail between my legs within a month. In hindsight I probably should have perservered with it, but at the time I was evaluating alternative approaches, so switching back was an acceptable outcome. Unity evaluated? Unsuitable? Great, back to work.

This time around, I don't have that luxury. I've painted myself into a corner and I can't see any other way out. I fear that unless I switch to Unity, I will end up abandoning this project due to lack of motivation. The catalyst for this terribly pessimistic thinking is my experiences trying to implement skinned animation in my game engine. I've followed many tutorials, rewritten the skinning a few times, and I actually got as far as having the skinned model rendering correctly... until I tried to animate it and then it all fell apart. *cue sad music*

Basically there's too many quirks in my setup that means no tutorials are directly transferable to my situation. eg: Different matrix representations of DirectX vs OpenGL. Different coordinate axes for animations vs meshes. YUp vs ZUp representations. Each of these would be easy enough to trial and error my way through, but combined, they have been kicking my butt for a couple of months.

Another factor is SlimDX. Back when I started this project, SlimDX was already starting to wane in popularity.  Then there's the fact I'm developing against DirectX 9. Back in 2010, that was a bit silly, since it was already 6 years old, but now in 2017, that just seems ludicrous! I'd already resigned myself to the fact that I'd eventually have to add DirectX 11 support, but realistically, when was I going to find time to do that?

So I've decided that I should cut my losses and embrace Unity.

The Good

So what am I gaining? A fully featured, mature game engine that is graphically superior to anything I could create. Access to a huge number of useful assets on the Unity asset store. Many useful tutorials on a variety of game development topics that are relevant to my exact situation, and don't require messy transpositions. I'm using Unity, they are using Unity. Profit!

The Bad

What am I losing? Pretty much all of the 90,000 or so lines of C# code I've written over the last 7 years. Actually, that's not entirely true, as Unity scripting can be done in C#, so a proportion of the back end code might be able to be re-used.

The Ugly

Ok, so what am I keeping? All the concepts, all the ideas, all the game mechanics and systems that I tested and tweaked should be able to brought across. Everything that makes it a game, rather than a game-engine.  There's much, much, more of this that's still in my game design workbooks than in code, so that's an advantage I suppose. If I waited another year before switching I'd be losing more work down the line.


I think I've made peace with the fact that I have to let the codebase go, and starting from scratch is actually not such a bad idea. Over the last few years I've slowly been refactoring my game engine to be more Unity-like in its approach anyway, eg: Using components and a similar User Input system.

One way of looking at it is that what I've created thus far was a prototype. A fairly elaborate prototype that I spent way too much time polishing, but a prototype nonetheless. Now that I have access to better tools I can hopefully make this crazy idea a reality. :|

Feb 4, 2017

Development Snapshot 17

Once again, most of the work was graphics related. It's refreshing to be making forward progress on tangible things, rather than back-end stuff nobody sees but me.

The animation system was the first thing I tackled. I was looking forward to making great strides on this item. It took more time than I expected so I had to put off the final parts (eg: BVH importer) until the next snapshot. In theory, the player character is animating properly, I just don't have any way of loading animation files yet.

Debugging Frame Timings

After that I spent a bit of time upgrading some of the debugging information the game spits out about frame timings etc. I was expecting to make a few small changes and call it day, but I went a bit crazy and replaced my old profiler with a much more flexible system that does most of the setup automatically. I celebrated this success by upgrading the frame time visualisation to add some colours to represent different parts of the frame. This should aid in optimisation down the track.

Top secret performance benchmark

The first thing I noticed with the improved frame timing information was that my text renderer is slow, taking up almost 30% of the entire General render pass when there is very little text on the screen. Fortunately, the text renderer is due for an overhaul a few snapshots from now to add support for scrollable content windows. The side effect should be improved performance, which is great.

Waypoint Tweaks

I also improved on the look of the waypoint path from last snapshot, adding a dark border. This doesn't sound like a big deal, but the way I originally wrote the waypoint path creation, this was pretty tricky.

Not too bad. But we can do better...

The border helps the waypoint stand out a bit better from the terrain, and tidies it up a bit in my opinion. I also did away with the triangular shaped notch at the start of each run, and I've had a few ideas for improving it further, but they will have to wait until next snapshot.

The most fun part was writing some geometry code to ensure the width of the border is constant even when approaching the sharp point of the arrow tip. The problem looked like it didn't have an easy solution until I hit upon the idea of using similar triangles, and had great fun optimising the equation until I ended up with something downright elegant (in my opinion):

v = normal1 + normal2
l = (2 * distance) / v.LengthSquared()
result = v * l

There wasn't much on the survival side this time around, but I did alter the poisoning debuff to lower maximum health instead of just eating away at your current health for a period of time.

I also fixed a handful of miscellaneous bugs, most of which have crept in during the last few months.

Next Snapshot

For the next snapshot I have a few fun things planned. First of all, the BVH importer, which should allow me to finally put the animation to bed. Also on the agenda is starting to work on the first biome that the game will support.

My original plan for the start of the game involved the player washing up on a beach after the cruise ship they were on was destroyed. With this in mind I created assets such as Palm Trees, Wooden Barrels, Life preservers etc and set up my test level as a literal sand box.

Now that I've developed the story more, the initial setup has changed drastically.  At the start of the game the player will  be dropped in a Temperate Forest, which couldn't be more different than the acres of sand and a few scattered palm trees I have now. So, in snapshot 18 I'm going to start the ball rolling on the biome generator. It will be somewhat of an experimental feature, but hopefully there will be something interesting to show.

I've also decided on the visual style of the game, which is quite exciting. I'll pretty much have to throw away most of my game assets and start again, but I have so few I'm not that worried.

Jan 12, 2017

Development Snapshot 16

The main issue I ran into this snapshot was inflexibility of old code. I ended up having to revisit alot of code that is over 3 years old. That code was unable to be extended without introducing a horrific mess, so I took the high road and invested some time refactoring those ugly parts. This cost me some time, but the end result was worth it.

The majority of work this time was in the graphics area. I overhauled the look of the UI somewhat, replacing the WindowsXP-like windows with something a little bit more modern (in my opinion). I also rewrote a large section of my render loop in an effort to reduce frequent state changes. Instead of just blasting out draw calls, switching shaders and such on the fly, I now enqueue render jobs that are sorted to minimise changes where possible.

Skinning System

This was the major piece of work this snapshot. The first task was creating a rigged model in Blender.  I took my existing quick and dirty model and threw together a simple bone structure (after I watched a few blender tutorials). This fell together quicker than I anticipated, which perfectly set me up for bitter disappointment when I failed at the next few hurdles....

Next was modifying my collada importer to support bones. My previous implementation was done strangely to say the least. Looking at the code again after several years, I drastically rewrote it to be more flexible, and added the bone/joint data parsing as well. I lost a bit of time debugging why all the transform matrices had no translation component before I finally realised that collada matrices are Column-Major and I was assuming Row-Major. Once I got that fixed, the matrices looked right.

Then I had to rewrite my model class to handle the fact that there will be joints pushing and pulling the vertices all over the place, instead of being static. My previous incarnation of my model class was a hastily slapped together mess that came out of an attempt to get the code working again after one of my aggressive refactoring efforts over the years. I treated it as such and gutted it, also making it work nicely with my new render job system.

I then added in the vertex blending code. To aid debugging other parts of the skinning chain, I decided to implement software tranformation of the vertices, rather than use the vertex shader at this stage. (It's much easier to inspect values if they are in the CPU vs the GPU). The only bit of excitement here was initially my bone indices were off by 1, leading to the following image:

That sweet looking cape is actually his mangled torso.

After fixing that I decided that visualising the bones and joints would make it much easier to debug the vertex blending. I got really messed up by this. No matter what I did, the bones wouldn't render right when moved. I got the joints rendering correctly in their bind pose, but anything else and it would stretch at a weird angle. It took me several days of hair pulling, but the problem turned out to be the order of some matrix multiplications.

I knew I needed to multiply the joint's bind-pose Transform Matrix (BPTM) by the desired transform (Y), (in this case, a simple rotation), but whatever order I used, eg:  Y x BPTM or BPTM x Y, the end result was not right. Then it suddenly occurred to me that the bind-pose transform matrix is actually composed of a rotation and translation, and there was a 3rd place I hadn't tried putting the Transform multiply... in between! So I decomposed BPTM into R x T and put Y in the middle : R x Y x T, and holy crap... Success!

At this point I realised the next step of creating an animation system was going to have to wait until the next snapshot, which was disappointing.  On the plus side, the model's vertices are being transformed as per the skinning equation, and is rendering correctly, which I'm pretty pleased with.


To close out the snapshot I also overhauled the look of the player's waypoints. I'm not completely happy with the look yet, but its a great improvement over the temp graphics that I've had in there for several years now. It also forced me break down the paths into runs of straights and turns, which will be required for smooth walking animations later on.

Original style waypoints

New style waypoints

Next Snapshot

For Snapshot 17, the animation system is high on the list. There's quite a few sub-tasks to tackle though, so I'm not sure if I'll get through them all. The toughest stuff should be behind me, but I'll see what happens.

Dec 22, 2016

Development Snapshot 15

In this latest snapshot my main focus was on further survival-related mechanics, such as food and water borne illness, as well as typical game functionality such as loading and saving the game. The snapshot took a bit longer than anticipated, as I also did a complete refactor on my user input system, which ended up creating some nasty bugs.

The end result is much cleaner and is not bound to the keyboard or mouse devices directly anymore, meaning I can inject faked keyboard or mouse events really easily, which is great for testing.

Food and Water-borne Illness

The main survival mechanic I implemented this time is Food and Water-borne illnesses. Basically, if you don't cook meat before eating or decide to drink from a random puddle you'll probably become sick. The consequences could be dire as this type of illness lasts several hours and during that time your food and water usage rates are increased.

Not quite sure about the wording at this stage.

I've also introduced the ability for effects to debuff the maximum value of stats. This makes for some interesting consequences. Starvation now reduces your maximum health by 20%, and dehydration reduces it by 40%. These two effects can combine (for a net loss of 60%), making you more likely to be killed by pretty much everything.

I like the nuance of the maximum value reduction because it enforces an immediate penalty eg: loss of health when the current value is clamped to the new maximum. For example, Food Poisoning lowers your maximum food by 50%, meaning you'll get the "hungry" debuff. Since your max food was lowered you cannot eat food to remove the debuff. You'll have to wait for the food poisoning to wear off. In this fashion, debuffs can trigger other debuffs causing an escalation of problems.

Another significant change to game mechanics is that health no longer regenerates passively all the time. Health regen now only occurs while you are sleep, which gives a further reason for players to not avoid it for too long.

With these changes in mind, I've grown to dislike the energy stat again. I think I've added and removed it 2 or 3 times since I started the project, and I've convinced myself to get rid of it for good. The Energy stat's role in the player character's economy was mainly to force the player to rest or sleep periodically, and ensure they can't work non stop for hours without consequences. Penalising this behaviour can now be achieved in different ways, so its largely unnecessary.

I'm planning to introduce another stat in the future that model's the player's need for shelter in a survival situation. This stat is related to surviving in cold or very hot conditions. More to come on that in the future.


Saving turned out to be pretty easy. I'd already written most of that code previously when I overhauled my serialisation system a little while back. However, it seems I'd only half-finished the loading code as it turned out to be full of bugs and took quite some time to get under control. But now it is possible to save and load the game state. Hurray!

The game currently works in a partial client/server configuration. Eventually the server will be used by the client for all game logic, but at present it's just responsible for generating the world and then going to sleep. The client then takes over and does all the simulation. The save gamestate code should be run against the server, but it doesn't have the latest information as entity updates are not yet flowing back to the server (perhaps in a future snapshot). So for now the save code runs against the client gamestate.

In theory, having access to save/load should also help debugging, as I don't have to start from a blank canvas every time I want to start up the game.

Onwards to Snapshot 16!

Snapshot 16 is going to feature some graphical overhauls to the game. The most significant of which is starting work on a skeletal animation system, so our poor protagonist can finally put his arms down and walk like a human. 

I'm very excited about this, as it's long overdue. I started building the basics of this a few years ago but abandoned it and moved on to more interesting things. In fact, there are several partially implemented bone/animation related classes tucked away in the source code that haven't been used for several years. I suspect I'll end up throwing most of it away to be honest.

Another upgrade is making the waypoint system work a bit nicer. The main problem is that it only ever displays the waypoints after its done a full plot of the path using A*. So if you decide to change direction midway through a tile movement it can be up to a second before it clears the existing waypoint path, plots a new one, and then draws the new path in the world. This lag between clicking on a tile and a marker appearing there is disconcerting to the player, and shouldn't be too hard to tweak to feel punchier.

On the survival front, I'm planning to implement poisonous foods such as mushrooms or berries. If the player character has a sufficiently high survival skill they should have no trouble identifying which is safe to eat and which is deadly.