Loksim3D Developer Stories – Part 1

If you read the “About” section of this blog, you may know that I am one of the developers of Loksim3D. To be honest, at the moment I am kind of the project’s lead developer. Never really wanted to be it, but – well – it just happened…

Loksim3D is an old – very old – project: The current version relies on a code base, where development was started in 2000. At least the graphics part of the code is that old, could be that some other parts are even older. Loksim currently uses Direct3D7 (yes seven) for 3D graphics. There does no longer exist any useful documentation for DirectX7. At least I cannot find any, if you know one please let me know. So yes, that’s not really up-to-date and therefore one – or *the* – goal for me is to update Loksim’s graphics engine to a more recent version. However, this is not an easy task.

I really learnt some interesting things analysing the existing 3D engine of Loksim, but a lot of things are just outdated and one would write things completely different for today’s hardware and with today’s C++. Additionally the code that interacts with DirectX7 is tightly coupled with the rest of the code.

Therefore I decided to rewrite the complete graphics stack to be able to upgrade the underlying graphics API. Before I started working on this task, the code of Loksim consisted of roughly 240.000 lines of code. It is just an approximation, because there are some libraries in source code included, which are not maintained by ourselves.

I started this task in the late summer of 2013. The first decision I made was to upgrade the whole project from VisualStudio 2010 to VisualStudio 2013 – I enjoy programming C++11 and VisualStudio 2013 offers much more features of C++11. The second decision was to implement all new graphics stuff in a separate VisualStudio project, an own static library. The main reason for this was my intention to set the warning level of the compiler as high as possible. However, in the existing codebase lots of warnings were generated when I tried to set the warning level higher. Therefore I decided to use a new project.

After all this more or less ‘organisational stuff’ I tried to find a strategy for the implementation (and integration) of a completely new graphics engine for Loksim. I identified the following different areas that interact with Direct3D7:
– Rendering of 3D-Objects. These objects are stored in a special XML-based file format (.3dobj and .l3dgrp)
– Rendering of rails. Rails can be seen as 3D objects that are generated by code
– Rendering of scenery/terrain. The terrain in Loksim is defined using an own file format (again XML-based) and heavily depends on the layout of the rail tracks
– Rendering of 2D content. The main application of 2D-rendering is the display of the driver’s cabin, which is only rendered in 2D

After identifying these four areas, I thought about how a migration might work the best. Of course one idea was to implement all areas at once and make a ‘big-bang integration’ afterwards. Mainly because I knew this would take a lot of time, I decided that this is not a desirable solution for me. I just want to release ‘my code’ regularly (in Loksim this means roughly twice a year), because otherwise I would just write code alone at home and would not get any feedback from users.

So I decided to start with 3D-objects only (I named this task ‘objectsystem’) and try to get a Direct3D7 version running again after the implementation of the required code parts of this area. I should add, that I did not want to change the rendering of the objects alone, I wanted to change almost everything in code that deals with 3D-objects. I already mentioned that Loksim objects are stored in XML files. These XML files were all parsed using a handwritten parser. I think this decision was made 14 years ago, because no usable (free) XML-parser existed at the time. However, this handwritten parser of course does not work ideally and nowadays nobody would think about writing his own XML parser. So using this nice Stackoverflow flowchart I decided to implement the I/O of 3D objects using PugiXML

Additionally I wanted to make life for 3D object authors easier:
– I wanted to extend the special Loksim file formats: ‘Loksim objects’ (.l3dobj) that consist of points and areas and ‘Loksim group objects’ (.l3dgrp) that consist of ‘Loksim objects’. However, a group object itself could not contain another group object. The new objectsystem should be able to deal with group objects that contain other group objects
– In Loksim objects the normal vectors (required for proper lighting) needed to be set for each point. If the same point (coordinate) is used in two or more areas, most of the time a separate point for each area had to be created. Otherwise the lighting of the object did not work very well. The new objectsystem should be able to store normal vectors for each area, which makes object autor’s life easier most of the time. Additionally this allows the automatic generation of normal vectors. Of course, defining normal vectors per points should still be supported if the object requires it (e.g. round objects)
– I wanted to support other 3D object file formats then the simple Loksim formats. I decided to use Assimp to support a bunch of different 3D model formats. It is really easy to use and supports much more features than Loksim itself can handle at the moment. So of course only a subset of all possibilities in these ‘external’ 3D model formats is currently supported, nevertheless object authors can use these formats as long as they think about what Loksim’s graphics engine currently supports.

Based on all these design goals I had to rewrite just everything that somehow deals with 3D objects: I/O, editing in the editor, converting to an internal mesh format, organisation in a tree structure, rendering, detection of visibility and so on

I published the first (somehow) working version of these changes internally (just for beta testers) in the end of April 2014. It took six further months of polishing, improvements, bug fixing, lots of cursing until a new Loksim version containing these complete revision of so an important part of the simulation could be published in the beginning of November 2014. Of course time is more than relative in this case: I do all this work in my free time: Sometimes I have more of it, sometimes not so much. Sometimes I am very motivated coding for Loksim, other times I prefer doing other stuff. And some times it is summer where a beer with friends is much more enjoyable than sitting in front of the computer in the evening 😉

Loksim3D 2.9

However, now the new version is out and hopefully users enjoy the now possibilities. Some stats of the new version: Loksim 2.9 now consists of around 265.000 lines of code. Around 20.000 lines of the new code is mainly focused on the 3D objectsystem, the rest of the roughly 5.000 other new lines is mostly ‘glue code’ to connect the existing code parts with the new objectsystem and some minor other new features.

In the future I will hopefully have some time to share more insights on the changes I made for version 2.9. In the meantime I am already working on a new ‘rail track system’, which will be my next step to achieve the goal of new graphics engine for Loksim

This entry was posted in Development and tagged , . Bookmark the permalink.