I can see the beauty of mathematics…You can hold infinity in your hands

After some consideration I’ve decided to revive and rework a product I wrote about 5 years ago both for commercial reasons, and as a redesign and architecture exercise. I thought writing about my experience spelunking into my old code might be useful, so I’ve decided to document my self humiliation for posterity. Part 1 of this series will focus on digesting and assessing code I wrote five plus years ago as a much less experienced developer and planning an approach to attack modernizing the product, and getting it production ready again.

Background:

This product was a great test bed for assessing my overall skill improvement. The original came in five major parts.

1. Interaction with a native C++ driver in win32 for multiple input devices and displays
2. A multi-threaded near-real time data processing engine
3. A gui program responsive on multiple screens.
4. A data recording system based on MySQL that performed analysis and export.
5. A plugin system for adding new analysis engines to the program after installation.

The original system was written using C#, along with a managed C++ wrapper to the native win32 driver dll for the touch screen devices. It also consumed LGPL libraries for generating excel spreadsheets and interacting with a MYSQL database.

Initial Breakdown:

This code stands up surprisingly well for 5 years of disrepair. I have to credit Microsoft, other than the outdated installer code and a couple of broken reference links it translated directly from Visual Studio 2010 to Visual Studio 2015 without a hitch, including jumping 3 minor versions of .NET. Overall I am relatively pleased with my design choices in the original version. The code is pretty readable and I followed good modularity patterns so the various pieces of the code are kept fairly discretely. My early work to divide the modules properly is going to reap dividends as I should be able to do the rewrite piece by piece if I conform to the original interfaces.

On the downside, I was handicapped at the time both by my ignorance of C#’s concurrency mechanisms at the time, and the relatively new presence of async/await. I hand rolled all the code directly using the System.Threading.Thread interface rather than dispatching the work out to a thread pool, and thus I ended up manually using things like reset-events and concurrent queues to handle multithreading at a much lower level of abstraction than I’d use today. I’ll re-write and modernize this to take advantage of the better concurrency primitives in F#, and use the mailbox processors to represent the linearization of events for each screen rather than hand rolling it with queues like I did before

Also, lets just get it out there. The GUI was just plain atrocious looking. I’m going to take an initial stab at redesigning it, but might enlist the services of a better designer this around. I used windows forms, which was the mature choice at the time. However, because of the limitations of windows forms application my application could only actually handle one cursor any screen at any time. This meant I had to do an ugly hack and store the cursor position on the “main” screen and recenter it after any touch events on the auxiliary screens, which is way less than optimal. Now that WPF has matured, I’ll probably invest in redesigning the GUI for that framework which will allow me to more cleanly separate the input events from the various devices and cleanup the command and control code considerably.

Other things I notice. The analysis package in C# is pretty hackish. I’d like to rewrite it in F# for clarity, and consider exposing a read-only API directly from the application to R/Python/Matlab as an export so I don’t have to constantly hand-roll statistical analysis into the program. This will allow for better integration with my target customers workflow and save me from having to do a custom analysis engine for every target.

Plan of Attack.

1.  Pull the latest drivers for the hardware I’m using and re-write a more mature, and tested version of the managed wrapper. I’ll probably open source this, as I see no reason not to.
2. Re-write the core processing engine in F# with more clean abstractions for concurrency
3. Re-write the data storage engine to use SQL-Lite, rather than hardcoding the data model into MySQL.
4. Add plugins for writing the data to mongo, mysql, and flat export to json/s3
5. Rewrite the GUI in WPF with multi-cursor support.
6. Add a data analysis API for processing the export data in R/Python directly, as well as modernizing the .NET plugin architecture
7. (Maybe) repackage the installer and set up for delivery over the internet.