How Agile Failed Us

Well, actually, Agile wasn’t to blame, it’s an obviously internet search tuned headline.  We broke it, and I’ll attempt to show how:

My team is trying something different.  For a few years now, we/they have been doing Scrum.  As in all successful agile implementations, it’s been tweaked to fit our workflow and calling it Scrum now is more a habit than a true description.  And it is working for us …mostly.  We’ve just embarked on something new and I thought I’d give a bit of a description of where we’ve come from and where we’re going.  Hopefully it’ll resonate with anyone who isn’t on my team reading this – especially the bits that I now see as obviously brain-dead!

Read more of this post

The Serial Killer Tooltip

Firstly, having seen in my statistics a search term used to find my last post, I feel I should point out how I came upon the name for my Serial Port Terminal Application.  I was poking fun at it being a “Killer App” (because really, if there was any software which proves Serial Comms as a technology, it was written years ago).  Combine that with Serial and you have a terrible pun, which seems to be one of the most important things of Free and Open Source Software.  This is not a post about the other type of serial killer.  Or indeed, a full time one.


For such a small feature of my program, the tooltip which shows up the timestamp of the time data was sent or received has caused me more consternation that probably the rest of the program put together.  I had to touch each of the three layers of my model-view-controller, with the smallest modifications going to the controller, which is odd – in general, the bulk of the functionality of an app should live in the controller.  So here’s how I did it…

After thinking about the UI briefly, taking my existing GTK+ knowledge, I decided it was going to be best to start with the model.  When I first started writing this app, I had imagined a feature like this and broke the YAGNI rule.  I had previously included a timestamp in the data I was collecting about the data to-ing and fro-ing.  My previous code was exhibiting a massive performance problem with the linked-list I was using, in that it was grinding to a halt whenever I clicked the switch to hex mode button.  I therefore decided that if I wanted to search this thing with decent performance, I would need to step my storage up a notch.  For this, I chose to use an in-memory sqlite database.  At work I’m already getting into a SQL head space, I’ve worked with sqlite before, and of course, its license terms make life easy for including in my liberally-licensed program.

Create a table, drop in the data as it arrives and pull it out in the most straight-forward manner imaginable.  I was worried that it would be a performance bottleneck, but I wanted to let the database prove itself otherwise.  Turns out it was good enough.  No reason to get fancy until the simplest thing isn’t possible.

Now for the interesting bit.  How do I pop up an arbitrary tooltip on the same widget, based on the text underneath the mouse?  This took a bit of doing.

First things first, we need to know the position of the mouse.  I attached to the MotionNotify event on the TextView, so now every time the mouse moves I store the X,Y coordinates for later.  As a proof-of-concept, I set the tooltip text during the MotionNotify event and voila, I had an arbitrary tooltip based on the mouse cursor position.  It was starting to be shaped like the actual feature.  I didn’t really want to store it myself, and I was kinda hoping that I would find an event with the mouse X and Y coordinates attached, but I’m not that lucky.

After several mis-fires and some failed attempts at googling, I eventually came upon the QueryToolTip event.  I think I tried this one pretty early on, but it turns out you need to set “HasToolTip = True” on the widget for which you’re hoping it will be called and you need to set the args.RetVal = True or nothing will show up.  Tricksy Hobbitses.

_textView.HasTooltip = true;
_textView.QueryTooltip += _textView_QueryTooltip;
args.Tooltip.Text = string.Format("X:{0},Y:{1}", mouseX, mouseY);
args.RetVal = true;

The next problem is knowing exactly what piece of text is underneath the mouse cursor.  We know the window co-ordinates, but that doesn’t help.  For a second there, I thought I was going to have to do some stupid math based on the font size, and the buffer lines, which would almost certainly turn out to be wrong or not accurate enough in all but the simplest of cases.  Luckily, it’s a common enough thing for GTK+ to have support for it built it:

int x, y;
_textView.WindowToBufferCoords(TextWindowType.Text, (int) mouseX, (int) mouseY, out x, out y);
var bufferIndex = _textView.GetIterAtLocation(x, y).Offset;

So that gives us the buffer index – that’s a solid start!  But it only tells us the character, not the timestamp associated with it.  So it’s time to revisit the database.  Every time we add data, we now need to know whereabouts in the buffer it sits.  So the simplest thing possible is to keep a running total of the buffer and add the length of the data whenever we dump data in the database.

It is now time to explain what I meant by the comment “My previous code was exhibiting a massive performance problem”.  When I started attempting to pull the timestamps out, I was either getting results that were wildly wrong or exceptions were being thrown with no clear reason.  The database decision really came into its own at that point, as I was able to just save the whole thing to a file and dissect it.  As soon as I opened it up and did a SELECT *, it became clear to me that I was storing waaay too much data.  I was storing the whole 256 byte buffer, regardless of how much data was actually sent.  I was storing the length of the buffer, rather than passing up the number of bytes I actually received.  Whoops.  Turns out it wasn’t a performance problem or a hex conversion problem or any of the other things I had considered.  I was just doing something dumb.  A quick fix and my performance problem disappeared, along with the conversion issues.  I was quite excited about that!  To the point where I re-instated the sent / received tag and the performance was still acceptable.  Great Success!!

So to round the feature out, it was a completely obvious SQL query and performance appears good enough.

DateTime result = DateTime.MinValue;
using(SqliteCommand dbcmd = _dbConnection.CreateCommand())
   dbcmd.CommandText = string.Format("SELECT time FROM SerialData WHERE " +
                                     "bufferIndexStart <= '{0}' AND bufferIndexEnd >= '{0}';", bufferIndex);
   var reader = dbcmd.ExecuteReader(System.Data.CommandBehavior.SingleResult);
   if (reader.Read())
      result = DateTime.FromBinary(reader.GetInt64(0)).ToLocalTime();

In the end, it was actually really obvious, but it took a bunch of google mis-fires to get there.  I wanted to add a scathing remark about the GTK documentation, but when I search there now, I find exactly the information I needed, including the caveats I mentioned above.  I still want to blame them somehow, but I haven’t figured out how I can do that successfully.

The Serial Killer

A little over a year ago, I decided to write my own Serial Terminal.  Most people’s reaction to this is “why?”  Serial comms is a technology well on its way out, and what they see as the critical hit, a bunch of terminal programs already exist.  Many of which are even open source.  So why would I bother?

Read more of this post

Software is like Cooking

In a previous post I mentioned that software development is a creative process.  In this post, I’m going to expand upon that a little, add some metaphors and later, if I’m feeling wild, I might even add a simile or two!  I’m sure F. Scott Fitzgerald is turning in his grave, given that I knowingly “laughed at my own joke” then.

 Coding is like cooking.  It is important that the flavours blend together to look and taste like they are part of a single entity.   It’s not really ideal to find a whole lemon buried inside a cake.  Or a stew.  Sadly, I see this kind of thing all too often in coding.

The recipe calls for a hint of lemon, so Engineers being Engineers, think “if a hint of lemon is a good idea, then think how awesome it would be if we had a whole fruit in there, from which we could squeeze the right amount of lemon juice from whenever we wanted.”  At which point there is now a lemon bolted on the side of your mixer and you find that when you try and bake a different variety of cake, the machinery forces you to have lemon in there.  Or more likely, it is a multi-flavoured juice dispenser, which is so heavy the mixer can barely stand up on its own any more.  And it leaks.

Which doesn’t at all lead me to my next point, so I’ll just jump to it awkwardly.  Something I find myself doing quite a bit is cooking blind.  Which is ironic, because sight is the main sense I actually employ when doing this.  What I mean by that, is that I don’t taste my food before serving.  It’s a bit inexact, and I admit, also somewhat hit and miss.  And largely, when it’s a hit, it’s due more to experience of knowing the right colours, textures and combinations of flavours that work.
So I guess what I’m saying with this post is dressed up prettily, but blindingly obvious.  Experience mitigates some of the worst sins, but if watching altogether far too many cooking shows on telly has taught me anything at all, it’s that life is infinitely better if one tastes food before it is served.

Reviewing review

Code review is something that nearly all software developers consider to be a good thing. Though frequently when I’m doing it, I feel out of my depth.  This is usually when I’m learning the technology or codebase.  And recently I’ve been reviewing not just an environment in which I am comfortable (a desktop application) but also a web application, which covers the whole stack right from database, webservice, client and javascript-based webpage.  My experience to date has been in embedded software, desktop configuration software, a network-based build server and a smattering of almost static html, so I’m feeling somewhat out of my depth when asked to cast an eye over a SQL stored procedure.
Read more of this post

Yours, sincerely embarrassed

Last week a ghost of christmas coding past came to haunt me.  Some code that I wrote two years ago was found to be broken.  I’m not surprised.  When I first heard about the issue, despite the fact that I no longer work in that team (or indeed on that code base, language or even CPU instruction set) one of the first things I did was pull up the code to take a look.

This particular area of code is quite neglected and I knew for a fact that other people had only touched it in passing, without needing to make the modifications needed for real understanding.  I’m not saying that other people couldn’t understand it, but as a release was due to happen the following day and that, despite the bug being two years old, when the symptoms are described, it sounds quite scary and speed was going to be important for fixing it.  About 1 in 20 times our security handshake would fail.  The legitimate user would be temporarily unable to configure the product.  Because of this, an impending release (due to be finalised the following day) would be put at risk.  So my experience in this area meant that I could immediately narrow the bug to two or three files in the whole repository and I knew more or less how they fitted together.  Speed boost, achieved.

Read more of this post

The Bad Time

Well.  I’m nearly five (I’m secretly waiting until I can start quoting A.A.Milne!) but the inevitable has finally happened.  My workplace is going through some “structural changes”, shall we say.  For the first time in my career, people I know and respect are losing their jobs around me.  I don’t want to go into my opinions of the specifics of the decision as I don’t have the full picture.  I certainly don’t have the whole set of numbers nor the strategic view.  But rather, I want to go into the emotional side of things.  Because actually, it’s quite a scary time for everyone involved.

Many people have been saying that the writing has been on the wall for a long time.  Certainly the sales data sent through on the internal financial reports haven’t been painting a rosy picture at all.  Given that this is my first time experiencing this level of staff redundancies, and ones so close to home, I can’t say with any kind of truthfulness, that I saw this coming.  I honestly believed that we would weather the storm and that because we are bidding for several large contracts at the moment, that we’d all be breathing a big sigh of relief sooner, rather than later, that we’d be in good shape again.  Apparently not.

Read more of this post