Tools For Project Managers

This was first published on the blog at my other business: Objectivity

I was asked recently: “What are the best tools for project managers?”. Of course, it’s an impossible question as there are so many tools and so many problems that need solving. It all depends on the context.

However, I’ll make the case for a free suite of tools by explaining how a team I was in used them in a critical situation. The suite is google docs and hangouts (instant messaging, voice and video calling).

For around 12 months, we (about 60 of us) had been developing a critical change for our long standing client (a very well known high street retailer). Deployment was time critical because there was an organisation change facilitated by our software and the comms had all gone out over the preceding months.

The weekend of deployment came and there were about 30 people who had a role to play. We set up a big hangout and everyone was on it. As each step was completed, progress was updated and there was quite a bit of chit chat with all our team. We’d done a number of dry runs so we were pretty confident of the overall process.

I checked the hangout at 2.00 in the morning and saw 10 people actively chatting about a complication. It was becoming increasingly clear that it was quite serious. Another 10 minutes and the PM initiated a conference call. The remaining experts were woken from their slumber and joined the hangout. Over the next hour, the PM directed the recovery. Small groups were requested to join the voice call via the hangout and then despatched to do further investigations and they reported back via the hangout; joining the voice call again as needed. The recovery plan formed and the PM created a google doc and we shared everyone in. The voice call grew to about 20 people.

There was a critical decision to take and we needed to brief the client (currently asleep!), give them our recommendation and get their decision. The PM took control of editing the document and he summarised the situation as we all watched. Corrections and elaborations were agreed and after 15 minutes we had: The plan, the briefing document, the decision that was required and our recommendation all succinctly written and approved by the 20 or so experts who were involved.

The PM rang the client and added them to the conference call. We all listened (on mute) as he explained the situation. Via the hangout, we fed back to the PM when we thought there was a misunderstanding or a point needed more emphasis. Gradually, the client understood the situation, his options and the pros and cons for each. He agreed with our recommendation and the communication plan for the rest of the night. The email confirmation was already written and was a pdf of the document we’d all collaborated on.

The plan was executed, the deployment recovered and all the systems opened up in the morning in good shape. Job done!

Sometimes it’s not the specialist tools that are the most useful for a PM.

Performance Testing for BI

This was first published on the blog at my other business: Objectivity

A while ago, we needed to run a significant number of performance tests for a BI application written on the Microsoft stack. The non functional requirements had been a bit neglected during the early stages of the project and when there was a change in the clients business it sharpened our focus and the number of concurrent users was to increase from c100 to c1200. Also some new key processes were coming into scope. Whilst the functionality remained the same, performance needed to be very quick as a dedicated desktop application was being decommissioned and the results provided by the BI system instead…

We needed to create certainty that some key reports would give sub second response at the same time as 1200 users were using the system in the normal way.

So what did we do?

Firstly we gathered log stats on the dying and existing systems so that we knew the usage patterns. Then we ran workshops with users to understand how the usage was going to increase after the business change. Finally, we created automated scenarios using Jmeter and designed the Perfmon metric collection so we could dissect the situation from a user and a hardware / software stack point of view.

Previous experience had shown us that the volume of data thrown out by these large scale runs is huge and if you can’t get a simultaneous visualisation of the Jmeter and Perfmon stats then analysis, diagnosis and action planning is frustratingly difficult and slow.

The first runs got to about 500 concurrent users before the system became unstable and by closely inspecting the stats at the point of inflection we created our first list of action items. Then followed a series of iterations where we tuned the software, tweaked the configuration and added more hardware until 1200 concurrent users could successfully pass all the non functional at the same time as the key reports could be run with sub second responses.

Of course it was a bit fraught along the way; My description above doesn’t really convey the lows and highs of the team when planned improvements didn’t yield the success we’d hoped for or small tweaks created a massive effect.

I was reminded once more of the difficulty of predicting the behaviour of complex systems and was grateful for lessons I learned 25 years ago from The Goal (Goldratt and Cox) about the nature of systems that are constrained.

Even though we’d run large scale performance testing before, one of the things that still managed to surprise us was the amount of data thrown out of the measurement framework. The length of time needed to do the analysis / diagnosis was a huge problem to everyone.

In fact, we then decided to look at tooling to see how we could speed up the process and evaluated a number of off the shelf tools. In short we found those to be too simplistic or too expensive with a steep learning curve. We created a lightweight framework with all data stored in a repository and the visualisation achieved through a customised Excel UI. But that’s another story!

If you’d like to know more about the journey we took to make this large BI System perform with a huge increase in the number of users and a tough sharpening of the necessary response times, please give us a call.