My Thoughts on Employee Performance Management Systems

The Poor Performance of Performance Management Systems

I have worked for three large organisations in my career.   In each of these, and in many other organisations, there is a tendency to put a performance management system in place that, on the face of it, ranks individual performance according to their ability to deliver on the goals and objectives of the organisation.  These goals and objectives are normally put into some employee "scorecard" or whatever the workplace calls it.  Once or twice a year there will be a "ranking session" or a "roundtable" where employee performance is discussed and, at the end of this session employees are sorted much in the way sports people are: from most valuable player to least valuable player.  Of course, the employee's input into this ranking session is limited to any performance discussions (if they're lucky) they might have had with their line managers before (and unfortunately sometimes after) this meeting takes place.  While the HR people often assure us that the approach is well researched and will spur the organisation on to greater heights, in my reasoned view this is simply not true.  It is one of the singly most demotivating activities that a workplace can indulge in.  Here's why I hold this view:

People respond to the incentives they're given.

Rewards are tied to having a successful outcome in the ranking sessions. Each individual therefore, tries to optimise the outcome of this for themselves.  It does not take too long to realise that there are two paths available to do this - and they're not mutually exclusive.   

First, you can do a good job.  Great!  That's what it's supposed to encourage.  But, each individual can also optimise their position in the outcome by engaging in behaviours that seek to move their peers further down in the ranking.  And that's just plain evil.  Many workplaces have recognised this so you will often see that performance metrics are 'enhanced' with behavioural measures as well.  And again, prima facie, this seems reasonable.  However, the reality is that minor disagreements between employees are often brought up as reasons why person X should be downgraded in favour of person Y.  So, what does this do? It stifles the ability of employees to challenge a process.  If people no longer feel comfortable challenging a process because of the perceived risk of drawing attention to themselves in a later ranking process performance suffers. We end up in a spiral to mediocrity.  But wasn't it supposed to make us achieve greatness?

In many systems you will see that the top 20% or so of people get rewarded very well, the next 60-70% will get some reward and the bottom 10-20% will get nothing or, in some cases, may even be fired.  The process results in people engaging in behaviours trying to keep themselves out of the bottom quantile.  If they shoot for the moon and miss they won't end up in the stars like Norman Vincent Peale would have us think.  No, they'll drift endlessly in the cold, dead, vacuum of space.

It's actually not about performance

Usually there will be some bell curve rolled out by the HR guys to give some 'theoretical' basis for the process.  But the whole idea is flawed. Have a look at this article that makes the point quite well.  The reality is that most people don't turn up to work every day with the intention of performing poorly.  They want to do a good job.  So, when a bell curve is fit it is no longer measuring their actual performance.  A different distribution would be needed for that.  Instead it's measuring two other things.

A good rating outcome is usually given by a consensus of managers.  If an individual can convince enough of these that they're performing well (irrespective of the reality) they will do better in a ranking session.  The behaviour actually encouraged is lobbying.  Subtle lobbying (and sometimes not so subtle), but lobbying nonetheless.  It's not about what you do but is about how well you can sell what you do.  It rewards spin doctoring.

Additionally a manager will use a ranking session to position him or herself.  If they can have significant proportion of 'high performers' how can the manager be a him or herself be a poor performer? Actual employee performance is subservient to internal politics at increasingly higher levels.

It removes the performance gap between teams. 

In any organisation there are some teams that perform really well and others that just can't seem to co-ordinate themselves.  They may have good people in them but they just cannot seem to get their act together.  Preparing for a ranking session requires that all people leaders rank their people prior to the ranking meeting.  During the meeting there is some process by which the results are blended.  This removes any distinction that naturally exists between teams because a more senior manger cannot afford to have one team stand out from another.  A manager of a poorly performing team is putting their 'best' team members in the same category as the 'best' from better performing teams.  The 'worst' in a high performing team may be a much better performer than the middle ranked people in a low performing team.  Yet they wear the same stigma as a true poor performer.  Managers are therefore not motivated to put in the extra effort to have their team actually 'outperform'.  The incremental reward in simply not worth the incremental effort and they cannot really manage the outcome.

It does not allow for managing expectations.

Ken Blanchard tells us that "Feedback is the breakfast of champions."  It's true.  Give people real, genuine feedback and they appreciate it.  If they've done a good job - tell them.  But also encourage them to do an even better job next time.  If you define where the bar is that becomes the target.  It should always be a little bit higher.  Encourage and support them to be the best they can be and then show them they can be even better than that.  

Tied to this notion of feedback is reward.  If you've followed me this far I think you can probably anticipate the problem here.  If you have a system that does not actually recognise performance how can you make your feedback count?  As a leader this was one of the things I struggled with the most. I'm sure I'm not alone. If you cannot do what you say you will do then all you're left with is silence.  Dr Randy Pausch relates a tale where he's ridden hard by a football coach.  One of the assistant coaches says "coach rode you pretty hard today didn't he?  That's a good thing.  When you're screwing up and nobody's saying anything to you anymore that means they gave up."  He's right.  Those people in you life that can offer genuine, sincere criticism are telling you they still care.  It is a horrible place to be in when you're not performing and all you hear is silence.  But if I cannot make my words count because I cannot guarantee the outcome what can I say?

Knowing how the process works I cannot commit to creating an expectation only to have to turned around by the whole process.  My silence does not mean I stopped caring - it means the organisation stopped caring.  My way of dealing with it was to be open and frank about what I thought about the process with my team.  All I could commit to was that I would fairly represent them to the best of my ability in the process and that I would fight for them if I felt they were being wronged by the process.  I managed their expectations by telling them what I thought of their performance irrespective of what the final outcome might be.  While I did sometimes see frustration with this (and boy, I have felt that same frustration) on balance I was often told that my frankness and honesty was appreciated.

People that are actually performing at a high level are incentivised to leave.

If you've bought into my argument so far then you will recognise that many organisations have a problem.  If the process does not actually measure performance then those that are actually doing a good job but go unrecognised are incentivised to leave.  "But the HR people said this will make us great," I hear faintly from the bleachers.  I'm sorry they will seek out opportunities elsewhere that do recognise their contribution.  They will seek to create their own happiness without you and those that stay will continue in a spiral to a mediocrity that is not of their own doing.  Until, that is, it is their time to leave.


Am I jaded by this system?  No.  I worked the system like anyone would.  But that does not mean that I bought into it.  My values told me I just had to lay the cards on the table with my team.  Give them the truth.  Sure, from a corporate perspective that's probably frowned upon.  The fact is I don't care about that.  Whatever you do you have to be able to go home at night and sleep well.  Even if you know that good people won't get what they actually deserve.

A Sprinkling of Pixel Dust

A Sprinkling of Pixel Dust

OK, so it has been a long time since my last post.  A lot has happened between then and now.  The biggest of which is that I quit my job at a major Australian Bank to undertake a doctorate in mathematical science.  I have to admit that I love the student life.  It lets me work on things I find fascinating and there is not the stress associated with working a full time job in corporate land.

But that's not what I wanted to blog about.  I wanted to share something I have been working on - for fun.  First a little math lesson - I'll try to keep it high level so you can get a appreciation for what I am doing without getting bogged down in too much notation.

A Short Math Lesson

Here is a picture of a puppy.

I know many of us will look at it and say Awwww, but I want to look at this picture as a mathematician would.  At it's most basic this picture is a matrix.  It just so happens that the numbers that make up this matrix represent shades of grey.  That's what the legend on the right is telling us.  We could completely describe this image by knowing all of those numbers and the dimensions of the matrix.  And, because it is a matrix, you can do all sorts of mathy things to it and I am going to describe one of them here.

It just so happens that to get a good approximation of this matrix we don't need to know all of the numbers that sit behind every pixel by using a bit of matrix algebra called the Singular Value Decomposition (SVD).  What this does is break a matrix into three components.  Recombining those components will give you your original matrix back.  Like this:

However, recombining just a portion of those three components, as shown in red below, will give you an approximation of the original matrix.  The greater the portion of those components used the better that approximation.  It works like this:

So, you can capture an approximation of a matrix with a smaller set of numbers than those contained in the original matrix.  That, boys and girls, is how jpeg compression works.  It's why .jpg files are usually smaller than .bmp files.  It's how a rover on Mars can transmit a high definition picture of the Martian surface without sending every pixel.  Let's try this on our puppy matrix.

Left to Right: 2 features, 5 features, 10 features, 20 features, 50 features

As we move from 2 to 50 features the approximation gets better.  In order to recover the full image we would need 300 features but you can see how just 50 features was able to get a good approximation of the image.  It is also worth noting that the higher order features contain less and less information about the matrix.  Put simply, most of the relevant information you need about a matrix is captured by the lower order ("rank" in matrix language) features.  A mathematician would say that the image on the left was a rank 2 approximation and the image on the right was a rank 50 approximation.  Now, as long as a matrix has some structure in it, the the mathematics of the SVD will find it.  The more obvious the structure the fewer features required to describe it.  Images are matrices with structure so it works very well with them but this idea will work just as well on any matrix.

Ok, the math lesson is over.  If you want to know more about the SVD though you can check out a lecture by Dr Gilbert Strang from MIT here

I'm now going to give you a flavour of what I have been playing around with.  Not because its part of my doctoral research, but because it is very cool.

Zombie Matrices

I first wrote the code I am about the describe to use on a Kaggle data set.  I used it on a data challenge being faced by Amazon but, because it took me so long to figure it out, I missed the submission deadline.  This particular problem was one which could be expressed as a sparsely populated matrix.  This sparseness meant that I could not apply the SVD method I described above but I still needed to know what the features of the matrix were so that I could make a good prediction about the missing values.  I needed to bring the matrix back from the dead - and I found that I could with some math voodoo.

After a discussion at a workshop related to my doctoral program I was fired up to revisit this code again.  I wanted to make it less tailored to the Amazon Challenge and give it a more general application.  I'm using images because they contain structure that everyone can see, but remember this can work on any matrix.

I'll start by sending a random 20% of our puppy image to pixel heaven.  Same puppy, but this time he looks like he's in the snow.

So how do we clean this image up?  This is the really cool part.  It turns out that the answer is related to the SVD we discussed earlier.  Instead of deriving the features from the original image as the SVD does I had to write some code to learn the features from those points that were available and then use these to try to recover the original matrix.  My approach is based on (but not exactly the same as) the approach discussed by Simon Funk here.  He used it when competing in the Netflix prize.  My code has a better initialisation approach and a configurable regularisation parameter to help protect against overfitting.

I used my code to train 10 features from the snowy puppy image above and merged the results back into the image.  I got the image below.  Notice how we've got the detail back on his nose, his whiskers are visible again and his fur is more easily discernible.

What if we remove 50% of the data?  Here's the before picture...

...and after there's still enough information left to estimate the original picture.  Again I only trained 10 features and we've still got the detail on his nose and eyelids, we're still able to make out his whiskers and, while it's not as crisp as it was before, we can still make out his fur.  He now looks like an impressionist painting.

Ok, one more time.  Let's blitz 90% of the pixels so there's not much left at all.

After the features of this pixel dust are learned by my code we get something that is clearly recognisable as the dog from the original image at the top, albeit a bit blurry.  I used 10 features again but if more are trained then a better approximation could possibly be obtained.

So what are the applications of zombie matrices?

I have shown that one application is in image processing.  This could potentially be of benefit to law enforcement or enhancing satellite imagery.  I also know from experience that real world data often contains missing values.  For a variety of predictive modelling tasks having a more complete data set is better.

Direct examination of the features extracted gives also meaningful information.  For example in the Netflix challenge it gave groups of customers that have similar taste in movies and groups of movies that had similar attributes.  In the Amazon challenge it gave me system access rules that could be applied to the different job roles at Amazon.  So this approach has a role to play in recommender systems.

My code still needs tweaking so that it runs more efficiently.  That's why I limited the features trained to 10.  For me, writing the code and seeing the results was just fun.