Why Evaluation in Public Procurement is Important
If the phrase “bid evaluation” makes you cringe, don’t worry: you’re not alone. If you don’t approach it the right way, it can be a huge hassle. That being said, it’s no secret that evaluation in public procurement is an important issue. Not convinced? It can lead to a lead to a variety of challenges from unsuccessful suppliers, which means plenty of unnecessary headaches for you down the line. Luckily, we’re going to walk through everything you need to know about moderating the scores of bid evaluators and handling moderation successfully.
So put down that bottle of aspirin! We’ve got a lot to cover:
- The benefits of having multiple evaluators
- How moderation helps you reach your goals
- 3 secrets that will help you moderate like a pro
Everything’s better together, and that includes tender evaluation.
Why Have Multiple Evaluators?
The first point to discuss is why it makes sense in many case to have multiple evaluators. It may seem like a pretty straightforward question, but it’s actually a more interesting one that it first appears. An Authority might need specialists for different sections of the tender evaluation– for example, an accountant to look at finances, and IT security expert (if relevant) to examine that section, etc.– so why not just have an “expert for each section? Doing that, the need for moderation disappears.
There are three reasons why in most cases multiple evaluators is a smart move. In our opinion, they are persuasive reasons to use multiple evaluators for all except the very smallest and simplest tendering processes.
- More opinions add value
- More opinions balance each other out
- More opinions means more stakeholder buy-in
1. When it comes to opinions, more is more.
Let’s consider for a moment the question of whether you want to use an internal or external evaluator. Here’s a quick quiz: which of the following is a benefit to having an internal evaluator?
- Familiarity with the work
- More willing to speak up with opinions
- Mitigating evaluation costs
That was a trick question. All of the above! Similarly, there are plenty of benefits to having an external evaluator (including increased objectivity and bringing new skills to the table). So which evaluator would you choose? Although we might try to make the process as objective as possible, evaluators inevitably bring their own views, experiences and ideas to the table. Getting the benefit of multiple inputs from a combination of internal and external evaluators can be worthwhile and lead to a better understanding of the bidder’s proposals, providing more accurate assessment and bid-scoring overall.
2. Checks and balances are built into the evaluation.
When you hear about bias (or even corruption) wreaking havoc in an evaluation, it’s almost always an evaluation carried out by one person. That’s no coincidence. It seems pretty clear that if one individual holds the power to determine the winning tender bid, the danger of bribery or other corrupt goings-on interfering are much greater. Even if there is no overt corruption involved, one person may have some biases against one supplier– perhaps an unhappy experience in the past– that affects their ability to make the best decision about the current situation. Involving multiple evaluators helps keep these dangers in check.
3. Keep the stakeholders happy!
This is the most subtle of the three factors we’re considering here, but it’s still very much important: having multiple evaluators can help get buy-in to the contracting decision. For most major contracts, there will be multiple internal stakeholders who need to work with the supplier once the contract is live. I have always found that those stakeholders are more positive towards the supplier if they feel they (or their close colleagues) have had a significant input into the choice of supplier.
A moderator is an essential part of bid evaluation in public procurement for a number of reasons.
Now that we’re all on the same page regarding the benefit of having multiple evaluators, it’s time to tackle our next question: why go through the moderation process– that is, discussing individual views to arrive at a single score for each bidder or evaluation criterion– in the first place? The alternative is to simply add up and/ or average the marks– so for example if three evaluators award 2, 3 and 5 out of a maximum of 5 for a particular question, we average that as 3.333.
The most convincing answer is highlighted by that simple example. Let us assume that a mark of 2 indicates a proposal that is poor and does not meet all the requirements in that area. A mark of 5, then, is clearly is highly satisfactory and may well offer additional value above the core requirements.
We just cannot know– simply averaging the scores is not a logical way to proceed. That’s why we would strongly suggest the moderation process is needed to obtain the most accurate and reasonable score.
Why do evaluators come up with different scores (and having chaired many evaluation panels, they do – not as often as some might think, but it does happen)? They may have misinterpreted the marking scheme. They may have brought their own bias or past experience to the table – “I know this supplier is strong in this area, even if they haven’t written a very good answer”. Or they may have missed or misunderstood part of the proposal, particularly in complex cases – that is probably the most common reason for discrepancies in the marketing. A quick discussion often leads to an evaluator saying “oh, I see what you mean” and happily changing their proposed score.
So we have established today that appointing multiple evaluators is usually a sensible approach, and a moderation process is also sensible. We will now look at how to make that moderation as successful – and watertight from a challenge point of view – as possible.
It can be daunting to jump into the public procurement bid score moderation process. Follow these 3 tips to make moderating easier than you ever thought possible.
How to Execute the Moderation Process
With that out of the way, we can really get into the good stuff: simple tips you can use to be the most effective and successful moderator you can be. I have been a part of many evaluations and chaired even more evaluation panels and boards over the years. It might be tempting fate to say this, but I’ve never suffered from successful challenge from a bidder to any decision I’ve had a hand in making. Finger’s crossed!
How have I achieved this? Aside from, I’m sure, some good luck, I follow a pretty simple strategy:
- The chair/ facilitator should not score the proposals
- There should always be a detailed record of the decision process
- The audit trail needs to be diligently managed
1. Bring some objectivity to the moderation process with an independent chair.
It may not be feasible for the smallest contracts, but there is much to be said for having an independent chair or facilitator for the process – someone who has no vested interest in who wins the contract. That could be an external person, or someone internal but from outside the immediate area of the contract. But even more importantly, the Chair should not personally score the proposals. Their job is to manage the moderation process (and probably to provide wider assurance for the whole evaluation). That is hard to do objectively if they are arguing for their own interpretation of the proposals and their own suggested scores.
The Chair does need to have read the bids, at least a quick skim, as they may have to refer to the content to help resolve any disagreements within the evaluation team. But their role is to explore the arguments for any scoring variations and help the team come to an agreed consensus score.
2. Maintain accountability by keeping a record of why/how scores were awarded.
There is a caveat here. I do not recommend that the formal notes of the meeting are too detailed. There will be some argument, there will be aspects of subjectivity creep into the discussion at times. It is not useful – and it provided ammunition for many future challenges – to have every bit of that recorded and available to be used in court! However, you do want to have a good justification, particularly for any very low or very high scores. There is no harm also in showing that at times, there was “healthy debate” between the evaluators.
This level of detail is what I’d consider appropriate in those situations:
“The evaluators agreed that the proposal fully met the requirement. There was debate around whether the proposal to provide online training for staff on the new system was “added value” and therefore indicated a mark of 5 rather than 4. But as training was a stated requirement, a mark of 4 was agreed.”
3. Manage the Audit Trail!
This is my most controversial recommendation. The individual evaluators will almost always read through the bids on their own. However, I ask that they do NOT award their own scores before the evaluation meeting. Of course, if they want to jot down some notes informally, to remind themselves of the key points ready for the moderation meeting, that is fine. But I do not want any record available that says my evaluators had very different scores for the same bid. I just want a single, agreed, moderated score for each response.
This is very simple when the evaluation is being done manually. But it can be more difficult with some eSourcing systems, which require individual evaluators to enter their scores. The answer may be to enter the scores after the moderation meeting – so everyone enters the same marks into the system. If this is not possible, then we have to revert to point 2. And this comes back again to our court case. The changes in marking post moderation in that evaluation may have been totally reasonable – as we said in part 1, evaluators do make mistakes that moderation corrects.
But if you have an audit trail, systemised or not, that shows adjustments, then you really need to explain very well and clearly how those changes were arrived at. Otherwise, you will give the courts reason to suspect that some sort of bias, incompetence, or something even worse came into play during that moderating process
Bid evaluation and score monitoring are complex processes, but with this advice you’re sure to be successful and avoid challenges.
Moderating bid evaluator scores certainly isn’t easy– there are lots of things to consider. Even so, we strongly believe that having multiple evaluators and properly executing the moderating process is one of the best things you can do in public procurement, and we hope that this article has helped make the process less daunting for you in the future!