Cloud Computing Protest Offers Lessons for Buyers & Suppliers

By Frank McNally posted 06-22-2017 12:10

  
cloud_computing_protest_article.jpg
A recent protest decision from the Government Accountability Office (GAO) provides several important lessons for procurement professionals who are buying cloud-enabled services as well as vendors hoping to sell them. Let's dive in!

Red River Computing Company protested the Department of Homeland Security's (DHS) request for quotations (RFQ) for agency-wide enterprise computing services and cloud computing services. Red River challenged both the technical and price evaluation, asserting that DHS's best-value determination was flawed. GAO denied Red River's challenge of the technical evaluation but sustained its challenge of the price evaluation. Each decision is instructive, so let's take a closer look.

Clarifications, Discussions, Exchanges...Oh My!

DHS issued the RFQ to GSA IT 70 schedule holders with the intent to award three blanket purchase agreements (BPAs), under which the agency could issue fixed price and/or time and material order types. Vendors would provide "access to infrastructure-as-a-service (IaaS) cloud services, either directly from cloud service providers or through resellers." The solicitation required that services include commercial, commodity-based Infrastructure as a Service (IaaS) cloud services authorized under GSA's Federal Risk Authorization and Management Program, or FedRAMP.

Lesson for suppliers: If you are hoping to provide any technology services in this day and age, they're going to almost always require FedRAMP pre-approval. This is effectively a go/no-go barrier, so if you don't currently have FedRAMP authorization for your services, you're going to want to get that ball rolling. This is no small undertaking, so check out FedRAMP's program overview to learn more.

The RFQ stated that DHS was exercising its authority to award without conducting exchanges, but reserved its right to hold exchanges based on individual offers. This is customary; agencies are permitted to award without holding exchanges with offerors, especially when awarding contracts pursuant to FAR Subpart 8.4, which is the relevant authority given DHS's solicitation against the GSA schedule. Despite this fact, Red River challenged that DHS treated them unequally when DHS evaluators sought clarification from eight of the offerors, but not from Red River.

GAO appropriately denied this portion of the protest where Red River argued that they received unequal treatment. Agency's are not required to seek clarification from all parties in this type of competition. As defined by the FAR, clarifications are limited exchanges between the Government and offerors, that may occur when award without discussions is contemplated. GAO noted that DHS evaluators indicated Red River's customer service approach (the issue at hand) reflected a lack of detail that, in order to correct, would have required "significant revision to Red River's quotation and would therefore require the agency to conduct discussions." Because there was no requirement for DHS to conduct discussions under this FAR Part 8 solicitation, GAO denied this portion of the protest.

Lesson for buyers: It is critical to understand the distinction between clarifications and discussions (which are two distinct categories of exchanges), and to understand your rights and obligations as an evaluator. If this were a FAR Part 15 solicitation with a competitive range determination, the situation would have been different.

Lesson for suppliers: When an agency states that it may award without exchanges, it is critical that you submit an offeror that reflects your best and final offer as there is no guarantee you will have the opportunity to clarify elements of your technical proposal or lower your price to be more competitive.

If an unstated evaluation criteria is used and no one is around to hear it, does it make a [protest] sound?

Another facet of this protest worth exploring is Red River's contention that its offer was treated unequally during the technical evaluation. Essentially, Red River attempted to do what many losing protestors often do, which is to persuade GAO that an agency's technical evaluation was improper or unfair. This is nearly impossible and rarely successful; GAO's de facto policy in reviewing protest cases is not to rehash an evaluation decision, nor to decipher the judgement of an awarding contracting officer or source selection official acting in accordance with the solicitation instructions. 

Red River's protest was a bit more nuanced than simply stating improper evaluation. They suggest that DHS used unstated evaluation factors, which has been successful in past decisions but is difficult to prove. See this excerpt:

Where an agency conducts a formal competition for the establishment of a BPA, we will review the agency’s actions to ensure that the evaluation was reasonable and consistent with the solicitation and applicable procurement statutes and regulations. Alexander & Tom, Inc., B-412358 et al., Jan. 21, 2016, 2016 CPD ¶ 46 at 4.  While agencies are not permitted to use unstated evaluation factors in evaluating quotations, an agency properly may take into account specific matters that are logically encompassed by, or related to, the stated evaluation criteria, even when they are not expressly identified as evaluation criteria.  Camber Corp., B-413505, Nov. 10, 2016, 2016 CPD ¶ 350 at 5.  Additionally, in evaluating quotations in accordance with the stated evaluation factors, agencies may properly consider the degree to which quotations exceed the solicitation requirements.  USGC Inc., B-400184.2 et al., Dec. 24, 2008, 2009 CPD ¶ 9 at 6-7.

The bottom line: GAO found no evidence to support an unreasonable evaluation based on the facts provided by Red River. And they rarely will, even when an agency's evaluation "does not expressly state" that tertiary experience or technical capability will be considered when that tertiary review is complementary or associated with the stated evaluation criteria.

Lesson for suppliers: While it is always good strategy to "write to the evaluation criteria," keep in mind that agency evaluators have wide latitude to consider all aspects of an offeror's capability and experience in determining their suitability for award. 

In Price Evaluation We Trust

Evaluating quoted prices for cloud computing services is quite a challenge, primarily because the price of cloud services depends largely on the degree of computing power or capacity consumed by the agency during performance. There's probably a more technical way to say that, but stay with me. 

DHS instructed offerors to complete a pricing model scenario table by entering a discounted price for five cloud service items to be delivered over five ordering periods, and in accordance with certain assumptions relevant to the overall pricing model (eg., 100% utilization on-demand for all ordering periods). DHS provided additional context for this requirement in the question and answer round prior to close of the solicitation:

Vendor Question:  Many [cloud service providers] offer [a] discount for reserving or committing to a certain percent of utilization.  Should we provide multiple pricing for reserved instances as part of the scope of the pricing scenario?

DHS Answer:  No, not for the pricing scenario provided. Assume on-demand prices at 100% utilization.

When it comes to price evaluation, an agency is required "to evaluate vendors on an equal basis and in a manner that permits the meaningful assessment of the total cost to the government for the required goods or services.  Cross Match Techs., Inc., B-293024.3, B-293024.4, June 25, 2004, 2004 CPD ¶ 193 at 6 n.2." Any situation where offeror quotations are evaluated using different assumptions constitutes an "apples to oranges" comparison and would therefore not reflect a meaningful comparison of vendor pricing.

In GAO's consideration of this argument, it looked at the way DHS evaluated the pricing offered by several of the intended awardees to determine whether the pricing model was completed in accordance with the solicitation requirements. GAO's discussion of its findings is instructive for any acquisition professional seeking to award vehicle-style contracts (e.g., BPAs and indefinite delivery contracts) for cloud services and should be carefully analyzed to understand these lessons learned. 

In fact, GAO did find that one of the intended awardees improperly completed the pricing model, resulting in inconsistent (and thus unreasonable) pricing evaluation, which led to the decision to sustain. However, unreasonable evaluation alone is not sufficient for GAO to sustain protests of this nature; rather, GAO must find that the protesting offeror was prejudiced by the outcome. In this case, GAO agreed that Red River was prejudiced by the inconsistent price evaluation, as the offending party's offer would not have been acceptable.

Effectively, GAO is saying that this inconsistent treatment of offeror pricing prevented Red River from consideration for award. Hence, prejudice.

Lesson for buyers: Evaluating proposed prices for consumption-based services such as cloud computing will benefit from a pricing model similar to that designed by DHS. However, it is critical that price evaluator's take great care to ensure their evaluation of offeror-proposed models is consistent with the solicitation instructions to avoid any "apples to oranges" analysis that could lead to inconsistent evaluation.

Lesson for suppliers: When an agency provides a pricing model template for evaluation, follow it explicitly. 

We hope this evaluation is useful for buyers and suppliers alike, and appreciate the challenges associated with both sides of this equation. Protesting an award decision is a necessary evil when an offeror believes it has been prejudiced, and when that prejudice can be proved an offeror should absolutely consider filing.

But in too many cases, protestors are looking to re-litigate a contracting professional's judgement when they believe they have been treated unfairly. This can be a costly mistake. GAO rarely if ever considers this to be grounds for sustaining a protest, preferring to yield wide latitude to agency evaluators in awarding to offerors whom they believe offer the best value product or service for their customer's need.

Please consider leaving a comment here or in our discussion section if you have something to add, or have questions about this analysis. It is critical for buyers and suppliers to understand each other, and this type of analysis is provided only for that purpose and not to critique the actions of one party or another.

After all, this is complicated business!

Image Courtesy of Pexels

0 comments
1717 views














Tag

  • Featured
  • Learning and Skills
  • Most Read
  • Supplier Resources

Public Spend Forum Recommends

1 to 3 of 33 total
Most governments and public sector bodies (contracting authorities) have an objective to increase the share of public sector expenditure that goes to smaller firms (SMEs / small and medium enterprises). There is an interesting economic, political and ...
There is no shortage of advice for businesses who want to sell products or services to the public sector. Unfortunately, some of this is not good advice, and there are firms who exploit the ignorance of many firms to sell expensive training, consulting, ...
Adverse Natural Events are a Threat to PPP Infrastructure Projects Infrastructure resilience is the ability to reduce the magnitude and/or duration of disruptive events. The effectiveness of a resilient infrastructure or enterprise depends upon its ...

Most Read

Join the Community

Get full access to research and content, networking opportunities, and other exclusive benefits by becoming a member today.

JOIN TODAY - FREE!