PoCs – Protecting your investment with the open road test

Feb 15th, 2010 | Posted by
Bookmark and Share

In a recent conversation with a vendor, I was asked if there was any way in which we [Celent] could encourage/support the standardization of request for information documents sent out by insurers. Think ACORD Standards for RFIs! At the heart of her question, was a sense of deep frustration at the evaluation process which inhibits the vendor from highlighting or differentiating themselves. I have some sympathy with this view.

IT departments have become very good at comparing the data for RFI/RFPs in two-dimensions. A structured, transparent evaluation process is an important step in choosing what it likely to be a significant IT investment in the case of a core system replacement. The evaluation process can be used to generate hard facts highlighting the differences in system functionality, company and delivery options.

For many insurers, the governance process requires a presentation to the project sponsors of a detailed quantitative analysis of vendor submissions, along with ranking and weightings. This is all well and good but this output can be incorrectly attributed the status of a statistically sound scientific evaluation. I have seen projects where a vendor has been chosen because they had 2 more points in the final analysis on paper.

Most of us would not buy a car based solely on a review of features and functions on the manufacture’s website or brochure. We’d be out there driving and deliberating the real experience of the open road, handling of corners, and comfort of the seats. All these factors are almost impossible to quantify in a brochure but vital in influencing our final choice.

And it’s similar in the world of core system evaluation. The proof of concept is vital addendum to the evaluation process. This is the time to put the vendor to the test. This stage of the evaluation should allow the buyer to really see how easy the product configurator might be, how well a vendor can understand requirements, how responsive/ organised the vendor is, or how much/little coding is required.

And in Celent’s view, proof of concepts should be paid for. It shows the commitment on part of the insurer, and it also encourages the vendor to be focused and committed to this important step. Being paid allows the vendor to justify allocating top staff otherwise committed to paid implementations. A recent evaluation project paid 4-figure sums to both short-listed vendors for a 3 month proof of concept.

The proof of concept is a great process by which to garner more buy-in from the business users, and to generate some excitement about what this change could bring to the organization. Adding the open road test to the brochure comparison will undoubtedly help the insurer make a more robust decision.

  1. Clare Dorrian
    Feb 25th, 2010 at 06:02
    Reply | Quote | #1

    Great post Catherine and absolutely aligned to our view.

    The POC is a an excellent way of driving evidence based insight as opposed to vendor biased content into procurement decisions. They embrace the end user (so important for downstream solution adoption and advocacy as roll out begins) and allow the organisation to truly road test product capability and vendor cultural fit without investing a significant amount of capital.

    We’re actually seeing an increasing number of our clients embracing the POC but often as a result of our active encouragement. Roll on the day this is the norm.

  2. Phil Ayres
    Mar 5th, 2010 at 12:45
    Reply | Quote | #2

    For core systems, I can imagine that a PoC is quite a lengthy and involved process, so some investment from the insurer seems reasonable. For other system, companies can consider the alternative to being tied-in with perpetual licenses by selecting software as a service (SaaS) solutions. If there is a solution that fits a company’s needs, and is backed by a flexible contract, SaaS offers the equivalent of a continuous live PoC. The vendor is constantly having to prove that they can deliver if they are to keep the customer’s business.

    After a certain amount of due diligence this may be a more cost effective approach for the customer. Rather than investing to trial multiple systems that are literally just being tested, SaaS gives the opportunity to benefit from the software for real and swap it out later if it fails to live up to expectations. For new requirements especially, including business process improvement / workflow automation, this can be a great alternative to “try before you buy”.

    Phil Ayres
    http://blog.consected.com