Usability Testing Benefits Obtained at The St. Paul Companies

The St. Paul Companies committed significant resources to a new E-Commerce initiative and website design, testing and implementation. Strategic business and financial commitments were made regarding a new program introduction, and ROI expectations were based on resource allocations, revenue expectations and the specific program start date. Planned service introduction dates were announced, and marketing and training resources were allocated around the introduction date. Therefore, the successful and on-time introduction of the new service had significant impact on bottom-line financial results from a program cost perspective as well as the expected strategic and revenue results of the new initiative.

The project team was assigned specific targets and program start dates. Project management was concerned that if the usability of the E-Commerce product was less than optimal the project would be delayed and costs would increase, if not make it impossible to carry out the program design and reach the ROI goals. The concerns focused on the fact that by using the same in-house staff for design and testing, the expertise and objectivity of the testing was likely to be compromised. To overcome these concerns an outside usability expert was brought in early during the design and stayed involved throughout final product testing.

The process involved the following stages regarding usability review and testing:

  1. The usability of the product was examined during several project review meetings. The usability expert reviewed the usability issues including user interfaces, data input sequence issues, user data availability issues and offered recommendations to streamline user interfaces.
  2. The usability expert reviewed the product as the various components went into Alpha testing stages and recommended changes to further improve usability.
  3. Beta testing of the product involved the usability team conducting supervised, digitally recorded user test sessions with a select group of target users. The St. Paul’s project management observed some of the test sessions. The entire test results were analyzed in detail and reported by the usability team, together with specific usability recommendations for final improvements.

The expert outside usability review and testing effort yielded significant benefits for the E-Commerce initiative. The design team estimates that at least 30-45 days of programming effort was saved by eliminating trial-and-error testing and revisions by including the usability team in design review meetingS and following’ the team’s recommendations. Also, the program introduction dates were kept on target, instead of slipping at least 30-45 days behind schedule, wasting marketing resources and disappointing upper management by missing specific targets. The project ROI was also kept to target by avoiding extra costs and delayed program start dates.

In our experience the right outside usability expertise offers the potential for a high return on such an investment. Our usability team was directed by Dr. Robert Baron, Founder of Serp-Maven, an SEO & Usability consulting firm. We were pleased with Dr. Baron’s involvement in our Internet and E-Commerce initiatives, based on his years of user interface expertise, his ability to communicate well and interact with both technical team members as well as all levels of management.

Project Manager
Mike Hager
The St. Paul Companies
Public Sector Services
385 Washington St. MC 503P
St Paul, Minnesota, 55102-1396