[EDITOR’S NOTE: This article is an abbreviated version of a paper by the same name that was presented at the Tunnelling Association of Canada’s annual meeting Oct. 26-28, 2014, in Vancouver, B.C. It is being published here with permission. For more information on the Tunnelling Association of Canada, visit www.tunnelcanada.ca]
When a constructor has been prequalified for a project, it increases the chances of project success on many fronts. In essence, this makes constructor prequalification an effective risk management tool. In fact, if the request for qualification is well-conceived and prequalification process conducted strategically, it can be argued this is the industry’s most effective tool in its risk management toolbox.
Unfortunately, the full benefit of constructor prequalification is often not realized because the prequalification process is a victim of misadministration and an evaluation process that is highly subjective. Specifically, prequalification documents are commonly ambiguous, too qualitative and subjective, overly standardized, and without clarity on how the shortlist of constructors will be developed. As a result, constructors often walk away, believing that process bias in some form was involved, there was inequality in the prequalification criteria, too many constructors were shortlisted, and they were generally not given a “fair shake.”
Today, infrastructure needs are high, and will likely remain high as assets age and population continues to increase. As a result, procuring best-value constructor services is extremely important. Owners must be as sophisticated in the soft-side of infrastructure development, like constructor prequalification, as they are in the hard-side, the actual design and construction of tunnel works.
The many advantages of using a constructor qualification process prior to the tender period (prequalification) include:
It is the author’s experience that there is almost no standardization of constructor prequalification across North America. This fact alone makes what could be a very effective risk management tool an almost blight on the industry. Constructors have no idea what to expect in a Request for Qualifications (RFQ) when a project is advertised. Additionally, RFQs are often riddled with a wide array of issues that limit their effectiveness.
Below is a tunnel constructor prequalification development flowchart that the author created to assist owners in the implementation of a successful prequalification program. The nature of the owner (public or private) and project specifics will alter the steps required, but this is a good general guide. Several of the steps noted in the flowchart are elaborated on in later sections to foster development of an RFQ that is well received by the tunnel industry.
Engage Engineering Team to Review Submitted Qualifications
Project Awareness Campaign/Constructor Outreach Program
This step in a prequalification development process (PDP) may seem unnecessary to the unsophisticated or inexperienced, but there are very good reasons for this in today’s tunnel industry. Like never before, owners are in competition with other owners to attract the best constructors, and few constructors are local to major tunnel projects. Additionally, there is a strong trend of European and other international constructors entering the North American tunnel marketplace. To address these facts of life, project awareness and constructor outreach programs are recommended.
Outreach should include notification of the project in as many industry media outlets and forums as possible to foster awareness.
Outreach should be started no later than the 30 percent design stage of a typical design-build-build project and preferably three months before the RFQ for design-build delivery. The ultimate goal of a project awareness campaign and outreach program is to attract qualified, responsive, and responsible constructors. Constructor interest in pursuing a project will be gaged by many factors, and owners should be aware of what can undermine the level of confidence a constructor has in an individual owner.
Typically, constructors will “walk” (not bid) if one or more of the following factors exist:
Develop RFQ Stakeholder List
On the surface, this step seems innocuous, but in reality, it has not been given proper consideration on a few of the author’s projects and caused delays in constructor procurement. Several times important stakeholders that should have been involved in RFQ development were not identified and later caused delays in project advertisement because their concerns were not addressed. Every owner project team should be proactive in contacting staff in other owner departments to seek approval of the RFQ process and format. Especially important is buy-in from the legal, purchasing, and finance departments.
Key Early Decisions in Prequalification Process
There are several important decisions that need to be made to facilitate an efficient PDP, including the following:
A few of these points are worthy of additional consideration. The number of constructors to shortlist is one of the most important of all decisions to be made in the PDP. Constructors can spend $50,000 or more to prepare a complex qualification package. Additionally, it is normal for a constructor to spend $100,000 or more in preparing a proposal for a major tunnel project. If the shortlist contains more than four to six constructors, it is likely some of the constructors will choose to decline bidding or simply “through a number at it.” This is often a cause for wide bid scatter on a project.
The author recommends five constructors be shortlisted such that if two drop-out, there are still three, giving many jurisdictions enough of a pool to award the work. The decision on the number of constructors to shortlist is even more important on a design-build project. The design-build team will spend upwards of $1,000,000 or more to prepare a proposal for a major tunnel project. If more than five teams are shortlisted, then there is inequity as the chances for award to any one team diminishes and the owner should seriously consider a stipend for proposal preparation to foster fairness in the process.
The other key early decision is to develop questions that can help assess the validity of criteria used to compare constructor qualifications. Many agencies struggle with how to select the correct criteria to differentiate between prequalifications.
The following questions will help in this regard:
It is not unusual for an owner to send out a draft of the RFQ to constructors for review and comment. This will engender confidence in the constructors that the owner is fair and balanced and likely translate into lower bid prices. If owners give constructors a sense that the RFQ process is fair, they will likely find a level of interest beyond what they might expect otherwise.
Develop Prequalification Timeline
Several factors, as outlined below, may need to be considered in determining when the RFQ is advertised and for how long:
Owners should not underestimate the importance of time and timing when going through a prequalification process. From the author’s experience, at least six to eight weeks should be set aside to allow constructors ample time to prepare their qualification statement. The complexity of the submittal package is a large consideration in selecting the timeframe. It is important to keep in mind that there is no standardized format for qualifications between owners. Therefore, it is common for constructors to have to start from scratch in preparing their qualifications packages.
Develop List of Project Specific Criteria
Development of project specific criteria to apply in the comparison of prequalification packages is one of the more critical steps in the PDP. Criteria can generally fall into technical and non-technical criteria. Specific categories of criteria to be considered include financial, legal, equipment inventory, staffing, safety performance, backlog, and work history (i.e., schedule adherence and total project cost compared to initial bid cost). Considering that successful tunneling is primarily related to staffing and means and methods, the criteria should focus heavily on those areas.
Select Prequalification Scoring Method
To properly assess the quality and capability of a constructor, some type of scoring or rating system should be applied to the prequalification review process. The key considerations in selecting a scoring method are fairness and objectivity, which is easily said but hard to accomplish. In general, courts have ruled that scoring systems are appropriate as long as they are rational and documented.
Discretionary scoring is discouraged but can be used if some framework is applied so the subjective judgments by evaluators are bounded or bookended. Hardly ever is the comparison of constructor prequalifications completed on a purely ordinal ranking (prequalifications ranked 1st, 2nd, 3rd, etc.) without some scoring basis, although this method has held up in courts. The author is not a fan of ordinal rankings with only subjective observations and evaluations as this approach has been fraught with bias and protests. One person views qualifications in a different way than another person might.
The author strongly recommends some form of a scoring system be applied for the comparison of prequalification packages. The primary reason is that scoring systems are an efficient way to aggregate complex information. Although no scoring system is perfect and all scoring systems have some degree of subjectivity, they do provide a framework within which rational comparisons can be made. Very often, constructors are in the dark about how the shortlist was developed. One of the key benefits of a scoring system is the reduction in ambiguity in the constructor selection process. The ideal situation is that whatever scoring method is used it provides the selection committee with the tools to make an informed and intelligent decision.
There are many rating or scoring systems that can be applied. Any system can be used as long as it provides meaningful distinctions among prequalifications of various merits.
Types of rating systems include, but are not limited to, the following:
An example combination method for evaluation and scoring of respondents packages from a major metropolitan agency is reflected below:
Although this example seems appropriate, it falls short in several areas. For one, equipment inventory is not considered. Means and methods of construction are where a project is won or lost, so neglecting to include some evaluation of equipment is not recommended. Also, the point totals for each category were not defined in some framework to establish how the points are assigned. Without that framework, a high level of subjectivity is introduced into the process.
A scoring system with a small number of gradations (poor, fair, and good) will not provide enough discrimination between various levels of proficiency. Even systems that use adjectival descriptions like excellent, good, fair and poor are less useful in communicating the differences between prequalifications than systems with far more gradations. A vital requirement for an owner is to document the relative strengths, deficiencies, significant weaknesses, and risks supporting the evaluation of constructor prequalifications. Also, the review process of constructor prequalifications should include the same evaluators for each package submitted.
Select Pass/Fail Criteria
No matter what scoring or rating system is outlined in an RFQ, most agencies include certain pass/fail criteria. In the majority of cases, if a constructor receives a fail designation on a specific criterion (fail designation depends on nature of question and whether a yes or no is noted), it precludes the constructor from being shortlisted.
Examples of pass/fail criterion are provided below:
Owners would be well-served to reduce the number of pass/fail criteria to an absolute minimum to increase bidder pool size and reduce post-shortlist protests.
Determine Minimum Score to be Prequalified
This is another critical step in the PDP. An agency can either determine the minimum score required for shortlist or develop the shortlist based on the top five scores. In this way, the shortlist is made up of, theoretically, the most qualified constructors. The minimum score to be shortlisted could be done in a number of ways, but one simple way is to determine an average rating in each question for all constructors. The total of the average ratings can then be considered the minimum score to be shortlisted.
As a side note, the author strongly discourages the use of terms in the shortlist process like “unqualified” or “not qualified.” These terms foster protests because they carry so much weight in the public domain.
Engage Engineering Team to Review Submitted Prequalifications
Tunneling is filled with unique descriptors, means and methods, jargon, nuances, and terms. It is vital for an owner not experienced in the industry to seek the advice of an engineering team to compare prequalification packages. The suggested approach for owners is to have legal and purchasing departments review any pass/fail criteria to confirm the respondent is responsive. Once that is complete, the engineering team can complete a comprehensive review and provide a recommendation on the shortlist.
If the owner implements a project specific selection and/or evaluation committee, it is best done by engaging staff from various owner departments. This fosters a more objective evaluation. If an owner selection committee is used, it is incumbent on the owner to have the engineering team review the scoring to ensure no disparate scores or obvious mistakes have been made.
The author believes constructor prequalification is one of the most significant risk management tools for the tunnel industry, but one that is greatly underutilized. It is often an ineffective risk management tool because the PDP does not receive the critical thinking needed and the process often involves significant subjectivity. When the PDP is done methodically and strategically, it has numerous benefits as outlined in this paper. If the recommendations in this paper are followed, it is much more likely our highly esteemed owners, who provide us a living and career, will be much more satisfied with the outcome of their infrastructure development.
Don Del Nero, P.E., C.D.T., is Vice President/Tunneling and Trenchless Practice Leader for Stantec Consulting Services Inc.