[EDITOR’S NOTE: This article is an abbreviated version of a paper by the same name that was presented at the Tunnelling Association of Canada’s annual meeting Oct. 26-28, 2014, in Vancouver, B.C. It is being published here with permission. For more information on the Tunnelling Association of Canada, visit www.tunnelcanada.ca]
When a constructor has been prequalified for a project, it increases the chances of project success on many fronts. In essence, this makes constructor prequalification an effective risk management tool. In fact, if the request for qualification is well-conceived and prequalification process conducted strategically, it can be argued this is the industry’s most effective tool in its risk management toolbox.
Unfortunately, the full benefit of constructor prequalification is often not realized because the prequalification process is a victim of misadministration and an evaluation process that is highly subjective. Specifically, prequalification documents are commonly ambiguous, too qualitative and subjective, overly standardized, and without clarity on how the shortlist of constructors will be developed. As a result, constructors often walk away, believing that process bias in some form was involved, there was inequality in the prequalification criteria, too many constructors were shortlisted, and they were generally not given a “fair shake.”
Today, infrastructure needs are high, and will likely remain high as assets age and population continues to increase. As a result, procuring best-value constructor services is extremely important. Owners must be as sophisticated in the soft-side of infrastructure development, like constructor prequalification, as they are in the hard-side, the actual design and construction of tunnel works.
The many advantages of using a constructor qualification process prior to the tender period (prequalification) include:
- A specific time period to discern what attributes a constructor needs to have to successfully complete the project at hand,
- Optimization of interest and consideration of project challenges because a smaller pool of constructors exists,
- Reduction in bid scatter by minimizing the number of constructors who will “throw a number at the project” and prequalified constructors have a greater chance of award,
- Receipt of much more comprehensive qualification packages,
- A much better understanding of the project by the constructor awarded the work,
- The greatest chance of getting the best-value constructor,
- Leveling of the playing field as the constructor pool will ideally have equitable experience;
- Elimination of constructors who do not have the experience to complete the work,
- Reduction in the overall cost of tender solicitation,
- Expedition of the evaluation and award process,
- A convenient framework for constructor selection,
- Protection of constructors from being awarded work they do not have the capability to execute,
- Significant reduction of low bidder bias,
- Reduced risk and enhanced end-product quality if critical subcontractors are included in the qualification process (It is worth noting that integrating subcontractors in the prequalification process creates more complexity and is frowned upon by most tunnel constructors),
- Reduced need for extensive engineer-derived means and methods as the bidder pool includes the most qualified contractors, and
- Identification of constructors who may not be able to perform due to a deep work backlog.To realize these advantages, it is imperative the process be transparent, objective, and quantitative in terms of scoring. A strategic qualification model provides a higher level of assurance that protests will be avoided and shortlisted constructors are truly qualified to conduct the work.
It is the author’s experience that there is almost no standardization of constructor prequalification across North America. This fact alone makes what could be a very effective risk management tool an almost blight on the industry. Constructors have no idea what to expect in a Request for Qualifications (RFQ) when a project is advertised. Additionally, RFQs are often riddled with a wide array of issues that limit their effectiveness.
Below is a tunnel constructor prequalification development flowchart that the author created to assist owners in the implementation of a successful prequalification program. The nature of the owner (public or private) and project specifics will alter the steps required, but this is a good general guide. Several of the steps noted in the flowchart are elaborated on in later sections to foster development of an RFQ that is well received by the tunnel industry.
- Develop Project Awareness Campaign and Constructor
- Outreach Program
- Develop RFQ Stakeholder List
- Make Key Early Decisions in Prequalification Process
- Develop Prequalification Timeline
- Develop List of Project Specific Criteria
- Select Qualification Scoring Method
- Select Pass/Fail Criteria
- Assign Weighting to Criteria (if used)
- Determine Minimum Score to be Prequalified
Engage Engineering Team to Review Submitted Qualifications
Project Awareness Campaign/Constructor Outreach Program
This step in a prequalification development process (PDP) may seem unnecessary to the unsophisticated or inexperienced, but there are very good reasons for this in today’s tunnel industry. Like never before, owners are in competition with other owners to attract the best constructors, and few constructors are local to major tunnel projects. Additionally, there is a strong trend of European and other international constructors entering the North American tunnel marketplace. To address these facts of life, project awareness and constructor outreach programs are recommended.
Outreach should include notification of the project in as many industry media outlets and forums as possible to foster awareness.
Outreach should be started no later than the 30 percent design stage of a typical design-build-build project and preferably three months before the RFQ for design-build delivery. The ultimate goal of a project awareness campaign and outreach program is to attract qualified, responsive, and responsible constructors. Constructor interest in pursuing a project will be gaged by many factors, and owners should be aware of what can undermine the level of confidence a constructor has in an individual owner.
Typically, constructors will “walk” (not bid) if one or more of the following factors exist:
- Multiple bids are due in the same timeframe,
- Time to prepare the qualification is too short,
- Overall marketplace saturation exists,
- Unaccounted for risk in the bid proposal,
- Unbalanced risk management program (i.e., risk shedding and not risk sharing),
- Use of non-standard front-end clauses,
- No geotechnical baseline report or other contractual risk tools (i.e. escrow bid documents, dispute review board, differing site condition clause, value engineering change proposal, etc.),
- Use of non-standard technical specifications,
- Owner has a bad reputation and/or is typically slow to pay,
- Too much emphasis on subcontractor qualifications,
- Highly prescriptive specifications that include too much engineer-derived means and methods,
- Poorly written, highly subjective, and ambiguous prequalification process,
- Project funding is not secured,
- Use of a non-proven contract delivery vehicle, and
- The owner has never constructed a major tunnel project.
Develop RFQ Stakeholder List
On the surface, this step seems innocuous, but in reality, it has not been given proper consideration on a few of the author’s projects and caused delays in constructor procurement. Several times important stakeholders that should have been involved in RFQ development were not identified and later caused delays in project advertisement because their concerns were not addressed. Every owner project team should be proactive in contacting staff in other owner departments to seek approval of the RFQ process and format. Especially important is buy-in from the legal, purchasing, and finance departments.
Key Early Decisions in Prequalification Process
There are several important decisions that need to be made to facilitate an efficient PDP, including the following:
- RFQ table of contents and format,
- Number of RFQs to issue for large projects,
- Whether or not to allow electronic submittals,
- Minority participation goals,
- Mandated table of contents to facilitate qualification review,
- Binding type (i.e., three-ring binder or comb),
- Number of constructors to shortlist,
- Multiple prequalifications to spread the work for large projects, and
- Standards that should be used for evaluation of qualification criteria validity.
A few of these points are worthy of additional consideration. The number of constructors to shortlist is one of the most important of all decisions to be made in the PDP. Constructors can spend $50,000 or more to prepare a complex qualification package. Additionally, it is normal for a constructor to spend $100,000 or more in preparing a proposal for a major tunnel project. If the shortlist contains more than four to six constructors, it is likely some of the constructors will choose to decline bidding or simply “through a number at it.” This is often a cause for wide bid scatter on a project.
The author recommends five constructors be shortlisted such that if two drop-out, there are still three, giving many jurisdictions enough of a pool to award the work. The decision on the number of constructors to shortlist is even more important on a design-build project. The design-build team will spend upwards of $1,000,000 or more to prepare a proposal for a major tunnel project. If more than five teams are shortlisted, then there is inequity as the chances for award to any one team diminishes and the owner should seriously consider a stipend for proposal preparation to foster fairness in the process.
The other key early decision is to develop questions that can help assess the validity of criteria used to compare constructor qualifications. Many agencies struggle with how to select the correct criteria to differentiate between prequalifications.
The following questions will help in this regard:
- Are the criteria fair, concise, complete, and unambiguous?
- Are the criteria onerous for the constructor to address?
- Are the criteria relevant to the decision process?
- Are the criteria too subjective?
- Can the criteria be manipulated in the prequalification package?
- Will the criteria conflict with the Quality Assurance article contained in most three-part technical specifications?
It is not unusual for an owner to send out a draft of the RFQ to constructors for review and comment. This will engender confidence in the constructors that the owner is fair and balanced and likely translate into lower bid prices. If owners give constructors a sense that the RFQ process is fair, they will likely find a level of interest beyond what they might expect otherwise.
Develop Prequalification Timeline
Several factors, as outlined below, may need to be considered in determining when the RFQ is advertised and for how long:
- Consent order or consent decree requirements,
- Design schedule,
- Site visit schedule,
- Construction schedule,
- Marketplace saturation,
- Conflicts with other projects in tender,
- Extent and type of financial information requested,
- Extent, type, and format of insurance requirements,
- Number and contents of affidavits, and
- Notary requirements.
Owners should not underestimate the importance of time and timing when going through a prequalification process. From the author’s experience, at least six to eight weeks should be set aside to allow constructors ample time to prepare their qualification statement. The complexity of the submittal package is a large consideration in selecting the timeframe. It is important to keep in mind that there is no standardized format for qualifications between owners. Therefore, it is common for constructors to have to start from scratch in preparing their qualifications packages.
Develop List of Project Specific Criteria
Development of project specific criteria to apply in the comparison of prequalification packages is one of the more critical steps in the PDP. Criteria can generally fall into technical and non-technical criteria. Specific categories of criteria to be considered include financial, legal, equipment inventory, staffing, safety performance, backlog, and work history (i.e., schedule adherence and total project cost compared to initial bid cost). Considering that successful tunneling is primarily related to staffing and means and methods, the criteria should focus heavily on those areas.
Select Prequalification Scoring Method
To properly assess the quality and capability of a constructor, some type of scoring or rating system should be applied to the prequalification review process. The key considerations in selecting a scoring method are fairness and objectivity, which is easily said but hard to accomplish. In general, courts have ruled that scoring systems are appropriate as long as they are rational and documented.
Discretionary scoring is discouraged but can be used if some framework is applied so the subjective judgments by evaluators are bounded or bookended. Hardly ever is the comparison of constructor prequalifications completed on a purely ordinal ranking (prequalifications ranked 1st, 2nd, 3rd, etc.) without some scoring basis, although this method has held up in courts. The author is not a fan of ordinal rankings with only subjective observations and evaluations as this approach has been fraught with bias and protests. One person views qualifications in a different way than another person might.
The author strongly recommends some form of a scoring system be applied for the comparison of prequalification packages. The primary reason is that scoring systems are an efficient way to aggregate complex information. Although no scoring system is perfect and all scoring systems have some degree of subjectivity, they do provide a framework within which rational comparisons can be made. Very often, constructors are in the dark about how the shortlist was developed. One of the key benefits of a scoring system is the reduction in ambiguity in the constructor selection process. The ideal situation is that whatever scoring method is used it provides the selection committee with the tools to make an informed and intelligent decision.
There are many rating or scoring systems that can be applied. Any system can be used as long as it provides meaningful distinctions among prequalifications of various merits.
Types of rating systems include, but are not limited to, the following:
- Method No. 1 – Pass/Fail System
- Method No. 2 – Adjectival Ratings
- Highly Acceptable
- Marginally Acceptable
- Method No. 3 – Color Coding
- Red, Yellow, Green, Blue
- Method No. 4 – Past Performance
- Substantial Confidence in Prequalification
- Confidence in Prequalification
- Limited Confidence in Prequalification
- No Confidence in Prequalification
- Unknown Confidence in Prequalification
- Method No. 5 – Points System
- Method No. 6 – Risk Assessment
- Method No. 7 – Combination of Methods
An example combination method for evaluation and scoring of respondents packages from a major metropolitan agency is reflected below:
- Organizational Management Approach – 20 points
- Past Project Experience and Performance – 40 points
- Key Personnel Qualifications and Experience – 30 points
- Safety Program and Performance – 10 points
- Pass/Fail Criteria
Although this example seems appropriate, it falls short in several areas. For one, equipment inventory is not considered. Means and methods of construction are where a project is won or lost, so neglecting to include some evaluation of equipment is not recommended. Also, the point totals for each category were not defined in some framework to establish how the points are assigned. Without that framework, a high level of subjectivity is introduced into the process.
A scoring system with a small number of gradations (poor, fair, and good) will not provide enough discrimination between various levels of proficiency. Even systems that use adjectival descriptions like excellent, good, fair and poor are less useful in communicating the differences between prequalifications than systems with far more gradations. A vital requirement for an owner is to document the relative strengths, deficiencies, significant weaknesses, and risks supporting the evaluation of constructor prequalifications. Also, the review process of constructor prequalifications should include the same evaluators for each package submitted.
Select Pass/Fail Criteria
No matter what scoring or rating system is outlined in an RFQ, most agencies include certain pass/fail criteria. In the majority of cases, if a constructor receives a fail designation on a specific criterion (fail designation depends on nature of question and whether a yes or no is noted), it precludes the constructor from being shortlisted.
Examples of pass/fail criterion are provided below:
- Has a surety firm completed a contract on your behalf, or paid for completion because your firm was default terminated by the project owner within the last five years?
- At any time during the last five years has your firm, or any of its owners or officers, been convicted of a crime involving the awarding of a contract of a government construction project or bidding or performance of a government contract?
- Constructor possesses a valid and current Constructor’s license for the project or projects for which it intends to submit a bid.
- Constructor has a general liability insurance policy with a policy limit of at least $1,000,000 per occurrence and $2,000,000 aggregate and an automobile liability policy with a policy limit of at least $1,000,000 per occurrence.
Owners would be well-served to reduce the number of pass/fail criteria to an absolute minimum to increase bidder pool size and reduce post-shortlist protests.
Determine Minimum Score to be Prequalified
This is another critical step in the PDP. An agency can either determine the minimum score required for shortlist or develop the shortlist based on the top five scores. In this way, the shortlist is made up of, theoretically, the most qualified constructors. The minimum score to be shortlisted could be done in a number of ways, but one simple way is to determine an average rating in each question for all constructors. The total of the average ratings can then be considered the minimum score to be shortlisted.
As a side note, the author strongly discourages the use of terms in the shortlist process like “unqualified” or “not qualified.” These terms foster protests because they carry so much weight in the public domain.
Engage Engineering Team to Review Submitted Prequalifications
Tunneling is filled with unique descriptors, means and methods, jargon, nuances, and terms. It is vital for an owner not experienced in the industry to seek the advice of an engineering team to compare prequalification packages. The suggested approach for owners is to have legal and purchasing departments review any pass/fail criteria to confirm the respondent is responsive. Once that is complete, the engineering team can complete a comprehensive review and provide a recommendation on the shortlist.
If the owner implements a project specific selection and/or evaluation committee, it is best done by engaging staff from various owner departments. This fosters a more objective evaluation. If an owner selection committee is used, it is incumbent on the owner to have the engineering team review the scoring to ensure no disparate scores or obvious mistakes have been made.
The author believes constructor prequalification is one of the most significant risk management tools for the tunnel industry, but one that is greatly underutilized. It is often an ineffective risk management tool because the PDP does not receive the critical thinking needed and the process often involves significant subjectivity. When the PDP is done methodically and strategically, it has numerous benefits as outlined in this paper. If the recommendations in this paper are followed, it is much more likely our highly esteemed owners, who provide us a living and career, will be much more satisfied with the outcome of their infrastructure development.
Don Del Nero, P.E., C.D.T., is Vice President/Tunneling and Trenchless Practice Leader for Stantec Consulting Services Inc.