Data center managers have a tremendous work load to maintain high sigma reliability for the entire workings of a data center. The ongoing daily pressure to maintain zero critical bus loss weighs heavily on one’s conscious. It’s a 7x24 challenge and now company management has added to that work load and placed the task of upgrading / expanding the existing data center or constructing an entirely new data center. So now a game plan has to be prepared to detail the scope of work (SOW) and how will the new scope be executed.
Data center managers also know that a data center is not your typical office buildout project and the planning requires a wealth of experience to prepare and the same level of experience to execute and deliver final product. There is one shot at completing the buildout task and it has to be done correctly. Post construction changes are difficult and expensive as the critical load is in operation. The data center has changed so dramatically in the last 5 – 10 years with power densities many times of what they used to be and the dynamics continuing to challenge us from all sides. So lets jump in at the deep end!
Upgrading A Data Center
Company management has assigned you with the task for the upgrade of the data center. An internal company team is selected outlining the overall objectives for short and long term goals. Initial meetings create the tasks before us;
- Development of business requirements.
- Understanding the existing space and its inadequacies.
- Detail of required new rack space.
- Estimate of new power density loads.
- Growth potential.
- Available space or real estate provisions.
- Transition procedure from existing space to new space.
After many team meetings and the development of the first draft of the Scope of Work (SOW), there comes a time when a budget is to be developed with a timeline. This is the point in time when a physical site construction build team is to be assembled.
Good progress has been made. The progress includes internal team assembly, tasks development, draft SOW, budgets and timeline. Now we need to develop the construction build team. At this point, there are basically two paths with the build team selection; Plan Spec & Bid (PSB) or Design/Build (D/B). Perspective vendors will have to be scrutinized to investigate their knowledge and skill in designing and building a data center. One other item to include in addition to the design and construction of the facility is the ongoing maintenance. This should be included in the budget for the entire facility including all equipment such as Generators, UPS & Battery, Cooling, Fire Protection, Electrical Systems and so on depending on the list of critical equipment on the site. Maintenance considerations can have large operational concerns with keeping the critical load on redundant environmental support systems while support equipment maintenance and testing procedures are being performed. We will expand on this discussion later on.
Plan, Spec & Bid involves in engaging an Architect (A/E) including Mechanical, Electrical, Plumbing and Structural Engineers and a separate Construction Manager/General Contractor. Design documents are provided for multiple GCs to put out for bid. Then in conjunction with the Architect, a GC is selected for your facility build. There are many design professional A/E firms to select from with data center experience. A separate construction team will be sourced requiring a second interview and vetting process determining the level of data center build experience.
Design/Build/Maintain: Design/Build (D/B) specialty companies provide a solution with a single point of responsibility. This solution will include all architectural/engineering and installation. This data center specialty contractor product and application experience is established because of repetitive construction projects in the data center industry. Design/Build companies are continuously exposed to new technology and how they improve installations and when combined with repetitive data center projects, it can be a large advantage to the client’s finished product. Additionally, a D/B contractor should also provide solutions for on-going maintenance for several reasons:
- The mechanical and electrical layout should have a design allowing maintenance without impeding normal operation. If a vendor's responsibility includes maintenance, all the more reason to design a system providing flexability in your electrical and mechanical systems. The last thing a data center manager wants to hear is, "A repair is needed, but the load has to be shut down". Avoidance of this scenario is very easily done when the designer has the vision to include.
- The cost for maintenance should be known for a 5 year period. Allocations for this cost should be included in the budget.
- The contractor interview and vetting process for the construction and maintenance should be combined. Inevitably, maintenance will have to be performed on the new systems and it should be known who and how this will be performed.
- The inclusion of maintenance creates the category of a Design/Build/Maintain (D/B/M) contractor.
Vetting A Data Center General Contractor
Overlooked many times is the vetting process for the general, mechanical and electrical contractor on-site project managers. When the rubber meets the road, these are the people that truly make a major difference. Contractors that work inside a data center environment have a higher level of technical prowess and experience. Also, how the general contractor’s office support the field personnel on overall code and equipment application on all major systems mostly covering electrical and mechanical is important. This magnifies the importance of the engineering and application experience of the General Contractor. For those with upgrades in an existing operational data center, the additional burden of having the installing contractors not knowing how to conduct themselves inside an operating facility exponentially increases the possibility of critical bus failure. The #1 reason for critical bus failure is human error. The additional cost of monitoring workers inside the operating facility adds to the overall project cost and consumes your staff’s time from performing their normal daily task.
The largest item in quality control is the mechanical system installation. Mechanical systems are many times left to interpretation of the installing mechanical contractor. No specific installation guidelines similar to the electrical National Electric Code (NEC) with Authority Having Jurisdiction (AHJ) inspections exist. Equipment manufacturers do include installation instructions with the equipment but are not part of the AHJ’s inspection criteria. For example, the AHJ will not review whether a nitrogen purge was used when brazing copper lines, proper minimum refrigerant vapor pipe gas velocities, correct oil trap locations, leak pressure testing and refrigerant line evacuation procedures to name a few (see image 1). These few items can pose major reliability and efficiency issues if not properly done. And frankly, how would you know if these procedures were not performed? The answer is you don’t but you will live with the burden and cost for years to come.
Client integration with the construction team to maintain communication with on-going project decision making enhances the final finished product and assist in maintaining completion schedule. The overall project in the eyes of the client is much more than the data center buildout that will include hardware transition, low voltage cabling, critical load software functionality and much more.
This construction integration is of high importance with a buildout consisting of an upgrade to an existing operating critical load. General Contractor development of “Methods of Procedure (MOP)” where installation procedures may impact ongoing operations is mandatory. This reinforces the aforementioned statement the General Contractor must have an overall technical background of all major equipment systems, interlocks, Sequences of Operation (SOO) and existing facility infrastructure. The MOP becomes a dependent communication tool so all parties involved thoroughly understand the task at hand and perform systematically to its end.
Theoretically, if the project is sizable, there could be dozens of MOPs written for a single project. It could be said this is why the design engineer is engaged but the General Contractor and the major subcontractors must first recognize the MOP initial necessity. It can also be said the repetitive involvement of the design engineer can add delays and cost to the project. A data center General Contractor should be able to draft project MOPs.
Commissioning, documentation and client training are to be provided at project commencement. Commissioning is a procedure providing an overall system test of various electrical, mechanical and other operational and redundant functions of the completed data center. It also should be noted that system commissioning is in addition to the manufacturer’s equipment start up. A SOO should be drafted during the system design stage and confirmed with the commissioning. It is surprising at times to find new equipment that fails the testing. This prevents a possible critical bus failure and the corrective action is paid by the equipment manufacturer or the contractor. Commissioning also has to be done even on smaller projects. For example, a fluid cooled CRAC system is installed with redundant fluid coolers and redundant pumps. The commissioning process should confirm a fluid cooler failure and/or a pump failure and the systems ability to provide seamless data center cooling. This is a small upgrade project but still has a commissioning process to confirm its integrity.
Final as-built documentation with AHJ final inspections should be compiled and forwarded to the client. This should include all application data and maintenance recommendations custom prepared to match installed equipment. Commissioning documentation should also be included.
Client training can be performed during the commissioning test. This is a great time to have first hand experience of the sights and sounds of the environmental support operation. For example, having three 16V diesel generators sequentially start and motor operated breakers close and open is not something a data center operator normally experiences. It is not expected for data center staff to be experts on these systems but they should have knowledge when and how to call for help.
In summary, the aforementioned discussion offers a condensed glimpse of the process to build a data center. One can see the dynamics of building a technical data center requires the entire major contractor team including general, mechanical and electrical to have a thorough knowledge base and experience on the latest vetted equipment and applications. With this in mind, all discussions point towards the selection of a data center Design/Build/Maintain contractor for several reasons:
- Single source responsibility.
- Has continuous exposure to the latest technologies including a wide range of multiple equipment manufacturers and their associated factory level applications.
- Includes system design layout to allow on-going maintenance and repair versatility while providing continued critical load protection.
- Provides best practices on all equipment installation.
- Ability to recognize the need for a critical path MOP and draft procedures for team distribution and task execution. This is very advantageous for existing operating facilities.
- Ability to provide data center level field technicians and supervision. Electricians and mechanical technicians who mostly construct residential or light commercial are not suitable for this high level data center buildout.
- Data center General Contractor can recognize and pinpoint possible bottlenecks and find solutions in advance of affecting budget and schedule.
- Design/Build/Maintain General Contractor does not leave the client with the possible overwhelming task of assembling an on-going maintenance support.
- A Design/Build/Maintain General Contractor continues with a full support maintenance solution.
Now its time to take a big breath and recognize a job well done. It was a very difficult task to not only perform the normal work load of the company but manage a very technical build. Additionally, the assistance provided with the on-going maintenance and support is comforting that no additional time and budget is required. The correct Design/Build/Maintain contractor was selected.