by Ian Seaton | Jul 26, 2017 | Blog

如果你錯過了本系列七個部分中的前面六個部分，我將化時間澄清一下，本文不是討論利用封堵板和地板密封條堵塞漏風洞口、分離冷熱通道、最小化或消除旁流或回流、降低可變風量風機的運行頻率、智能調節(jié)通風地板開孔率或測量服務器入口溫度。我不認為這些做法是“考慮”，相反，我認為這些做法的性價比是最高的。這些做法不是數(shù)據(jù)中心最佳實踐中最先進或最前沿的，而是作為最新實際的是無需置疑的。根據(jù)目前既有的行業(yè)標準和實踐指南，這些氣流管理策略是你開始從控制流量和溫度（氣流管理手段）中獲益之前的最低起點，也是在數(shù)據(jù)中心提高效率和見到成效的關鍵。

考慮數(shù)據(jù)中心的氣流組織，可以在很大的層度上告知我們，利用優(yōu)秀的氣流組織手段，可以降低數(shù)據(jù)中心運營成本。我演示了，在大多數(shù)的運行情況下，在服務器的風扇功率轉變到高功耗前，數(shù)據(jù)中心能在比大多數(shù)從業(yè)人員想象高得多的溫度下運行。在第二部分中，我介紹了兩個構思良好，執(zhí)行良好的實驗研究項目的數(shù)據(jù)。這些數(shù)據(jù)表明，在不影響服務器工作性能的情況下，數(shù)據(jù)中心的工作環(huán)境可以接受更加高的溫度。然后，我建議與傳統(tǒng)觀點相比，實現(xiàn)無制冷化的數(shù)據(jù)中心是比較現(xiàn)實的。和主流的標準和行業(yè)的指南相比， ICT設備廠家提供了的更加寬泛的濕度標準。

本系列的前五部分，提供了來自制造商產品信息、中立實驗室研究結果和數(shù)學模型的論證，這些證據(jù)共同為設計、建設和運行數(shù)據(jù)中心（無冷卻裝置或制冷劑冷卻）的有效性提供了相當有說服力的論據(jù)。在最后一篇文章中，我演示了如何使用ASHRAE“X”因子來預測在不同溫度環(huán)境下對服務器壽命和可靠性的影響。到目前為止，我們的重點一直放在我們計算機設備的健康和福利上，并分析為這些設備提供對應環(huán)境的費用。今天，我從兩個重要的條件出發(fā)：我們將討論這一切對我們員工的影響，我將有更多的問題而不是答案。

最后需要考慮的內容是，在入口溫度較高的情況下，服務器風扇的噪聲會隨之增加，我們需要評估噪聲的影響。ASHRAE手冊提供了在較高的服務器入口溫度下，一般噪聲增加值的估計值，總結如下表1所示：

這些噪聲隨著風扇的功率的增加而增加，正比于風扇運行功率的5倍率，這意味著風扇的轉動轉速增加20%，將會導致噪聲增加4dB。然而，我第一次基于此方法進行計算時，發(fā)現(xiàn)結果有所不同。我查看了SHRAE技術委員會9.9 IT分委員會，編制的不同入口溫度下的服務器功率增加值，并根據(jù)我在本系列第1部分中使用的方法，來預估風扇的功率。簡而言之，我們假定，在入口溫度為20℃時，1臺服務器的功率是800W，風扇的功率為80W。當溫度升高時，服務器的功率升高值等同于風扇功率增加值，這些數(shù)據(jù)將在表2中的能耗Δ列中列出。

每一個溫度梯度增加值，與風扇功率的增加的百分比是精密對應的，然后，我利用風扇的功率與流量的比例關系(Q1/Q2)3 = P1/P2 （風扇功率與風扇轉速的三次方對比關系），取算風扇功率增加的立方根，計算相對于基準風扇的轉速比，在這種情況下，20℃和22.2℃的轉速比，例如?1.1 = 1.032，或10%的功率增加對應3.2%的轉速增加，對應的噪聲增加值參考如下公式（譯者注釋：此處文字描述的不是特別準確，以表格為準）：

由于我們不計劃研究服務器內部，并改變風扇尺寸，da/db的倍數(shù)始終是1和70倍，基于10的對數(shù)，1的對數(shù)值為0。因此，除了最后增加一個數(shù)值外，我們不需要關注其他任何值。因此，例如，在進口溫度為30℃的條件下，相對基準功率，風扇的功率會增加9.3%，因此50乘以以10為底的1.093的對數(shù)為后值為1.931dB，其計算值為入口溫度從20℃增加到30℃時，服務器噪聲的增加值，其值明顯小于 ASHRAE’s 表 1推薦的4.7 dB。

如我們表2中的計算值，尤其是數(shù)據(jù)是基于10的對數(shù)值。根據(jù)ASHRAE文檔中介紹20%風扇轉速增加產生4dB噪聲增加的例子，再次檢查我的方法，50乘以1.2的值以10對數(shù)是3.959，接近“4”，是合理的假定，因此，這些差異可能歸因于不同入口溫度下對服務器風扇轉速的不同假設。再仔細研究，ASHRAE的計算服務器主要基于A1級的服務器，而非目前通用的A3級服務器。但我們比較表6中的不同場景值時，讀者將要驗證是，他們使用較新的服務器還是使用傳統(tǒng)服務器。

在去年的MIPROS會議上，對數(shù)據(jù)中心的噪聲進行了深入的研究，其中包括一個實際數(shù)據(jù)中心的案例的研究。他們得出來的一個結論是，熱通道的噪聲普遍比冷通道的噪聲高，從表3的數(shù)據(jù)中心部分證明了此結論。雖然本案例收集噪聲數(shù)據(jù)，能幫助我們理解數(shù)據(jù)中心內的噪聲水平，但是，我不確定這些數(shù)據(jù)能為我們討論溫度升高時，對數(shù)據(jù)中心噪聲有多大影響，和提供多大的幫助。例如：從機房的平面圖中，收集了很多列間空調對噪聲的影響數(shù)據(jù)，發(fā)現(xiàn)列間空調對噪聲的影響比冷熱通道的影響更大一些。

表4的數(shù)據(jù)為表3中的列間空調的廠家噪聲數(shù)據(jù)，這些噪聲數(shù)據(jù)，可以表明列間空調的噪聲對機房的整體噪聲有多大的影響。

這個產品特別有趣的是，它規(guī)定噪聲的測試數(shù)據(jù)是距離噪聲源6英尺處的數(shù)據(jù)，而數(shù)據(jù)中心案例中，冷熱通道中的噪聲數(shù)據(jù)是冷熱通道端頭的噪聲數(shù)據(jù)，這個測試點距離噪聲源的距離是2英尺，不管測試點處是服務器機柜或者列間空調。而定義噪聲的損失方程是20乘以log10的距離值，因此，1.8米的噪聲的損失值為15.56dB，如果我們計算70%的風機轉速的時候噪聲為72.4 dB，在加上6英尺的噪聲損失值15.56 dB，噪聲源總的噪聲值88 dB。距離噪聲源(20log102) 2英尺處的噪聲值損失為3.5 dB，我們從噪聲源的聲壓中減去此數(shù)據(jù)，那么我們在冷熱通道中列間空調的背后測試的噪聲值應該是85.5 dB。我懷疑，如果機房空間內采用房間級精密空調或者室外的節(jié)能空調機組，Miljkovic的研究數(shù)據(jù)會有很大的不同。

事實上，服務器風扇并不是數(shù)據(jù)中心噪聲的唯一來源，當空調設備放置于機房地板區(qū)域內時，它們才是房間內噪聲的最大貢獻源。然而，我提醒讀者，我們今天討論的目標是，溫度升高時對噪聲的影響，這意味著由于制冷設備噪聲的影響，我們的目標是有利的。表5總結了不同風機類型的噪聲特性，在一般的產品中，剔除一些特殊案例，我們將看到服務器中的軸流風機和冷卻設備中的離心風機噪聲水平。

我們之前計算服務器入口溫度升高時，噪聲增加值Δ的方程，是有利于我們的冷卻設備的，因為我們利用了優(yōu)秀的氣流組織手段來增加了服務器的入口溫度，從而降低了我們的冷卻設備的風扇頻率。事實上，我們已經發(fā)現(xiàn)數(shù)據(jù)中心僅僅通過良好的氣流管理來減少需求，從而將其冷卻單元冗余從N 1增加到2N的例子。當我們可以將風扇轉速從80%轉/分降低到50%轉/分時，我們不僅可以兌現(xiàn)75%的節(jié)能支票，還可以享受10.2分貝的噪音降低，或1/10的聲音強度或大約1/3的聲音幅度。考慮到噪聲與距離呈對數(shù)減少，即在數(shù)據(jù)中心內部，可以減輕或基本上消除機械設備噪聲的程度，取決于較高功率水平下的服務器風扇噪聲源，被空間隔離控制程度，以便將周邊工作區(qū)域的危險降到最低。

數(shù)據(jù)中心在高溫的工況下運行時，由于缺乏精確性和分辨性，需要進一步對相關的噪聲問題進行闡述，下表6中，我基本不同的數(shù)據(jù)點的一些比較，對噪聲問題進行了不同的數(shù)據(jù)匯編。70 dBA和80 dBA基準值來自多個數(shù)據(jù)中心，是數(shù)據(jù)中心噪音水平的典型范圍，其中包括所有IT設備以及電氣和機械設備。75.5 dBA基線是Miljkovic研究報告中的所有熱通道和冷通道測量的平均值，84 dBA基線來自ASHRAE假設，此假定可能有些危言聳聽，或許數(shù)據(jù)中的功率密度都沒有達到所設定的功率值造成的。這四個基準值下的左欄顯示了ASHRAE A3級服務器，在不同溫度增量下服務器風扇噪聲的增量，而這基準值下的右欄顯示了ASHRAE A1級服務器的噪聲級預測。

為了評估這些噪聲功率級的增加值是否準確，我們需要引入整體環(huán)境來綜合考慮此問題。噪聲的基準值，可能因為冷卻設備的類型、位置、和冗余方式受到影響。在噪聲值大于85 dBA以上時，需要考慮進行聽力保護。表7，展示了在不同的噪聲標準下，最大的人體噪聲暴露時允許值和推薦值，為了保險起見，建議至少在數(shù)據(jù)中心內做一次有效的噪聲測試。此外，最近的報告指出，在服務器入口溫度增加到35℃以上時，需要邀請聲學工程師參與，來制定監(jiān)測和保護計劃，以及在數(shù)據(jù)中心的建筑空間中，設計緩沖的建筑元素來吸收或者轉移噪聲。

綜上所述，除了包括保護數(shù)據(jù)中心工作和參觀人員的聽力免受損害和遵守相關的安全規(guī)定外，還有一項研究表明，噪音暴露會影響人體的分心、內分泌反應等以及相關的精神和心血管疾病，由此看，關注數(shù)據(jù)中心的噪聲設計和操作，是非常有意義的事情。我已經報告了相關的研究數(shù)據(jù)和基本的工程計算原理，來說明如何預測數(shù)據(jù)中心內的危險噪聲值可能出現(xiàn)的區(qū)域。然而，由于缺乏廣泛的噪聲研究，我們不可避免的向更高密度和更高效節(jié)能方向前進，伴隨著更高的操作溫度，建議更準確地理解在機房地板空間外所有可移動的冷卻設備的影響。了解通道封閉的結構組成和帶有封閉的排氣機柜降低了多少的噪聲值，以及哪些實用方法可以消除數(shù)據(jù)中心工作員活動區(qū)域的空氣流動，這也很有用。

Airflow Management Considerations for a New Data Center – Part 7: Server Acoustical Noise versus Inlet Temperature

In case you missed the first six parts of this seven-part series, I will take just a moment to clarify that this will not be a discussion on the criticality of plugging holes with filler panels and floor grommets, separating hot aisles from cold aisles, minimizing or eliminating bypass and recirculation, deploying variable air volume fans, intelligently locating perforated floor tiles and measuring temperature at server inlets. I do not consider any of those practices to be “considerations”; rather, those practices are what I call the minimum price of admission. None of these practices fall into the state of the art or leading edge categories of data center design, but are firmly established as best practices. By all established industry standards and guidelines, these airflow management tactics are the minimum starting point before you can start benefiting from being able to control airflow volume and temperature – the activity of airflow management, and the key to exploiting both efficiency and effectiveness opportunities in the data center.

Airflow management considerations will inform the degree to which we can take advantage of our excellent airflow management practices to drive down the operating cost of our data center. In previous installments of this seven-part series, I demonstrated that data centers could be run warmer than conventional wisdom would suggest before increased server fan energy reversed mechanical plant savings before server performance was adversely affected and before server price premiums consumed mechanical plant savings. I then suggested chiller-free data centers are much more realistic than conventional wisdom might purport and provided evidence that IT equipment OEM’s tend to generally allow for wider humidity ranges than mainstream standards and industry guidelines. The first five parts of this series provided evidence from manufacturers’ product information, independent lab research results and math models that together make a rather compelling argument for the efficacy of designing, building and operating data centers without chiller plants or refrigerant cooling. In the last piece, I demonstrated how the ASHRAE “X” factor can be used to predict the effect on server life and reliability operating in different temperature environments. Up to this point, our focus has been on the health and well-being of our computer equipment and controlling the expense of managing the environment for that equipment. Today, I depart from my previous focus in two important ways: we’ll discuss the effect of all this on our staffs and I’ll have more questions than answers.

The final consideration has to do with the effect of operating the data center at a higher server inlet temperature on fan noise produced by the servers in response to those temperatures. The ASHRAE handbook provides some general estimates for increased noise exposure at higher server inlet temperatures, summarized in Table 1 below.

These noise level increments are based on fan laws that describe sound power levels of fans increasing with the fifth power of rotational speed, meaning that a 20% increase in fan speed will result in a 4dB increase in noise level.2 However, my own first pass at this baseline assessment produced slightly different results. I looked at the server energy increases at different elevated inlet temperatures compiled by the ASHRAE Technical Committee 9.9 IT Sub-committee3 and backed into a fan energy estimate per the methodology I used in part 1 of this series.4 In short, I assumed an 800-watt server with a nominal fan energy budget of 80 watts at 68?F inlet temperature and applied the total server energy increase at each temperature increment to the fans. Those results are reported in the Energy Δ column in Table 2 below.

The Increase column is merely the percent increase in fan energy at each temperature increment. Then I applied the affinity fan law of (Q1/Q2)3 = P1/P2 (changes in fan energy are the cube of changes in fan rpm), by taking the cube root of each increase in fan energy to calculate the RPM ratio against the baseline, in this case 68? and 72?, e.g. ?1.1 = 1.032, or a 10% increase in energy equated to a 3.2% increase in fan speed. The noise Δ is calculated by the following equation:

Since we don’t plan to open up our servers and change any fan sizes, d_a/d_b will always be 1 and 70 times the base ten log of 1 is 0, so we don’t need to concern ourselves with anything but the last addend. Therefore, for example, at 86?F server inlet temperature, our fan speeds have increased 9.3% from our base condition, so multiply 50 times the base 10 log of 1.093 to get 1.931 dB as the increased server fan noise at 86?F versus 68?F. This 1.931 dB is significantly less than the 4.7 dB expected increase in sound power level from ASHRAE’s Table 1 above, as are all the calculated values in Table 2, especially since decibels are a base 10 logarithmic scale. Double checking my methodology against the example from the ASHRAE passage of a 20% fan speed increase producing a 4 dB sound increase, 50 times the base ten log of 1.2 is 3.959, close enough to “4” to be a reasonable confirmation, so the differences are likely attributable to different assumptions about server fan speeds at different inlet temperatures. On closer examination, the ASHRAE calculations appear to be based on legacy Class A1 servers, rather than on current generation Class A3 servers. When comparing the different scenarios in Table 6, readers should consider whether their experience will be with newer servers or legacy equipment.

A very thorough study of data center noise was presented at last year’s MIPROS Conference and included noise level measurements from an actual case study data center. One of their conclusions was that noise levels were typically higher in hot aisles than in cold aisles, which is only partially demonstrated by their data summarized in Table 3 below. While this data is useful as contributing to the data set for helping us understand noise levels in data centers, I am not sure how much useful information it will contribute to our discussion on the effects of higher temperatures on data center noise. For example, the site plan from which this data was collected showed various numbers of in row coolers in different rows and the data suggests the in row coolers may have been more responsible for noise levels than the difference between hot aisles and cold aisles.

Table 4 below is from the manufacturer’s documentation for the row-based coolers used in the study reported in Table 3 and application of these sound specifications show how much they contribute to the overall noise data collected.

What is particularly interesting in this product data is that it specifies sound measurements at six feet from the sound source, and the data center study sound data was all recorded at head level in the center of either a hot aisle or cold aisle, which puts those data points all within two feet of sound sources, whether that be a server cabinet or a row-based cooling unit. The equation for determining the loss of sound pressure over distance is 20 times log base 10 of the distance, so at six feet that loss would be 15.56 dB. If we use the 72.4 dB at 70% fan speed from the manufacturer’s documentation and add that 15.56 loss at six feet, we calculate 88 dB at the source. At two feet from the source (20log102) we get 3.5 dB, which we subtract from the source sound pressure level and get 85.5 dB in the center of a hot aisle right behind one of those coolers. I suspect the data from the Miljkovic study would be quite a bit different if the space was using perimeter precision cooling or economizer air movers outside of the white space.

In fact, server fans are obviously not the only source of noise in the data center and when cooling units are on the floor, they are typically the largest contributor to the overall noise level. However, may I remind the reader, our subject today is the effect on noise levels of raising temperatures, which means our objective here will typically work in our favor with cooling equipment. Table 5 summarizes the noise characteristics of different fan types and in a broad brush stroke, we will see axial fans in servers and centrifugal fans in cooling equipment, with a variety of notable exceptions.

The noise Δ equation we used earlier to calculate noise increases in servers at higher temperatures works in our favor with our cooling equipment as we take advantage of excellent airflow management and resultant higher temperatures to ratchet down our cooling unit fans. In fact, I have seen examples of data centers increasing their cooling unit redundancy from N 1 to 2N merely by reducing demand through good airflow management. When we can reduce our air movers from 80% RPM to 50% RPM, we not only cash a check for a 75% energy reduction, but we enjoy a 10.2 dB reduction in noise, or 1/10 the sound power or approximately 1/3 sound amplitude. Given the logarithmic reduction in noise over distance, the degree to which mechanical plant noise can either be mitigated or essentially removed from the data center space, the effect of server fan noise sources at higher power levels can be space-contained to minimize the hazard in perimeter work areas.

To further illustrate our lack of precision and resolution on the issue of noise in the data center at higher temperatures, I have compiled the different scenarios I have discussed in Table 6 below to offer some comparisons of these different data points. The 70 dBA and 80 dBA baselines are from multiple sources as the typical range for noise levels in data centers, which will include all the IT equipment, as well as electrical and mechanical equipment, i.e., cooling. The 75.5 dBA baseline is the average of all the hot and cold aisle measurements in the study reported on by Miljkovic and the 84 dBA baseline is from an ASHRAE assumption8 that may be somewhat alarmist or perhaps based on projected densities not yet realized in most of the data center marketplace. The left-hand column under each of those four baseline scenarios shows the incremental increase in server fan noise at the different temperature increments for ASHRAE Class A3 servers and the right-hand column under each of those scenarios shows those noise level predictions for ASHRAE Class A1 Servers.

To determine if these estimated incremental increases in sound power level are problematic, they need to be considered in terms of the total environment into which they are being introduced. The baselines could be adjusted up or down by the type of cooling deployed, the location of deployment and level of concurrently running redundancy. Hearing protection programs are mandated beginning at exposure to 85 dBA and Table 7 shows maximum exposure time requirements and recommendations for different hazard levels. Prudence would suggest the efficacy of at least doing a noise level check in any data center. Moreover, the data reported on here clearly points to the need to involve an acoustics engineer for any plan to allow server inlet temperatures to creep up into the 95?F range for the development of a monitoring and protection program, as well as developing mitigation architectural elements into the total space design for absorption and redirection strategies.

In conclusion, in addition to protecting data center workers and visitors from suffering hearing loss and complying with relevant safety regulations, there is a body of research indicating a relationship between noise exposure and such non-auditory conditions as a distraction, endocrine responses, psychiatric disorders and cardiovascular disease.10 It obviously makes sense to pay attention to this element of our data center design and operation. I have reported on relevant research and basic engineering principles to show where auditory hazards in the data center can be expected. Nevertheless, there is a shortage of extensive research and our inevitable movement toward higher densities and increased energy conservation awareness, accompanied with likely higher operating temperatures, suggest it would be useful to more precisely understand the impact of moving all cooling equipment outside of the white space. It will also be useful to understand how much noise exposure is reduced by aisle containment structures and cabinets with closed loop exhaust chimneys, and what practical approaches might eliminate all air movement for the data center from people-occupied space.