Imperfect Orchestration: Inside the Data Center’s Struggle for Efficiency

Computational Culture 8, July 2021

For the data center industry, efficiency is an imperative and a struggle. Machines require constant power and cooling, yet often have very low utilization rates, where jobs are actually being computed. To improve efficiencies, Alibaba began co-allocating jobs on the same array of machines and released datasets from this production cluster. This article explores this dataset via a series of data visualization. These images show, even on its own narrow terms, the dataset fails to achieve the vision of efficiency. The high diversity of job lengths and types create unevenness and conflict in the data center—work itself disrupts optimal efficiency. For the industry, efficiency should best be handled by machines, with automation freeing up operators for higher-level, higher-impact tasks. However, the logic of efficiency may itself be questioned. Efficiency introduces a particular framing, defining a problem, establishing a goal, and offering a compelling road map forward. If efficiency is clarifying, it is also obfuscating, bracketing out alternative ways of understanding these sociotechnical systems. As hyperscale technology companies dominate data centers, often what is becoming more efficient are exploitative or extractive operations. Efficiency should then be challenged, but without falling back to a familiar valorization of inefficiency. As the data center grows in significance while decentering human labor, it becomes important to develop a critical theorization of efficiency attuned to the new conditions of this machine-centric future.