Cloud Computing what is it's environmental impact, the real cost.

Introduction

There is a large misconception when discussing the subject of cloud computing from some business who believe that cloud computing is "green" and environmentally conscientious this is a naive view to take, which leads to the inappropriate analysis of the actual impact involved with cloud computing. Cloud computing at it's core is another computer hosted over the internet, it's nothing more and nothing less. This also comes with it the environmental implications from the way a cloud provider goes about facilitating businesses ability to operate. There is also this strange belief with many businesses that cloud computing will solve all their problems and this belief often fails to take into consideration the real engineering and more or less logically scientific analysis of that cost of operation, while it may shift the CapEX responsibility away from the business it often comes with OpEx (Operational expenditure) and if not examined carefully environment cost.

Cost of Running a Data Center

Data centers are at their core a cluster of computers interlinked together with networking, server, cooling, lighting, and other hardware. All these devices use electricity and this means a consistent stable supply of electricity is required. The actual cost of running a data-center is a complex interplay with regularly hardware maintenance cycles and continuous power delivery systems, and redundancies. The servers themselves it depends on equipment and age of the hardware installed in the data racks provided by a cloud provider, some architectures are often more efficient than others, and the kinds of workloads with their supporting hardware. On the brains of the cloud server side the cpu architecture type can impact power consumption quite heavily. For example Amazon have their Graviton Series processed based on ARM architecture the reason this is significant is because ARM based CPUs naturally consume less power because of their lack of a CISC design, where x86-64 architecture might have 1000's of instructions in it's instruction set ARM only has a few hundred, this means chips themselves can be much smaller and take up less space, allowing for more cores to be easily integrated into those chips leading to less silicon space being need and leading to reduced power consumption.

The AI element

AI accelerators can consume large amounts of power, the reason for this is many AI models are not actually particularly "efficient" models when training can slip into sub-optimal configurations leading to efficiency reductions , mathematically this is represented by a higher dimensional space, and that space has local minima and absolutely minima, the process of training a model is "pacing around" this higher dimensional space looking for that absolute maxima, this process is tuned via hyperparameters, such as the learning rate, and other input values, these are tunables that affect the overall result of the machine learning model. To train AI models often uses graphics processing units such as Nvidia accelerators, such as A100's and other models from Nvidias line, again these units are quite power hungry processors, and need significant backing from hardware stand point of raw processing power and the cooling necessary to keep them operating optimally.

Practice Advice for how to minimize cloud environmental impact.

Designing a cloud deployment to reach "Green IT goals" is a complex process anyone who say it is not does not understand the intracies of the cloud infrastructure that underpins the worlds systems.

1. Artificial intelligence - Based workloads can benefit by optimizing their training process, there are many ways to optimize AI workloads, but one very effective one which also reduces for example in LLMs is to use quantized models and neural-pruned models, this is not unlike how the human brain performs a process called "neural pruning" getting rid of the connections which are redundant or useless, and quantization refers to storing a model in a manner that reduces the overall processing power necessary by reducing the data sizes necessary to store the models parameters, ideally this optimization is done for example by using smaller data types given a model stores parameters as float values for example under C++ float datatypes take up 64bits of precision or 8 bytes as you scale up to billions of parameters from the likes of LLaMA scaling the model down in terms of space occupied reduces considerably, this actually reduces the power consumption requirements for workloads which are using an LLM via inferencing mode.

   Hardware choice - AI workloads can often have optimization done by using different hardware, the mainstream technology world often uses Nvidia based accelerators this can often be a good choice, however if a model is compatible with Google GCP TPU units, they can actually cause an actionable reduction in power consumption graphics cards are all about pure brute-force, this is often not the most desirable method of training models purely because they consume large amounts of unnecessary power, on the other hand Google TPUs can offer massive power reductions in an AI workload, TPUs also known as Tensor Processing Units they are purpose built chips which run at much lower frequencies than Nvidia accelerators, and this leads to large power reductions, Google chose to use TPU technology because they saw their AI workloads consuming large amounts of power beyond what they currently had, and determined it would result in a very large increase in cost, so Google designed the TPU - Tensor Processing Unit, the purpose is in the name Tensors are generalization of vectors into the R^n space, that is any arbitrary dimension. If AI accelerators continue to be developed there was a promising field called analogue AI accelerators, these offered singificant improvements in power requirements for datacenters, Mythic AI was a company planning on bringing an analogue processor to market which combined the best of digital and analogue together, unfortunately the company has been radio-silent for a number of months.

   
   

   Distributed Computing implementation - Another key way to reduce power consumption of cloud workloads is to distribute the workload between the edge and the core of a cloud deployment, where it is feasible looking at using edge based processing at a business where possible but also using the core for anything that requires considerable resourcecs, this reduced power consumption by avoding the compute power resources that cloud may have and the infrastrcture as this is a double edged sword, more infrastructure more redundnacy more power, so if you can successfully move part of your cloud workload to premises it can further ben used to reduce power demands, there is many different designs for this kind of system, but for example IoT sensors can be used in this case, where sensor data is distributed via a mesh network, with a primary aggregator node which takes all the data from the mesh network. In a Peer-to-Peer based design, there is also Cooporative based computing where the core and the edge are engaged to improve workload and resource performance as well as overall energy demands. It should be noted that some workloads are suitable for this and others are not, as this design is what we call heterogenous computing it has many details that must examined carefully. Water consumption - This is a big elephant in the room data centers naturally consume a lot of water, but with the recent advent of artificial intelligence the amount of water being consumed by cooling solutions of data centers is leading to a huge explosion in power demends, on another end of the spectrum there is ways to get the benefits of cooling with reduced environmental water impact and that is via engineered-fluid cooling such as NOVEEC engineered hydrocarbon fluid which boils at a low temperature, non-toxic, non-flammable fluid which achieves unparalleled cooling performance, while also retaining fluid cooling, while this may be a more expensive approach racks can be submerged into laying down trays or they can be installed into vertical standing racks, or they can use designs that are "partially submerged" where the heaviest heat generating components are submerged and the rest of the server not, this leads to a reduction in the amount of engineered fluid being needed reducing costs, in some cases exotic cooling can potentially be used see System Cooling Methods

2. Dynamic Scaling - During periods of long low utilization to save on both cost and potential environmental impact autoscalers can be used to reduce the overall amount of compute power available, this might include either dynamically reducing the number of virtual machines and their runtime or moving workloads via vertical downscaling to lower powered hardware, this autoscaling down in processing power "should" generally lead to reductions in power consumption therefore allowing your business to better reach the actual "green IT goals" you maybe targeting.

   
   

3. Cloud Provider responsibility - Some cloud providers will use renewable energy sources to power their facilities, this does lead to a reduction in grid power dependency during the day light hours, but will lead to increased on peak power consumption, as many people are aware on-peak between 6-12PM at night when everyone is home should lead to proportionally higher energy consumption the same can be applied to a cloud provider, lets face it cloud providers and companies in general will perform green washing, to make them look more "ESG" friendly than they really are, so even if you use dynamic autoscaler reduction your reduction may not actually lead to power consumption reductions as the virtual machines you power down may be reallocated to another client or customer leading to a zero sum game, where your cost reductions did not result in any improvements in your "green IT" perceived goal.

   
   

4. Battery arrays and online power storage - In some cases cloud providers may instead of drawing on the grid in the day use backup battery redundancy systems, to provide facility power during night hours, this ensures the system does not pull power directly from the grid at night, this ensures power reductions by using onsite storage. It should be noted that batteries even have an environmental cost, something many people misconceive about battery storage is the cycle durability and how fast they can degrade

5. leading to premature failure of the batteries, this inherently leads to recycling being required for the failed batteries which may result in environmental costs as lithium-ion battery are far from "fully recycled", this offsets the perceived benefits from using batteries and potentially ruins the green rating of your IT goals.

   
   

Scenario Example.

AgriData Central - Are a company who uses data insights from farming practices to improve product yields and reduce costs while also maintaining an ethical operational practice. They use a combination of cloud computing to perform analysis on corps, weather and other data sources to predict potential issues with harvesting and forecasting profits for a farming company.

Edge based computing - Is combined with cloud computing here in cooperative computing, sensors are used to monitor the ground nutrient content as well as the ph, humidity, and other data point, this information is then fed into a mesh network in a slave-master or multi-master configuration lightweight sensors take in the data and send it to the multi-master integrators which contains basic data every 30 minutes where the master nodes, then process the data into a JSON format to be dispatched to the cloud server on the other end. ArgiData Central also uses Google Cloud GCP to perform deep learning processing with their TPUs, AgriData did consider using Nvidia but found it did not align with their goals and data protection. Google GCP TPUs offered large amounts of scalability with added environmental targets due to the companies goals they try to ensure they reduce their energy and enviromental impact.

Cloud offloading - Google TPUs are used at the network core of the cooperative infrastructure to provide reduced power consumption and googles other services such as their cloud machine learning services. Google scalable Kubernetes clusters are used to ingest the data then issue it to TPUs, for processing.

Sensor data - To further make nodes more environmentally friendly and improve repair rates, sensors are also sending periodic health reports on battery information which contains temperature measurements from critical components like batteries, and how predictive maintainence is able to determine when batteries may potentially fail as well as integrating their charge status and solar cell power status monitoring for degradation of the cells over time.

Sensor nodes - The nodes themselves are powered by RISC-V, AgriData chose RISC-V for it's ARM-like based implementation and reduced operating costs associated with actual ARM, and wrote it's own realtime operating system for data monitoring, the board which the sensors are based on are custom designed by AgriData, with dual power redundancies, using lithium-iron-phosphate batteries to improve their thermal durability as AgriData is based in Australia and the node had to withstand heavy amounts of weathering in high temperatures.