Analysis shows building own AI workstations costs approximately 10x less than renting cloud GPUs.

For those who have recently completed machine learning coursework and want to build intelligent code and neural networks without expensive cloud costs, building your own machine may be the answer. While training intensive neural networks requires substantial computing power that lightweight laptops cannot provide, renting GPUs on cloud platforms like Amazon Web Services (AWS) can cost hundreds or even thousands of dollars. According to Jeff Chen, an AI techie and entrepreneur at Stanford University in the US, building your own machine is approximately ten times cheaper than relying on cloud platforms.

Chen believes a suitable machine can be built for about $3,000 (~£2,300) without including tax. At the heart of this system is an Nvidia GeForce 1080Ti GPU, a 12-core AMD Threadripper processor, 64GB of RAM, and a 1TB SSD card for data. Additional components include a fan to keep the computer cool, a motherboard, a power supply, and a case to house everything.

Once the personal rig is built, the only recurring cost is power consumption. It costs $3 (£2.28) an hour to rent a GPU-accelerated system on AWS, whereas it costs only 20 cents (15p) to run on your own computer. According to Chen's calculations, after two months this works out to being ten times cheaper, though the gap decreases slightly over time as the computer hardware depreciates.

There are trade-offs to consider. As Chen explained, "There are some drawbacks, such as slower download speed to your machine because it's not on the backbone, static IP is required to access it away from your house, you may want to refresh the GPUs in a couple of years, but the cost savings is so ridiculous it's still worth it." While an Nvidia GeForce 1080Ti isn't the best GPU on the market, it performs adequately for most users unless dealing with extremely large datasets and models. The GeForce 1080Ti costs $700 compared to Nvidia's datacenter-grade Tesla V100, which costs between $8,000 and over $10,000. Chen noted that "The datacenter grade V100 is 25 to 75 per cent faster depending on workload, you don't get the 4 to 8x [speedup] promised because you get bottlenecked by Amdahls law and slow IO. You get more memory but 11Gb on the 1080 Ti should be good for most cases."

Nvidia has banned the use of cheap GeForce chips for datacenters after updating its end-user licence agreement, forcing cloud providers to purchase more expensive options. Jensen Huang, CEO of Nvidia, stated that GeForce cards were only really suited for gaming and cryptomining. Other alternatives exist, such as using Google's tensor processing units or buying pre-built machines, but these are not cheaper than building your own. Chen recommended sticking to Nvidia's GPUs, explaining that "Based what I've seen, [TPUs] don't seem that much cheaper but promises to be much faster. I've looked into [cheaper alternatives to Nvidia] and decided against it. Most lack full software community support and AMD's Vega is 25 to 50 per cent slower."