Cash In on the AI Boom by Renting Out Your Spare Compute

If you own an at-home server, a gaming computer, or just a laptop that doesn’t get much love, listen up. You can now put that spare computing power to use and earn some passive income in the process. AI companies are hungry for more compute to run AI inference—the process of using a pretrained model to respond to queries—and they’re willing to pay you for it.

“Imagine Uber or Airbnb, but for AI-inference computing tasks,” says Ilman Shazhaev , founder and CEO of Far Labs , based in Abu Dhabi.

The AI boom has spurred on the construction of massive data centers , often damaging local communities by raising electricity prices, straining local water resources, causing environmental damage and noise, and being just plain ugly. Huge data centers are likely not going anywhere—training new frontier models and running AI models from leading companies will likely still be the purview of these behemoths. But now, several companies are providing AI inference on smaller, mostly open-source models. They are running inference on the preexisting computing power spread throughout homes and small businesses, and compensating the owners.

“Everyone thinks the only way to do it is data centers. And data centers are extractive for the communities in which they’re built, and they don’t return services or taxes or much of anything to the people there. So why not just turn this whole thing on its head?” says John Federico , founder and CEO of Evolving Edge , in Austin, Texas. “The compute power is out there. If you can orchestrate it, then you’re actually adding value to those communities directly.”

The idea isn’t entirely new: From 1999 to 2020, a volunteer-based project called SETI@Home used spare computers to search for signs of extraterrestrial life in radio-telescope data, for instance. But now, commercial companies are eager to use the same strategy. Shazhaev’s Far Labs is launching its platform Far AI in the coming weeks, while Federico’s Evolving Edge is currently in open beta. Other companies, like Bless Network , Salad , and Gradient have started to provide similar platforms over the last year.

Connecting to the network

Federico has been a computer hobbyist since youth, and he has amassed a whole server in his basement to run his projects. “It just hit me one day—there’s all this talk about not having enough compute, and I just thought, well, 92 percent of the country has broadband, and you have people like me who have mini data centers in a closet,” he says.

Federico sees the potential hosts as people much like himself who have already invested in home servers, and he aims to make the process of selling spare compute as seamless for them as possible.

“Sign up for the program, install an application,” Federico says. “All we want to do is run jobs on your machine when you tell us we’re allowed to. The only thing we do is monitor the resource usage. And of course, you can give us a schedule.” With a large enough network of devices, the platform would have compute available whenever it’s needed.

Privacy and security are primary concerns for such hosts. To reassure the users that their local data is secure, and that no malware will be downloaded to their devices, the team open-sourced their scheduling software. “The node software is open source, so anyone can look at it, see what it does,” Federico says.

Shazhaev of Far Labs explains that the company’s software is designed around a principle known as “least privilege,” which grants both the host and the user the minimum access possible to accomplish the task. Inference runs as an isolated workload with authenticated, encrypted communication and explicit limits on the GPU, CPU, memory, storage, and network resources it may use. Customers do not receive arbitrary access to the host machine, and providers can inspect resource use, pause the node, revoke access, and remove the software at any time.

The protection also works in the other direction. Workloads are segmented, and only the minimum required information is exposed to an individual node. Sensitive enterprise workloads can be restricted to controlled hardware rather than routed through consumer devices.

Divide and conquer

Massive data centers still have advantages from the user perspective: top-of-the-line GPUs and CPUs, high-speed networking, thick cables, and sophisticated cooling . User devices are usually less powerful, more varied, and less reliably connected to one another.

“This is quite a difficult issue from the science angle,” Shazhaev says. “You want to do a similar level of tasks that are happening in those high-infrastructure data centers, and run them on the user device with limited capacity.”

Evolving Edge’s Federico says this is an issue for the largest, state-of-the art AI models. But those are not always needed and are often not even preferred. “There are numerous companies, once they reach a certain scale, suddenly paying for tokens on a state-of-the-art frontier model [that] no longer makes sense for their needs,” he says. “Instead, they are fine-tuning open-source models for specific tasks that they have in their business. These models don’t require anywhere near the resources that some of the state-of-the-art models do. It’s just using the right tool for the job.”

Smaller, open-source models can often fit on a single user device. But if that fails, there are tools to split a single inference task over multiple GPUs or CPUs. Evolving Edge is using an open-source tool called Ray to perform this splitting, while Far Labs has developed its own proprietary software that not only splits the workload but also wraps the splitting in a layer of security and reliability-providing software.

“One thing we have done is we shared the model,” Shazhaev says. “We take the model, we cut it into many pieces, and then these pieces will be distributed through different devices. And we have an orchestrator and a load balancer which manage the task flow, so each device processes a part…

News is gathered automatically from public robotics & AI feeds on a schedule.