Hello homelabbers,

I want to share my story of building a home cluster for AI and software for it. I would appreciate your feedback.

Over time, I have acquired several computers, including two personal ones for work and gaming, one under the TV for gaming with gamepads and watching videos, an old PC repurposed as a server, and a collection of laptops. I’ve set up ethernet cables and employed Unifi for networking.

Considering the abundance of GPUs at our disposal, I’ve concluded that we can leverage them for various projects, such as running AI models for image generation. However, it wasn’t evident how I could use all my machines running different OSes and frequently used for gaming or work.

I’ve started tinkering and built a project to solve my needs over time. It gives me remote access to the cluster, allowing me to run docker containers, inspect them, and more. It runs on all three major OSes and has an auto-update feature to simplify cluster management. I apologize for posting a link to my project web page. I haven’t figured out a better way to illustrate the idea.

Is there anyone else who has built something similar? Do you believe this tool would benefit those engaged in home lab setups?

Does building a product out of this software make sense? My idea is to utilize the spare computational capacity from home or office computers to power your projects and save money on cloud services. The goal is to build a service that simplifies the cluster setup and management (i.e., avoid installing custom Linux distros, configure networking, NAT traversal, install dashboard software, etc. - just install the client on your machines, and that’s it). Here’s a demo featuring a stable diffusion running on my home machine.

    • fair-dikobraz@alien.topOPBEnglish
      1·
      10 months ago

      Indeed, inter-node communication is needed. I support a few communication protocols:

      1. Communicating through STDIN, STDOUT, STDERR with the container process. The link is automatically established between the initiating process and the target process. This form of communication is convenient for console applications when you want to pass some simple input to the command and see the result.
      2. Uploading and downloading files to and from the container. This is convenient if you run a big batch job, like generating a bunch of images or a video using AI.
      3. Communicating through an open port. This is what that HuggingFace demo is using. It spins up a server for image generation, and clients send requests and read responses. MPI can also be supported this way - you’ll need to open a port and start MPI processes in the containers you want to communicate. If using this method, it is up to the user to set up communication.

      So far, I have yet to consider providing any additional communication mechanisms. The most convenient and efficient way depends on the use case. That’s why I’m seeking feedback. Would you be using such a system for some of your projects?