I'm building this with random laptops: generally i5s, and I think one's a Core Duo - it's half decent thing from 2011. Might even throw in an RPi2 for good measure.
Make sure you read through this before starting. You want to know what you're getting into. Watch out, though - this is a long post.
NotesSince we're working on multiple computers here, not everything is going to be cut-and-paste. I will make sure that it's as clear as possible, however. There won't be any hand-waving or assumptions of prior knowledge.
I'm doing this with Linux Mint 18 on my primary laptop.
- Since the benefit of this tool is sharing computations between computers, you need a way to route that information. Hence, an ethernet switch. Go for a gigabit, since you don't want the switch to be your bottleneck. I was cheap and got myself the 5-port version, and I'm already kicking myself. Go for the 8-port version at least. Here's the 5-port version I got, for consistency's sake.
- A ton of ethernet cables. You can do with short ones, but you'll want to connect the switch to your router so you can access the computers from outside their own tiny network.
- Leftover computers. You won't be using these for anything else, so make sure you don't need them.
Install Ubuntu Server to each computer. I used 16.04.1 LTS. For those with a penchant for funny names (or who need to deal with annoying, obtuse colloquialisms on the web), that's the Xenial Xerus edition.
You'll need to keep track of computer names ("hostnames"), and install consistent users (so the username and password are the same on each machine). I like to increment the computer names with letters. For the purposes of this guide, we'll use:
hostname: grendel-a username: beowulf password: hrunting
Create installation USB flash driveGo here to download the 16.04.1 LTS server ISO. That link initiates the download; here's a bit of context.
Burn the ISO to a USB flash drive. Note that you'll lose everything on the USB you use. Run the following command twice, once before plugging in your USB and then a few seconds after. The new entry when you run it the second time is your flash drive.
lsblkFor example, this is the output when I run that command with a flash drive plugged in. Note on the right where it specifies the mount point (at /media/me/storage), and on the left where it shows me the name is sdb.
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 167.7G 0 disk ├─sda1 8:1 0 487M 0 part /boot └─sda5 8:5 0 167.2G 0 part ├─mint--vg-root │ 252:0 0 159.3G 0 lvm / └─mint--vg-swap_1 252:1 0 7.9G 0 lvm [SWAP] sdb 8:16 1 1.9G 0 disk └─sdb1 8:17 1 1.9M 0 part /media/me/storageIf your USB is mounted, it has to be unmounted first - else weird things can happen in the next step. Trust me: once I didn't unmount a partition before copying it with dd, and my MBR was wiped out instead. In the example above, unmounting would work like this:
umount "/media/me/storage"If we pretend your USB is the sdb device, this is the command you'd run (I'm assuming the ISO was saved to the default "~/Downloads" location). Swap out the 'sdb' part with what the lsblk command indicated. Also be aware that if you mess this up, you will probably destroy whatever computer you're running the command with. Just FYI.
sudo dd if=~/Downloads/ubuntu-16.04.1-server-amd64.iso of=/dev/sdb bs=4M status=progress
Install to each computerYou're gonna have to follow this procedure with each computer in the cluster. There isn't a simple way around it, that I'm aware of.
Plug in the flash drive, and reboot/turn on the computer. Make sure it's connected to the web. Press ESC, F1, F2, F11 or F12 to choose a startup device...if those don't work, Google:
"how to choose startup device BIOS" & [your computer type]e.g., "ThinkPad T420".
When it boots up, there's a few settings to make sure of. Most of them should be straightforward: choosing a keyboard layout and a default language, for instance. Just in case of problems, here's a full walkthrough.
It asks for confirmation; [right arrow] and [enter].
Another confirmation. [right arrow] + [enter].
Unless you have weird internet, leave this blank and press [enter]. If you do have a weird internet setting, then I can't help you.
GRUB installed to the MBR. [enter].
Final Adjustment: power management
There is one more thing, though: you'll be running a bunch of these computers in the cluster, and you don't want to have to deal with them individually. For one thing, that would be a huge mess. So to keep them consolidated, you'll want to keep their lids shut while they're running: and if you don't change a setting, they'll just go to sleep when you close the lid. src.
sudo nano /etc/systemd/logind.conf
Find the line:
and change it to:
Then restart that bit of the system:
sudo service systemd-logind restart
Assembling the Cluster HardwareThis is pretty straightforward: start with your ethernet switch.
- Run a cable from your router to the #1 port on the switch.
- Run a cable from each computer in the cluster to a port on the switch. I'm not going to try and correlate port numbers and computer numbers. Pretty sure it doesn't matter.
And that's all it takes to assemble the cluster hardware! You should be able to ping your computers and get a response:
I built one a couple of years ago for embarrasingly parallel GIS. Two things you might find of interest:
1) Sometimes adding a slow node can slow your cluster down rather than speed it up as everybody waits for the slow node to finish its jobs. Depending of course on the relative size of the jobs and the relative speed of the nodes.
2) A lot of your time may be saved by building a roll-your-own ubuntu installer disk/usb
A description of my build is here: http://www.purplelinux.co.nz/?p=160
Hello i am still new with this clustered computing thinggy. I have a question, what kind of data does this cluster computer process?
So I was wondering if it was possible to combine two old macbook pros from around the same time to make them faster as one computer...? I know Ubuntu doesn't have Mac code so it cannot be ran on a Mac... so is it possible? If so... how?
Post a Comment