Skip to main content


Showing posts from January, 2017

Show CPU info via command line

This gives a ton of information - way more that I generally ever need.
less /proc/cpuinfo This is the tidy version.
lscpu This is the min and max clock speed of the CPU:
cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_max_freqcat /sys/devices/system/cpu/cpu*/cpufreq/scaling_min_freq This is a cool command to keep track of the current CPU clock speed.
sudo watch -n 1 cat /sys/devices/system/cpu/cpu*/cpufreq/cpuinfo_cur_freq

Ubuntu Server 16.04 not detecting wifi card

This turns out to be a relatively simple issue.

This command is my go-to for internet connection diagnostics, but it wasn't showing my wifi card.
ifconfig This shows all interfaces, not just the activated ones.
ifconfig -a to activate a hardware device after determining its name, run:
ifconfig "device name" up Bonus: this gives you way more than you'll ever need to know about the hardware capabilities of the device.
iw list

(Internet via Ethernet) + (SSH via wireless)

= (sustained SSH during MPICH computing)
I've been losing SSH connection after starting process jobs on my new beowulf cluster.  This is my current fix, since my theory is that the network switch is so clogged with MPI-related communication (which does take place via ssh) that there's no bandwidth left for my administrative SSH connection.  Theory supported by observation that when I plug my unrelated control machine into the switch it can't ping google. assumptionsUbuntu 16.04.1 LTSworking wireless and ethernet: I had to do this and this. sources check wifi hardware capability Run the command  iw list And look for a section like the following.  If it includes 'AP' (see emboldened bit), you're golden.  If not, look for a different wirel…

Basic Vim

The cheat sheets and guides out there don't seem to provide a practical intro to Vim.  I'm not able to use MS Code on one of my primary interfaces, so I'm looking for the next best thing.  Vim, so I've heard, is probably it.  This is a great little tutorial to introduce the basics.

There's two modes: command mode, and insert mode.  Command mode is where you do things that would normally be accessed via cursor, arrow keys or a menu, and insert mode is where you type letters and they appear on the screen and you can use the arrow keys like you're used to.  When you open vim, you start in command mode.

This should get you to about a nano level of proficiency.
Basic Usageopen foo | vim foosave file | :wquit file | :qCommand mode | [ESC] | k Move cursor left, right, up, down | h l | jInsert here | iInsert new line below | oDelete char under cursor | xHere's a nice cheat sheet for further u…

RNN with Torch and MPI

This is being installed on machines running Ubuntu Server 16.04.1 LTS.  Does not work on Linux Mint (the torch install script doesn't detect that OS).

Most of the following installations have to be performed on each computer.  I didn't re-download everything, since it was going to be put in the same place, but I did cd in and re-run the installation procedure.  That ensured the necessary files were added to all the right places elsewhere in the system.

Here, I'm walking through the process of running Torch on a cluster.  CPUs, not GPUs.  The performance benefit comes from the slave nodes being allowed greater latitude in searching for local optima to 'solve' the neural net.  Every so often, they 'touch base' with the master node and synchronize the result of their computations.  Read the abstract of Sixin Zhang's paper to get a more detailed idea of what's happening.  As far as the implementation goes, "the idea is to transform the torch data …

A Homemade Beowulf Cluster: Part 2, Machine Configuration

This section starts with a set of machines all tied together with an ethernet switch and running Ubuntu Server 16.04.1.  If the switch is plugged into the local network router, then the machines can be ssh'd into.

This should be picking up right where Part 1 left off. src.  So.
Enabling Scripted, Sudo Remote Access The first step in the configuration process is to modify the root-owned host files on each machine.  I'm not doing that by hand, and I've already spent way too long trying to find a way to edit root-owned files through ssh automatically.

It's not possible without "security risks".  Since this is a local cluster, and my threat model doesn't include -- or care about -- people hacking in to the machines or me messing things up, I'm going the old fashioned way.  I also don't care about wiping my cluster accidentally, since I'm documenting the exact process I used to achieve it (and I'm making backups of any data I create).

Log into…

When Ubuntu Server doesn't let you keep the internet device drivers

So I ran into a problem while setting up Ubuntu Server 16.04.1 LTS for my cluster: one of my computers had restricted hardware drivers for the wifi and ethernet devices, and while the drivers were available on the install media, they weren't transferred over.  Something about non-free stuff.  So after installation, I was stuck with a computer that had no access to the web.

After a ton of searching, I a) found what the hardware problem was, and b) realized that I can use the install media as the installation source - that might sound obvious, but the keyword there is "can", not "should be able to". It's badly documented and very not-obvious.

Anyway, here's the source for using the install media.  Part of the install process is choosing which sets of packages you want to include - things like OpenSSH Server stuff or Mail Server stuff.  There's also one called Manual Package Selection.  It didn't seem to do anything during the install (though I d…

Install a package without internet

So first of all, this isn't original.  Credit goes here.  But it's fantastic, and I wish I'd known about this a long time ago.  As usual, for my own memory/use: and actually, I'm just going to clean up what the other guy said.  He did a great job.
On the Internet-less computer: In the terminal enter:
PACKAGENAME=<The name of the Package to install> and then
apt-get -qqs install $PACKAGENAME | grep Inst | awk '{print $2}' | xargs apt-cache show | grep 'Filename: ' | awk '{print $2}' | while read filepath; do echo "wget \"${filepath}\""; done > A ready-to-use downloader for the package has now been created in the home folder.  Open your home directory in the file browser and move the file to the top-level directory of your flash drive.  Then eject your flash drive.
On the computer with Internet: Insert your flash drive, and open your flash drive in the file bro…

A Homemade Beowulf Cluster: Part 1, Hardware Assembly

A beowulf cluster lets me tie miscellaneous computers together and use their cpus like one large processor...I think.  Never done this before, still working on the details.

I'm building this with random laptops: generally i5s, and I think one's a Core Duo - it's half decent thing from 2011.  Might even throw in an RPi2 for good measure.

Make sure you read through this before starting.  You want to know what you're getting into.  Watch out, though - this is a long post.
Notes Since we're working on multiple computers here, not everything is going to be cut-and-paste.  I will make sure that it's as clear as possible, however.  There won't be any hand-waving or assumptions of prior knowledge.

I'm doing this with Linux Mint 18 on my primary laptop.
Primary Sources Setting up the cluster: src 1,  src 2, src 3 Hardware IngredientsSince the benefit of this tool is sharing computations between computers, you need a way to route that information.  Hence, an …

A Recurrent Neural Network to generate sentences: step-by-step assembly (Mint version)

This one predicts (ok, procedurally generates) text word-by-word.  Others do it letter-by-letter, but then you have to deal with misspellings.  src. Inspiration.
Dependencies I'm basing this off an installation script referenced in the github readme (here).  Instead of blindly running the script, I checked its contents and discovered it fails when run on linux mint.  So I simplified it, and changed/added a thing or two since it wasn't working anyway.  In case anyone wants to blindly run my version, here it is all cleaned up.

Updating your sources is always a good idea prior to installing a bunch of software.  Also, the package below makes it easy to add new repositories (see below).
sudo apt-get update sudo apt-get install -y python-software-properties Now things are getting serious.  This stuff includes GCC 4.9, since it seems GCC 5 isn't compatible with Torch 7.  There was a weird error where cmake wasn't installed when I put all the packages in the same apt-get insta…