Project Documentation

Cluster Nodes and IP Addresses

The distributed computing cluster consists of eight servers. Each server is assigned both a 10 Gbit/s interface and a 1 Gbit/s interface, using the following addressing patterns:

Server 10 Gbit/s IP (192.168.88.x) 1 Gbit/s IP (192.168.0.x)
Server 1192.168.88.11192.168.0.21
Server 2192.168.88.12192.168.0.22
Server 3192.168.88.13192.168.0.23
Server 4192.168.88.14192.168.0.24
Server 5192.168.88.15192.168.0.25
Server 6192.168.88.16192.168.0.26
Server 7192.168.88.17192.168.0.27
Server 8192.168.88.18192.168.0.28

The 10 Gbit/s interfaces are used for high-throughput cluster communication, while the 1 Gbit/s interfaces are used for management, SSH access, and NFS traffic.

Installing Ubuntu

  1. Download the Ubuntu ISO: ubuntu.com/download/desktop.
  2. Create a bootable USB drive using a tool such as Rufus.
  3. Prepare each server by configuring a RAID 10 array.
  4. Press Ctrl + H during startup to open the WebBIOS utility.
  5. Launch the Configuration Wizard.
  6. Create a RAID 10 spanned drive group:
    • Assign two drives to span 0 and two drives to span 1.
  7. Confirm and finalize the RAID configuration.
  8. Boot from the USB drive and load the Ubuntu installer.
  9. Complete installation using default options. Be sure to select the option to install additional drivers.
  10. Reboot the server once installation is complete.

Installing OpenMPI

sudo apt install openmpi-bin openmpi-common libopenmpi-dev
mpirun --version

Example hostfile configuration:

node1  slots=16
node2  slots=16
node3  slots=16

Configuring SSH

Assigning Static IP Addresses

  1. Physically connect all servers to each other via Ethernet.
  2. Open the network settings on each machine.
  3. Switch the IPv4 configuration mode to "Manual."
  4. Assign a unique IP address to each server within the same subnet.
  5. Repeat this configuration after each reboot unless the settings are made persistent.

Enabling Passwordless SSH

ssh-keygen -t rsa
ssh-copy-id user@node2
ssh-copy-id user@node3

Configuring NFS

Server Configuration

sudo apt install nfs-kernel-server -y
sudo mkdir -p /srv/nfs_share
sudo chown nobody:nogroup /srv/nfs_share
sudo chmod 777 /srv/nfs_share
sudo nano /etc/exports

Add the following line to /etc/exports:

/srv/nfs_share 192.168.1.0/24(rw,sync,no_subtree_check)

sudo exportfs -ra
sudo exportfs -v
sudo systemctl enable nfs-kernel-server
sudo systemctl start nfs-kernel-server

Client Configuration

sudo apt install nfs-common -y
sudo mkdir -p /mnt/nfs_share
sudo mount -t nfs 192.168.1.1:/srv/nfs_share /mnt/nfs_share
sudo nano /etc/fstab

Add this line to /etc/fstab:

192.168.1.1:/srv/nfs_share /mnt/nfs_share nfs defaults 0 0

sudo mount -a

Installing John the Ripper (Jumbo Edition)

Follow the official installation documentation:

To enable OpenMPI support, include the --enable-mpi flag when running ./configure. Additional information can be found here:

Running JtR with OpenMPI

Verify Directory Structure

All nodes must have access to JtR in the same directory path. If paths differ, place the JtR directory on an NFS-mounted location such as /mnt/nfs_share and run all commands from there.

Example MPI Command

mpirun -hostfile hosts.txt ./john /mnt/nfs_share/hashes.txt \
--format=raw-md5 --pot=/mnt/nfs_share/shared.pot \
--output-filename /mnt/nfs_share/status