In this class, you will write software that you will be unable to run on your laptop or PC. This is why we provide you with shell access to a large “number crunching” computer.
I suggest that you develop code on your local PC, test it with a little bit of data to make sure it runs. Then you commit your changes to a GitHub repository. Then you ssh into the server, check out your code from GitHub, compile it, and run it.
Now, here is the problem: As soon as you loose Internet access to the number cruncher, your process dies. Here is the solution:
screen(alternatively you can use
nohub). You are now “within a screen session”.
If you want to check in on your program’s progress, you can re-attach your screen session as follows
screen -rd(with -d, any shell who was already attach gets automatically detached)
Screen and tmux also have a couple of other features, such as different tabs. I recommend learning more about this tool, because it is really useful.
Here a list of other bash tools that I regularly use
htop: for seeing which processes are running and how much memory or CPU they are using
scp) copy files over ssh
ls -lhdirectory listing with dates, owners, and softlink targets
treedirectory listing of full subtree
lessread a text file page by page
find . -iname "*.java"lists all files in this subdirectory that match the file name pattern *.java
killall $processnamekills all processes with this name, e.g. all java programs. Check process name with
historylists the last commands you entered
tar -czvf archive.tar.gz $file1 $file2 $dir3...create a tar.gz archive containing files or directories. (Alternative
tar -cJvfcreates a tar.xz archive which takes longer to compress, but yields smaller file
mavenfor dependency management of java programs
pyenvfor installing python libraries locally
nixto manage pipelines of intermediate steps and data in a reproducible way.
ln -screate a softlink to a file — useful if you have different experimentation directories, which all needs the same (large) file.
cut -f1 $file(or
awk) prints the first column of a tab separated file.
sed -e "s/$search/$replace/g $file > $outfilesearch and replace in file, then write to $outfile
sort -u $filesort lines in a file and remove duplicates
grep -e "$pattern" $filefind patterns in file.
To connect to the server, you can use
ssh. However, if your Internet connection is unstable or you want to suspend your laptop, use
I also recommend learning how to write bash scripts to avoid having to memorize long command lines.
As you can only connect to c01 through agate, it is painful to transfer files. Here is a trick: Create an SSH Tunnel! Keep this one running in the background:
ssh -L $fancyPort:c01.cs.unh.edu:22 agate.cs.unh.edu
Now you can scp/rsync to agate.cs.unh.edu:$fancyPort and files will automatically land on c01. (in fact, you are rsyncing to c01, not agate. You can also use this connection for ssh connections. Remember: it is just a tunnel!