Howto use Linux shell tools

In this class, you will write software that you will be unable to run on your laptop or PC. This is why we provide you with shell access to a large “number crunching” computer.

I suggest that you develop code on your local PC, test it with a little bit of data to make sure it runs. Then you commit your changes to a GitHub repository. Then you ssh into the server, check out your code from GitHub, compile it, and run it.

Nohub / screen / tmux

Now, here is the problem: As soon as you loose Internet access to the number cruncher, your process dies. Here is the solution:

  1. You ssh into the server
  2. You start a screen session by typing screen (alternatively you can use tmux or nohub). You are now “within a screen session”.
  3. You start running your code. (I highly recommend to redirect stdout and stderr into a file.)
  4. Now you hit “Ctrl-a d” to detach the screen session. This means, you have left the screen session, but whatever you were starting within, continues to run.
  5. You can drop your internet connection – stuff in the screen session keeps on running

If you want to check in on your program’s progress, you can re-attach your screen session as follows

  1. You ssh into the server
  2. You reattach the session with screen -rd (with -d, any shell who was already attach gets automatically detached)

Screen and tmux also have a couple of other features, such as different tabs. I recommend learning more about this tool, because it is really useful.

Bash Tools

Here a list of other bash tools that I regularly use

Basic Shell

Software development

File magic

SSH / Mosh

To connect to the server, you can use ssh. However, if your Internet connection is unstable or you want to suspend your laptop, use mosh instead.

I also recommend learning how to write bash scripts to avoid having to memorize long command lines.

SSH Port tunnel

As you can only connect to c01 through agate, it is painful to transfer files. Here is a trick: Create an SSH Tunnel! Keep this one running in the background:

ssh -L $

Now you can scp/rsync to$fancyPort and files will automatically land on c01. (in fact, you are rsyncing to c01, not agate. You can also use this connection for ssh connections. Remember: it is just a tunnel!