Aims

The aim of this lab is to introduce more basic Linux commands and tools

Essential Linux Directories

Most linux environments have a similar directory structure. The following is a list of the most important directories and their purpose.

Many of these folders are read + execute only for normal users and for the most part we don’t need to touch them.

For our specific lab build most of the DCC tools we use (Maya,Houdini, Nuke,Renderman) are in the /opt folder.

NCCA Specific folders

We have a number of NCCA specific folders on the system, some of these are network shares / mounts and some are local to the machine. The following is a list of the most important folders.

/transfer

The /transfer folder is a mounted 1Tb local hard drive. This is shared with the windows partition and should be use for local work.

This is specific to the machine and is not shared with other machines directly (more later).

This should be used as a destination for local renders and caches. It should be considered “volatile” so any data you want to keep should be copied to a more permanent location (home directory)/

/public

The /public folder is a network share when various things for teaching and learning are stored, in particular I use the /public/devel/23-24 folder to store programming and development resources. I will refer to this folder a lot in the classes.

/public/bin/2023 is the folder where all the goScripts are stored. These scripts set up the environments for the various DCC tools to run. We will look at these in more detail later.

/net

The /net folder is connected to the network filesystem NFS via autofs and will attempt to automount any network shares of the /transfer drives of any Linux machines. It only mounts a given share when that share is being accessed and these are unmounted after a defined period of inactivity. Automounting NFS shares in this way conserves bandwidth and offers better performance.

To make this work we can use cd /net/[machine-name] this will then mount the [machine-name]/transfer folder for us to access as shown below.

Environment Variables

Environment variables are a set of dynamic named values that can affect the way running processes will behave on a computer.

All operating systems have ways of setting environment variables and running processes can read these variables to determine how they should behave. For example the TEMP environment variable specifies the location in which programs place temporary files.

When you login to the Linux labs a number of environment variables are created for you, for example the location of Renderman is set using the variable RMANTREE.

In bash we refer to environment variables using the $ symbol, so to see the value of RMANTREE we would type echo $RMANTREE

To show all of the environment variables currently set we can use the env command.

As you can see there are a lot of variables here, we can look at individual ones using the printenv command.

To show how environment variables are used the following awk script reads the ENVIRON array and prints out the name and value of each variable.

awk 'BEGIN{for(v in ENVIRON) print v}'

I feed this into the more command via a pipe to print them out.

To set an environment variable we use the export command. For example to set the variable PROJ_DIR to the value ~/Proj we would type export PROJ_DIR=~/Proj

When you set an environment variable in a session it is only valid for this session. If you want to make it more permanent you need to add it to your .profile file.

common environment variables

$PATH

This environment variable is used to specify a set of directories where executable programs are located. When you type a command into the shell it will search the directories in the $PATH variable for the command. If it finds it then it will execute it, if not it will return an error.

We can add custom locations to this PATH within our own folders. It is typically best to add these to the .profile file in your home directory as this will be set once when you login.

There are two approaches to adding to an existing environment variable, you can either append to the existing variable or prepend to it. Prepending is typically the best approach as it will ensure that your custom locations are searched first.

The prepend approach is shown below. We add our custom path to the start of the existing PATH variable.

export PATH=/my/custom/path:$PATH

The append approach is shown below. We add our custom path to the end of the existing PATH variable.

export PATH=$PATH:/my/custom/path

We are now going to add a local scripts folder to our path, to do this we add

export PATH:~/scripts:$PATH

to our .profile file. As this is only read when we login, we can either logout and back in again or we can type source ~/.profile to read the file and update the environment variables.

Test Scripts

First we will make a folder to put our scripts in.

mkdir ~/scripts
cd ~/scripts
touch hello.sh

Using an editor we can all the following to the file

#!/bin/bash
echo "Hello World"

By default the touch command will generate a file with the following file permissions

-rw-r--r-- 1 jmacey ncca 0 Sep  9 10:44 hello.sh

This means that the file is read/write for the owner (jmacey) and read only for everyone else. To make the file executable we need to use the chmod command.

chmod +x hello.sh

This will change the permissions to

-rwxr-xr-x 1 jmacey ncca 0 Sep  9 10:44 hello.sh

making it executable. We can now run the script by typing ./hello.sh

Unix file permissions

Unix file permissions are a way to control who can read, write and execute a file. There are three types of permissions for each file, these are read, write and execute. These permissions are set for three groups of users, the owner, the group and everyone else. The permissions are shown as a series of 10 characters, the first character is the file type, the next three are the permissions for the owner, the next three are the permissions for the group and the final three are the permissions for everyone else.

````chmod```

chmod has two main modes of usage, either the octal mode as shown in the table below or the symbolic mode.

octal mode takes an octal triple for the user, group and world permissions, for example

chmod 755 hello.sh

will set the permissions to rwxr-xr-x which is typical for a script used by members of a group where the owner is the only one who can write to the file.

The symbolic mode uses a series of letters to set the permissions, for example

chmod u+x hello.sh

will add the execute permission to the owner of the file. The following table shows the various letters and their meanings.

#!

The hashbang or shebang is the first two characters of a script file. It is used to tell the shell what program to use to run the script. For example if we want to run a python script we would use

#!/usr/bin/env python

In this case we are using the env command to find the python executable. This is useful as it will find the first python executable in the PATH variable. This means that we can use different versions of python by changing the PATH variable (we will look at this later when we install pyenv in the next lab).

Homework

create a simple python script in your scripts folder that prints out hello world. You can use the #!/usr/bin/env python to run the program.

References