Introduction to Unix Labs (Summer 2019)

Contact

Dates

Description

Rules

  1. Broaden your knowledge about Before Class topic before coming to the class. Most of the activity points are awarded in the discussion related to this topic.
  2. Be curious. If you are not familiar with some command, tool, technology or principle try to learn and understand it on your own. There are tons of tutorials on the Internet and most of the information is in man pages. After that you can validate your knowledge in discussion with me. This is the most efficient way of learning.
  3. To obtain a credit you need to:
    1. Obtain at least 10 points for in class activity. Best strategy is to aim for 1-2 points per class.
    2. Obtain at least 75% points from all scripting tasks.
    3. Propose and implement a useful script which solves some real problem. Optimally write a script for your own usage.

Notes

Follow-up

Homework Submission (Scripting Tasks)

Agenda

Date Schedule
20.2.
22.2.

Before Class

  • Your motivation for UNIX. Who uses UNIX? What are advantages and disadvantages of UNIX? What devices are suitable/not suitable for UNIX? How old is UNIX? What other operating systems do you know? What is github and what is good for?

On Class

  • Labs organization. Motivation. Motivation. Motivation. First touches with UNIX.
  • Login to KDE session. Running firefox and konsole via Alt-F2.
  • Creating a github account.
  • Running file manager mc in konsole. Browsing the filesystem. Check wikipedia for Filesystem Hierarchy.
  • Running text editor nano in konsole. Editing file. Alternative editor can be mcedit which is part of mc.
  • There exist putty program for Windows. You can use it to connect to the unix lab computers. Follow this guide.

Scripting Tasks

  • [2 points] Create github repository named intro19 and send me email with it’s link for cloning. Make the repository private and add github user asch to the collaborators. You can set up email notifications to be notified when I submit points for the task. This task is done from web browser. No command line commands are needed.
  • [01/test2.txt] [2 points] Create folder 01 and in that folder create file test2.txt which contains text “I Love GIT.”. This can be also done from web browser. No command line commands needed.
27.2.
1.3.

Before Class

  • Last time we used KDE Desktop Environment with konsole terminal emulator. What other Desktop Environments you know and what terminal emulators they have? Is it possible to use konsole in different Desktop Environment than KDE? Why/Why not?
  • What other web browsers are installed on UNIX lab computers?
  • How to set shortcuts in KDE? Set shortcut such that Alt+k executes konsole and Alt+f executes firefox. (You can use different shortcuts.)
  • How to copy/paste in Linux?

On Class

  • Trying different Desktop Environments (xfce, mate, gnome, …).
  • Environment configuration (shell control/completion/shortcuts, PRIMARY and CLIPBOARD selections). Bash Keyboard Shortcuts.
  • Basic shell commands (man, w, who, whoami, id, echo, apropos). Shell commands for working with file system hierarchy (ls, cd, cat, tac, mkdir, rmdir, rm, cp, mv, touch, pwd, tree, chmod, umask). Shell commands making output readable (head, tail, more, less, grep). Counting lines, words and bytes (wc).
  • Chaining commands together with pipes (|). Redirections in all variants (>, >>, <, 2>, &>, tee).
  • Useful special files (., .., .abc, /dev/null). Symbol for home directory (~).
  • Globs and wildcards (*, ?, [a-z], [^a-z]). Nice article about Globs.

Scripting Tasks

  • [02/setup.txt] [1 point] Learn more about environment configuration and configure your system for your best needs. I.e. tweak your UNIX computer the way you like it. Write down a text file (80 characters per line, couple of paragraphs) what your setup is (desktop environment, text editor, file manager, internet browser and other favourite applications, tips and tricks). Write it in nano or another text editor in terminal. Use web browser only for upload.
  • All of the following tasks have to use only commands we used on class.
  • [02/print.sh] [1 point] Write a command for listing files in the current directory sorted by date of modification.
  • [02/last-mod.sh] [1 point] Write a command for listing 22 most recently modified files.
  • [02/smallest.sh] [1 point] Write a command for listing 20 smallest files from /bin which contain letter a.
  • [02/two-chars.sh] [1 point] Write a command for listing all hidden files in your home directory that contains at least 3 letters in their name. This has to work from any location.
  • [02/last-line.sh] [1 point] Write a command that prints last 3 lines from every file ending with .h in /usr/include.
  • [02/incl.sh] [1 point] Write a command which copies all files which do not have any number in their name from /usr/include to ~/my_includes.
6.3.
8.3.

Before Class

  • Last time you did much more than expected. Awesome job guys!
  • What is POSIX? What is bash/zsh/ksh/dash?
  • Is it possible to chain commands together? Is it possible to write your own program for chaining programs together?
  • What is standard input/output? What is standard error output? Write a program in your favorite language which reads some text from standard input (stdin) prints “Hello world!” to standard output (stdout) and “Hello error!” to standard error output (stderr).
  • What does '>>' do?
  • With pipes we can redirect standard output to standard input of the subsequent program. Is it possible to redirect standard error output with a pipe (|)?
  • What is a wildcard? What is a shebang? What is the difference between compiled language and scripting language?
  • What will 'ls ?' output? What about 'ls ??'. And 'ls ???', 'ls ????' and 'ls ?????'? What about 'ls [a-z]' or 'ls [a-z][a-z][a-z][a-z]'? Why?
  • What will 'ls *' do in empty directory? And in directory only with hidden files? Why?
  • Everyone will prepare 5 or more scripting tasks using all the stuff we learned last time. E.g. print first 10 lines containing character ‘a’ from /etc/passwd and save the output to the file. Be original, tasks have to be unique and complex, i.e. no activity points for duplicate and trivial tasks. You can save it to your git repository.

On Class

  • Comment on github notifications and homework submissions on time. Use github for saving Before Class preparation and also for notes. Test your homeworks on different inputs. Try to invent corner cases.
  • Solving scripting tasks you invented.
  • Different shells and different standards (bash, zsh, ksh, dash, sh). POSIX. Comparison of shells.
  • Tools for management (ssh, scp, screen, tmux, top, htop, df, free, ps, pgrep, pkill, kill, killall).
  • Tools for working with text files (diff, vimdiff, patch, comm, cut, paste, split, tr, sort, uniq, join). Calculator (bc).

Scripting Tasks

  • [03/group-sum.sh] [1 point] /etc/group contains names of system groups. Write a command to print only column with group names. Then write a command, which prints one number - all group IDs summed together.
  • [03/unused-groups.sh] [1 point] /etc/passwd contains users present on system with their groups. Print system groups which are not present in /etc/passwd.
  • [03/patch.sh] [1 point] Take some program you wrote and create a patch which replaces every digit with character ‘X’ in its source code. (Hint: Use tr for digit replacement).
  • [03/copy.sh] [1 point] All of the files above copy to folder /tmp on different computer.
  • [03/mem.sh] [2 points] df displays free disc space. Print total number of available space in Megabytes, i.e. sum together available space from all partitions.
  • [03/cpu.sh] [4 points] /proc/cpuinfo contains information about CPU. There is a row called flags which contains capabilities of the CPU. Create a command which will output name of the CPU on the first line and then all flags of the CPU sorted alphabetically. One flag per line indented with 2 spaces. Do this for 2 different CPUs and create an output with three columns. First column for the first CPU, second for the second CPU and third containing flags common for both CPUs: Hint: New line is \n character. The output doesn’t need to be properly aligned but strive for the best result you can achieve! Example:

    Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz Common
      3dnowprefetch                           acpi                                   [many more flags]
      abm                                     aperfmperf                             xtpr
      [many more flags]                       [many more flags]
      xtpr                                    xtpr
13.3.
15.3.

Before Class

  • What does program join do? Is it useful for 03/unused-groups.sh?
  • Again: Write a program which reads from stdin and writes to stdout and stderr. Use it in shell with redirections <, >, 2>.
  • What does ** (NOT POSIX) glob mean? Try it on ls /**/ etc.
  • What is a shell variable? What $PATH variable means (Hint: When you write cat and execute what is going on?)?
  • What actually cat is? Is it command of the shell? What is the difference between cat, /bin/cat, ./cat?
  • Is writing a shell script the same as writing a program?
  • What is a pull request? What is it good for?
  • Prepare one complex scripting task named $sur.sh where $sur is 3 letter prefix of your surname without diacritics. Create a pull request to the asch/intro19. The file $sur.sh has to contain only assignment, not the solution.

On Class

  • Git is not for communication. Use e-mail.
  • Variables ($var), environment (env, $PATH) and exporting variables (export).
  • Finding what binary is executed by written command (whereis, which).
  • Control-flow in shell scripts (;, if, while, for, &&, ||, test, [], true, false).
  • Shell functions (func() { }).
  • Shell arguments ($1, $2, …).
  • ~/.bashrc, ~/.profile for setting up your shell.
  • Aliases (alias l='ls -l').
  • Writing a shell script (#!/bin/sh), checking what shell we have (ls -l $(which sh), making it safe (set -euxo pipefail) and lint it (shellcheck). Line wrap (\). Line limit 80 chars.
  • See man bash for complete reference.
  • See man dash for much more simple reference without bash specific constructs. (Recommended!)

Scripting Tasks

  • [04/mega.sh] [5 points] Implement an executable script which implements all scripting tasks from second class. The script accepts one argument which is name of the scripting task and performs the operation required by the scripting task. For example mega.sh print.sh will do the same action as scripting task [02/print.sh] and mega.sh smallest.sh will do the same action as [02/smallest.sh from the second class and so on. This has to be just one script which means all code has to be inside this script.
  • [04/$surname.sh] [1 point for every hard task] Pick 5 hardest tasks from my GitHub asch/intro19 repository and solve it. Name it equally with script on the github. I will not give points for easy tasks.
20.3.
22.3.
(Plan)

Before Class

  • Again: Write a program which reads from stdin and writes to stdout and stderr. Use it in shell with redirections <, >, 2>.
  • What is POSIX? What is it good for? What shell is POSIX compliant? What is a “bashism”? Give an example of “bashism”. Is it a good or bad thing?
  • Since now all pull requests with complex scripting tasks will be honored with 1-2 extra points (If I like it.). The limit is 1 pull request in a week.
  • How a shell function “returns” a value? What if I want to save result of some function to variable.
  • What other stuff are you missing for comfortable writing of shell scripts?

On Class

  • Function “return” value (echo). Capturing output of function/command to a variable ($(expr)).
  • Exiting from the shell (exit).
  • Command construction from arguments (eval).
  • Run on background (&).
  • Signals for job control (SIGINT (Ctrl-C), SIGSTOP (Ctrl-Z), SIGCONT (fg/bg)).
  • List jobs in current session (jobs).
  • Reading from stdin (read).
  • Here-doc redirection (<<).
  • Special Parameters ($*, $@, $#, $?, $$, $!, $0).
  • Parameter Expansion (${parameter:-word}, …).
  • Arithmetics $((expr)).
  • Reading arguments comfortably with shift and getopts.
  • Grouping Commands Together ((list), { list; }).
  • Introducing dash: The shell you want to use for shell scripts!
  • Reading man dash as a source of all knowledge needed for scripting.
  • Tools for http interaction (curl, wget, httpie).

Resources