Getting good at Unix

I am in no way a unix master. i read this article on another blog and i feel the need to post it here. So this is for my personal motivation. enjoy :)




I recently got a question from a friend of mine that essentially said “I’d like to be awesome at UNIX, but I’m not sure where to start or what to do”. She’s a pretty nerdy woman who runs Linux on her laptop and has serious geek tendencies already, so I wrote up the following advice.

Here’s my 12 step program to become a nerdy UNIX master (or at least journeyman). It’s intended to be gone through twice. Each step has a “novice” level for the first time through and an “expert” level for the second time through. Also, throughout the process you should be brushing up on sh scripting and an interpreted scripting language of your choice (with the suggestions being perl, ruby and/or python). Use a shell script if you’re calling lots of external processes for information. Use perl/ruby/python if you’re doing any moderately complex control statements.

1) Install a version of UNIX on a computer. This could be any distro of Linux, FreeBSD, OpenSolaris or something else. You probably want to choose something you think will be useful at a future job. Right now, I’d suggest CentOS or Gentoo for server-based Linux distros or FreeBSD. The second time through the 12 steps, install a virtual server using Xen, jails or some similar virtualization technology and make the installation automated (eg. kickstart with a specific profile and, possibly, post-install scripts that set up users, point to network resources, etc).

2) Create users on your system. Configure the creation process so it uses skeleton files to create a usable user environment. The second time through these steps, set up an LDAP server that you can authenticate to. Set up your user directories so they auto-mount from a remote server.

3) Set up your essential host configuration files in RCS/CVS/git so you have versioning of the changes you make to your system.. Before making any major changes to the files, make sure you check them in using CVS/RCS/git. Also, check the files in after you’ve verified the file changes. The second time through, write a ‘cvsedit’ wrapper that does the check in automatically before calling vi on the file.

4) Learn regular expressions. You’ll use them with grep, perl, ruby, sed, awk and a lot of other applications. Knowing them will be your friend. This is something where you’ll keep picking things up as you go along. The second time through these 12 steps take some earlier scripts you wrote with regexps in them and do optimization on them and see how much you can increase search speed and reduce run time.

5) Set up a mail server that will send mail out. Make sure it doesn’t accept outgoing connections and only sends mail from local users/processes. Make sure you test this out by trying to connect from somewhere else and trying to relay through your system to make sure you did it right. The second time through, configure your mail server to accept external mail for delivery to local users. Feel free to set up anti-spam filtering and such as well. Also, feel free to try Postfix or qmail as an alternative mail server.

6) Install and configure a DBMS on the box. Mysql is a good choice. Postgresql is another choice if you want to take the road less traveled. Create some tables, write some queries, do some stuff with it (which you’ll likely really do next step). The second time through, install from source. Set up database replication between multiple hosts. Take archive/redo logs from an active server and use it to bring your second server up to date.

7) Install Apache or other httpd server. Set up some CGIs that provide dynamic results. Have the CGIs read, write and update the database tables. Feel free to install WordPress or something similar and see how they work and modify them to tweak how they work. The second time through the steps, install from source and use something like mod_ruby or mod_perl to improve performance. If you don’t mind doing some java, use an application server like tomcat, jboss or glassfish to mimic the traditional 3-tier web stack

8) Automate performance monitoring on the system. Put together something that makes MRTG graphs of disk usage, network usage, etc. The second time through these steps, set up SNMP and poll results from a remote system. Store the results in your database server. Run analyses to identify aberrations in the system and see if you can make any correlations to determine the cause (maintenance, holidays, software/config changes, etc).

9) Make sure the system has multiple disks in it and set up mirroring (LVM, Solaris Disk Suite, etc). Use this to mirror your root disk. The second time through these steps, migrate to a RAID 1 + 0 setup then to a RAID 5 setup and compare the performance results on reads and writes for small, medium and large files and understand what differences there are and why they’re there.

10) Install DNS and DHCP servers on your system. Use these to manage your LAN. Connect new systems and see if they get the proper IP addresses and network information you think they should. Make sure the clients point to your DNS server when they get the DHCP information. The second time through, have the DHCP clients set up to use your box as the default gateway. Set your system up to proxy and filter appropriate services, and reject the rest.

11) Write a network client that retrieves information from a server somewhere and does something with it. Retrieve an RSS feed from a remote web server and figure out if it’s been updated since the last time you polled it and send you an e-mail with whatever the recent changes are. The second time through, write a network server that you can query with a client to get some specific information. For bonus points, write a server appropriate to being called from inetd and another that does threading/fork-exec.

12) Secure your system from all the holes you’ve put into it in the past eleven steps. Lock it down so only people from certain networks or systems can access your system. Set up ipchains in such a way that you can do what you want to do, but other people are prevented. Set up tripwire to detect changes on the system. The second time through these steps, try to break into your system. Or have someone else you know try to break into the system while you’re watching them.

While you’re doing these steps, see if there’s any part you enjoy a lot more than the others. If you like the database stuff, maybe you want to be a DBA. Perl/Ruby and/or Java, maybe you want to be a programmer. DNS/DHCP/proxy gateway stuff, maybe you want to be a network engineer. As listed here, this will help you become a good UNIX generalist and give you a good grounding in what you need to move into whatever specialization you’d like.

So that’s your 12/24 step program for becoming a UNIX master.