less_retarded_wiki

Unix

"Those who don't know Unix are doomed to reinvent it, poorly." --obligatory quote by Henry Spencer

Unix (coming from "UNICS" referring to "MULTICS", plurar Unixes or Unices) is an old operating system developed since 1960s as a research project of Bell Labs, which has become one of the most influential pieces of software in history and whose principles (e.g. the Unix philosophy, everything is a file, ...) live on in many so called Unix-like operating systems such as GNU/Linux and BSD (at least to some degree). The original system itself is no longer in wide use (it was later followed by plan9, a project which by now is itself also pretty old), the name UNIX is currently a trademark and a certification. Nonetheless Unix is not significant for being a fossil operating system in a museum but rather as a concept, for as someone once said: Unix is not so much an operating system as a way of thinking.

In one aspect Unix has achieved the highest status a piece of software can strive for: it has transcended its implementation and became a de facto standard. That is to say it became a set of interface conventions, programming principles, "paradigms", cultural and philosophical ideas rather than being a single system, a blob of 1s and 0s, it lives on as a concept that's being reimplemented, imitated and adopted to various degrees. This is remarkably important and puts Unix among the most significant terms in technological world, as we now don't depend on any single Unix implementation but instead have a choice of great variety of Unix systems which we can switch between without too much trouble, just like for example the C language (which was developed as part of Unix) is nowadays an abstract language enjoying many different implementations. This is invaluable as prerequisite for true technological freedom, as freedom of choice prevents monopolization, and as a consequence stands as yet another argument for using Unix systems more.

The main highlights of Unix are possibly these:

• Unix philosophy: a kind of general mindset of software development, based on minimalism, frequently summed up as "do one things well" (rather than "do everything but poorly") and "make programs work in collaboration with other programs", advising on using universal text interfaces for communication etc. In practice this manifests as so called pipelines, a way of chaining programs (typically using the pipe | operator, hence the name) by sending one program's output to other program's input. This philosophy is in contrast with so called "Windows philosophy": that of creating big, bloated "monolithic" programs.
• everything is a file: Unix chose to use the file abstraction to enable universal communication of programs with hardware and among themselves, i.e. on unices most things such as printing, reading keyboard, networking etc. will be likely implemented as reading or writing to/from some special (sometimes just virtual) file. This has the advantage of being able to just use some file reading library or syscall, not having to access physical memory bits in memory, which may be difficult, unsafe etc.
• Text centrism, preference of working with the command line (avoiding GUI), preference of text formats, value on portability (even before performance), sharing of source code, freedom of information and openness, connection to hacker culture, valuing human time over machine time, ...
• ...

Unix is significantly connected to software minimalism, however most unices are still not minimalist to absolute extreme and many unix forks (e.g. GNU/Linux) just abandon minimalism as a top priority. So the question stands: is Unix LRS or is it too bloated? The answer to this will be similar to our stance towards the C language (which itself was developed alongside Unix); from our point of view Unix -- i.e. its concepts and some of their existing implementations -- is relatively good, there is a lot of wisdom to take away (e.g. "do one thing well", modularity, "use text interfaces", ...), however these are intermixed with things which under more strict minimalism we may want to abandon (e.g. multiple users, file permissions and ownership, also "everything is a file" requires we buy into the file abstraction and will often also imply existence of a file system etc., which may be unnecessary, even multitasking could be dropped), so in some ways we see Unix as a temporary "least evil" tool on our way to truly good, extremely minimalist technology. DuskOS is an example of operating system more close to the final idea of LRS. But for now Unix is very cool, some Unix-like systems are definitely a good choice nowadays.

There is a semi humorous group called the UNIX HATERS that has a mailing list and a whole book that criticizes Unix, arguing that the systems that came before it were much better -- though it's mostly just joking, they give some good points sometimes. It's like they are the biggest boomers for whom the Unix is what Windows is to the Unix people.

History

In the 1960s, Bell Labs along with other groups were developing Multics, a kind of operating system -- however the project failed and was abandoned for its complexity and expensive development. In 1969 two Multics developers, Ken Thompson and Dennis Ritchie, then started to create their own system, this time with a different approach; that of simplicity (see Unix philosophy). They weren't alone in developing the system, a number of other hackers helped program certain parts such as a file system, shell and simple utility programs. At VCF East 2019 Thompson said that they developed Unix as a working system in three weeks. At this point Unix was written in assembly.

In the early 1970s the system got funding as well as its name Unix (a pun on Multix). By now Thompson and Richie were developing a new language for Unix which would eventually become the C language. In version 4 (1973) Unix was rewritten in C.

Unix then started to be sold commercially, consequence of which was its fragmentation into different versions such as the BSD or Solaris. In 1983 a version called System V was released which would become one of the most successful. This fragmentation along with the lack of a unified standard led to so called Unix Wars in the late 1980s, which in turn spawned a few Unix standards such as POSIX and Single Unix Specification.

For zoomers and other noobs: Unix wasn't like Windows, it was more like DOS, things were done in text interface only (even a TUI or just colorful text was a luxury) -- if you use the command line in "Linux" nowadays, you'll get an idea of what it was like, except it was all even more primitive. Things we take for granted such as a mouse, copy-pastes, interactive text editors, having multiple user accounts or running multiple programs at once were either non-existent or advanced features in the early days. There weren't even personal computers back then, people accessed share computers over terminals. Anything these guys did you have to see as done with stone tools -- they didn't have GPUs, gigaherts CPUs, gigabytes of RAM, scripting languages like Python or JavaScript, Google, stack overflow, wifi, mice, IDEs, multiple HD screens all around, none of that -- and yet they programmed faster, less buggy software that was much more efficient. If this doesn't make you think, then probably nothing will.

How To For Noobs

UNDER CONSTRUCTION

Note: here by "Unix" we will more or less assume a system conforming to some version of the POSIX standard. To view POSIX standard yourself, refer to the web (e.g. https://pubs.opengroup.org/onlinepubs/9699919799.2018edition/) or install the POSIX manual pages on your system (e.g. apt-get install manpages-posix)

This should help complete noobs kickstart their journey with a Unix-like system such as GNU/Linux or BSD. Please be aware that each system has its additional specifics, for example package managers, init systems, GUI and so on -- these you must learn about elsewhere as here we may only cover the core parts those systems inherited from the original Unix. Having learned this though you should be able to somewhat fly any Unix like system. Obviously we'll be making some simplifications here too, don't be too pedantic if you're a pro Unix guru please.

Also a NOTE: terms such as command line, terminal or shell have different meanings, but for simplicity we'll be treating them more or less as synonyms here.

Learning to use Unix in practical terms firstly means learning the command line and then a few extra things (various concepts, philosophies, conventions, file system structure etc.). Your system will have a way for you to enter the command line that allows you to interact with it only through textual commands (i.e. without GUI). Sometimes the system boots up to command line, other time you must click an icon somewhere (called terminal, term, shell, command line etc.), sometimes you can switch TTYs with CTRL+ALT+Fkeys etc. To command line virgins this will seem a little intimidating but it's absolutely necessary to know at least the basics, on Unices the command line is extremely powerful, efficient and much can only ever be achieved through the command line.

The gist: unsurprisingly in command line you write commands -- many of these are actually tiny programs called Unix utilities (or just "utils"). These are installed by default and they're tools for you to do whatever you want (including stuff that on normie systems you usually do by clicking with a mouse). For example ls is a program that writes out list of files in the working directory, cd is a program that changes working directory etc. There are many more such programs and you must learn at least the most commonly used ones. Many of them are oriented on processing text (because text is a universal interface), and that mostly by lines. Good news is that the programs are more or less the same on every Unix system so you just learn this once. There also exist other kinds of commands -- those defined by the shell language (shell is basically a fancy word for the textual interface), which allow us to combine the utilities together and even program the shell (we call this scripting). First learn the utils (see the list below).

PRO TIP: convenient features are often implemented, most useful ones include going through the history of previously typed commands with UP/DOWN keys and completing commands with the TAB key, which you'll find yourself using very frequently. Try it. It's enough to type just first few letters and then press tab, the command will be completed (at least as much as can be guessed).

You run a utility simply by typing its name, for example writing ls will show you a list of files in your current directory. Very important is the man command that shows you a manual page for another command, e.g. typing man ls should display a page explaining the ls utility in detail. Short help for a utility can also usually be obtained by writing -h after it, for example grep -h.

Unix utilities (and other programs) can also be invoked with arguments that specify more detail about what should be done. Arguments go after the utility name and are separated by spaces (if the argument itself should contain a space, it must be enclosed between double quotes, e.g.: "abc def" is a single arguments containing space, but abc def are two arguments). For example the cd (change directory) utility must be given the name of a directory to go to, e.g. cd mydirectory.

Some arguments start with one or two minus characters (-), for example -h or --help. These are usually called flags and serve either to turn something on/off or to name other parameters. For example many utilities accept a -s flag which means "silent" and tells the utility to shut up and not write anything out. A flag oftentimes has a short and long form (the long one starting with two minus characters), so -s and --silent are the same thing. The other type of flag says what kind of argument the following argument is going to be -- for example a common one is --output (or -o) with which we specify the name of the output file, so for instance running a C compiler may look like c99 mysourcecode.c --output myprogram (we tell the compiler to name the final program "myprogram"). Short flags can usually be combined like so: instead of -a -b -c we can write just -abc. Flags accepted by utilities along with their meaning are documented in the manual pages (see above).

To run a program that's present in the current directory as a file you can't just write its name (like you could e.g. in DOS), it MUST be prefixed it with ./ (shorthand for current directory), otherwise the shell thinks you're trying to run an INSTALLED program, i.e. it will be looking for the program in a directory where programs are installed. For example having a program named "myprogram" in current directory it will be run with ./myprogram. Also note that to be able to run a file as a program it must have the executable mode set, which is done with chmod +x myprogram (you may have to do this if you e.g. download the program from the Internet). Programs can also take arguments just like we saw with the built-in utilities, so you can run a program like ./myprogram abc def --myflag.

Now to the very basic stuff: browsing directories, moving and deleting files etc. This is done with the following utils: ls (prints files in current directory), pwd (prints path to current directory), cd (travels to given directory, cd .. travels back), cat (outputs content of given file), mkdir (creates directory), rm (removes given file; to remove a directory -rf flag must be present), cp (copies file), mv (moves file, including directory -- note that moving also serves for renaming). As an exercise try these out (careful with rm -rf) and read manual pages of the commands (you'll find that ls can also tell you for example the file sizes and so on).

Files and file system: On Unices the whole filesystem hierarchy starts with a directory called just / (the root directory), i.e. every absolute (full) path will always start with slash (don't confuse / with \). For example pictures belonging to the user john may live under /home/john/pictures. It's also possible to use relative paths, i.e. ones that are considered to start in the current (working) directory. A dot (.) stands for current directory and two dots (..) for the directory "above" the current one. I.e. if our current directory is /home/john, we can list the pictures with ls pictures as well as ls /home/john/pictures or ls ./pictures. Absolute and relative paths are distinguished by the fact the absolute one always starts with / while relative don't. There are several types of files, most importantly regular files (the "normal" files) and directories (there are more such symbolic links, sockets, block special files etc., but for now we'll be ignoring these). Unix has a paradigm stating that everything's a file, so notably accessing e.g. hardware devices is done by accessing special device files (placed in /dev). Just remember this concept, you'll hear about it often.

NOTE: On Unices files often don't have extensions as it's often relied on so called magic number (first few bytes in the file) to decide what kind of file we're dealing with. You will see files with extension (.sh, .txt, ...) but notably for example compiled programs typically don't have any (unlike for example on Windows).

Files additionally have attributes, importantly so called permissions -- unfortunately these are a bit complicated, but as a mere user working with your own files you won't have to deal too much with them, only remember if you encounter issues with accessing files, it's likely due to this. In short: each file has an owner and then also a set of permissions that say who's allowed to do what with the file. There are three kind of permissions: read (r), write (w) and execute (x), and ALL THREE are defined for the file's owner, for the file's group and for everyone else, plus there is a magical value suid/sgid/sticky we won't delve into. All of this is then usually written either as a 4 digit octal number (each digit expresses the three permission bits) or as a 12 character string (containing the r/w/x/- characters). Well, let's not dig much deeper now.

PRO TIP: there is a very useful feature called wildcard characters helping us process many files at once. Most commonly used are * and ? wildcards -- if we use these in a program argument, the arguments will be expanded so that we get a list of files matching certain pattern. This sounds complicated but let's see an example. If we write let's say rm *.jpg, we are going to remove all files in current directory whose name ends with .jpg. This is because * is a wildcard character that matches any string and when we execute the command, the shell actually replaces our argument with all files that match our pattern, so the command may actually internally look like rm picture1.jpg picture2.jpg picture3.jpg. ? character is similar but matches exactly one character (whatever it is), so to list for example all files whose name is exactly three characters we can write ls ???.

Here is a quick cheatsheet of the most common Unix utilities:

NOTES on the above table:

• Typically there are two ways of feeding input data to a utility: either by specifying a file to read from or by feeding the input on to the utility's standard input. This also applies to the output. Using standard input/output is a more "Unix" way as it allows us to chain the utlities with pipes, make one program feed its output to another as input.
• Utilities try to follow common conventions so that it's easier to guess and remember what flags mean etc., for example -h is commonly a flag for getting help, -o is one for specifying output file etc.
• Specific Unix systems will normally have more feature rich utilities, supporting additional flags and even adding new utilities. Check out manual pages on your system. You'll probably have to learn about common utils that aren't part of POSIX, e.g. wget, history, ssh, curl, sudo, apt and more. And of course there are thousands and thousands of additional utilities/programs you can download, program or otherwise install on your system.

Now on to a key feature of Unix: pipelines and redirects. Processes (running programs) on Unix have so called standard input (stdin) and standard output (stdout) -- these are streams of data (often textual but also binary) that the process takes on input and output respectively. There may also exist more streams (notably e.g. standard error output) but again, we'll ignore this now. When you run a program (utility etc.) in the command line, standard input will normally come from your keyboard and standard output will be connected to the terminal (i.e. you'll see it being written out in the command line). However sometimes you may want the program to take input from a file and/or to write its output to a file (imagine e.g. keeping logs), or you may even want one program to feed its output as an input to another program! This is very powerful as you may combine the many small utilities into more powerful units. See also Unix philosophy.

Most commonly used redirections are done like this:

• command > file: redirects output of command to file file (rewriting its content if there is any).
• command < file: redirects input of command to come from file.
• command >> file: output of command will be appended to file (i.e. added at its end).

Pipelines are similar: they are chains of several programs separated by a "pipe" character: |. This makes a program feed its output to the input of the next program. For example ls | grep \.html will run the ls command and pass its output (list of files in current directory) to grep, which will only filter out those that contain the ".html" string.

Several commands can also be written on a single line, they just have to be separated with ;.

Example of doing stuff in a Unix terminal (# character starts a comment -- these are here only to describe what's happening in the example):

> pwd
/home/drummyfish
> ls
Pictures    Documents    Downloads
git         Videos
> cd Downloads
> ls
free_software_song.midi  hentai_porn.mp4
lrs_wiki.txt
> rm hentai_porn.mp4     # oh noes, quickly delete this
> cd ../git; ls
Anarch      comun       Licar
raycastlib  small3dlib
> cd Anarch
> wc -l *.h *.c | tail -n 1    # count lines of code in .h and .c files
14711 total
> cat *.h *.c | grep "TODO"    # show all TODOs in code
      (SFG_game.backgroundScaleMap[(pixel->position.y          // ^ TODO: get rid of mod?
  RCL_Unit     direction;  // TODO: rename to "angle" to keep consistency
  TODO: maybe array functions should be replaced by defines of funtion names
  /* FIXME/TODO: The adjusted (=orthogonal, camera-space) distance could
  RCL_Unit limit1, // TODO: int16_t?
  RCL_Unit depth = 0; /* TODO: this is for clamping depth to 0 so that we don't
         increment == -1 ? i >= limit : i <= limit; /* TODO: is efficient? */\
  RCL_Unit limit1, // TODO: int16_t?
       increment == -1 ? i >= limit : i <= limit; // TODO: is efficient?
  // TODO: probably doesn't work
#if 1 // TODO: add other options for input handling (SDL, xinput, ...)
> echo "Remember to fix the TODOs in code!" >> TODO.txt  # add note to TODO file

Shitty Things About Unix

Unix is awesome and lovely but could be better. It was a "first attempt" at this kind of operating system and as such couldn't have hit everything right, even though it mostly aimed surprisingly well and turned out to have made correct decisions where it mattered the most. Sadly the plan9 project, meant to be a reiteration on Unix, failed miserably and its design wasn't any good. Here is some criticism of Unix through the lens of LRS (to do it better next time):

• The shell language is kind of shit, it was evolving on-the-go and is pretty ugly. Ideally the shell language could even be the same or at least very similar to the compiled language the system is written in (C shell made some effort in this direction).
• Some stuff is possibly overcomplicated, for example file permissions. These wouldn't have to be part of the base system, they could be left to handle by specialized programs. By our strict view we could even drop multi-user support as we wouldn't require "security" and stuff (multiple users could still share one computer, but it could be handled simply by creating different directories for each). Speaking of directories, they could perhaps be removed as a "concept", there could only be regular files distinguished by prefixes that would be part of their name (in the end it might even look the same from user's perspective, but a "no directory" paradigm could maybe lead to simpler implementations etc., in the end there's a hint on this paradigm e.g. in that the mv command is described as "rename"). Then we could also do away with the distinction between normal users and the superuser, remove user groups and shit. In a radical case we might even go as far as dropping multitasking, processes and all this -- see our contemplation on the ultimate LRS operating system.
• The whole system design could lean towards even more minimalism and elegance as demonstrated by Forth, for example in that the programming language and the system wouldn't even be two separate things. The full-fledged operating system could really be a natural bootstrap of a simple programming language, its gradual expansion by adding simple commands, while the collaboration of these simple commands could remain respecting the Unix philosophy.
• ...

See Also

• Unix philosophy
• Plan9
• Linux
• GNU
• BSD
• Window$

Powered by nothing. All content available under CC0 1.0 (public domain). Send comments and corrections to drummyfish at disroot dot org.