GNU/Linux Command-Line Tools Summary Gareth Anderson Chris Karakas - Conversion from LyX to DocBook SGML, Index generation Revision History Revision 1.1 28th February 2006 Revised by: CK Corrected typos, generated new index (9000 index entries!). Revision 1.0 6th February 2006 Revised by: GA Major restructuring, now in a docbook book format. Removed large chunks of content and revised other parts (removed chapters and sectioned some areas more). This is likely the final release by the author, I hope that someone finds this guide useful as I do not intend to continue work on this guide. Revision 0.7.1 25th February 2005 Revised by: CK Set special characters in math mode, produced PDF and PS with Computer Modern fonts in OT1 encoding and created correct SGML for key combinations. Revision 0.7 5th December 2004 Revised by: GA Updated document with new grammatical review. Re-ordered the entire Text section. Removed a fair amount of content. Revision v0.6 20th April 2004 Revised by: GA Attempted to fix document according to TLDP criticisms. Added notes and tips more sectioning. Now complying to the open group standards for the UNIX system trademark. Document should be ready for TLDP site. Revision v0.5 6th October 2003 Revised by: GA Fixed a variety of errors as according to the review and made some consistency improvements to the document. Revision v0.4 15th July 2003 Revised by: GA Made small improvements to the document as suggested (so far) by the thorough TLDP review, improved consistency of document and made small content additions. Revision v0.3 26th June 2003 Revised by: GA Minor errors fixed, updated the appendix with information for finding where a tool is from. Fixed referencing/citation problems and improved further reading and intro sections, added an audio section. Revision v0.2 20th April 2003 Revised by: GA This is the initial public release. Added more code-style then before, broke text-section into more subsections. Improved consistency of document and fixed various index entries. Revision v0.1 27th March 2003 Revised by: GA This is the initial draft release (the first release to be converted from LyX to DocBook SGML). This document is an attempt to provide a summary of useful command-line tools available to a GNU/Linux based operating system, the tools listed are designed to benefit the majority of users and have being chosen at the authors discretion. This document is not a comprehensive list of every existent tool available to a GNU/Linux based system, nor does it have in-depth explanations of how things work. It is a summary which can be used to learn about and how to use many of the tools available to a GNU/Linux based operating system. _________________________________________________________________ Table of Contents 1. Introduction 1.1. Formats 1.2. Who would want to read this guide? 1.3. Who would not want to read this guide? 1.4. Availability of sources 1.5. Conventions used in this guide 1.6. Resources used to create this document 1.7. Feedback 1.8. Contributors 2. Legal 2.1. Disclaimer 2.2. License 3. The Unix Tools Philosophy 4. Shell Tips 4.1. General Shell Tips 4.2. The command-line history 4.3. Other Key combinations 4.4. Virtual Terminals and screen 5. Help 6. Directing Input/Output 6.1. Concept Definitions 6.2. Usage 6.3. Command Substitution 6.4. Performing more than one command 7. Working with the file-system 7.1. Moving around the filesystem 7.1.1. Finding files 7.2. Working with files and folders 7.3. Mass Rename/copy/link Tools 8. Finding information about the system 8.1. Date/Time/Calendars 8.2. Finding information about partitions 9. Controlling the system 9.1. Mounting and Unmounting (Floppy/CDROM/Hard-drive Partitions) 9.2. Shutting Down/Rebooting the System 9.3. Controlling Processes 9.4. Controlling services 10. Managing users 10.1. Users/Groups 11. Text Related Tools 11.1. Text Editors 11.2. Text Viewing Tools 11.3. Text Information Tools 11.4. Text manipulation tools 11.5. Text Conversion/Filter Tools 11.5.1. Conversion tools 11.6. Finding Text Within Files 12. Mathematical tools 13. Network Commands 13.1. Network Configuration 13.2. Internet Specific Commands 13.3. Remote Administration Related 14. Security 14.1. Some basic Security Tools 14.2. File Permissions 15. Archiving Files 15.1. tar (tape archiver) 15.2. rsync 15.3. Compression 16. Graphics tools (command line based) 17. Working with MS-DOS files 18. Scheduling Commands to run in the background 19. Miscellaneous 20. Mini-Guides 20.1. RPM: Redhat Package Management System 20.2. Checking the Hard Disk for errors 20.3. Duplicating disks 20.4. Wildcards 20.4.1. Standard Wildcards (globbing patterns) 20.4.2. Regular Expressions 20.4.3. Useful categories of characters (as defined by the P OSIX standard) A. Appendix A.1. Finding Packages/Tools A.1.1. Finding more useful tools A.1.2. Finding a particular tool(s) A.1.3. Finding package(s) A.2. Further Reading A.2.1. General Further Reading A.2.2. Specific Further reading A.2.3. Online Manual And Info Pages A.3. GNU Free Documentation License A.3.1. PREAMBLE A.3.2. APPLICABILITY AND DEFINITIONS A.3.3. VERBATIM COPYING A.3.4. COPYING IN QUANTITY A.3.5. MODIFICATIONS A.3.6. COMBINING DOCUMENTS A.3.7. COLLECTIONS OF DOCUMENTS A.3.8. AGGREGATION WITH INDEPENDENT WORKS A.3.9. TRANSLATION A.3.10. TERMINATION A.3.11. FUTURE REVISIONS OF THIS LICENSE Bibliography Index _________________________________________________________________ Chapter 1. Introduction This document is an attempt to summarise the many command-line-based tools available to a GNU/Linux based operating system. This guide is not a complete listing (I doubt it's possible to document all available programs), this document lists many tools which are available to GNU/Linux systems and which are, or can be useful to the majority of users. Each tool description provides a quick overview of it's function and some useful options for that individual tool. The tools listed that require a GUI, usually the X windowing system, are those listed in the Graphics Tools section. All other tools are completely command-line-based and do not require a GUI to run. If you are looking for information on GUI based tools you will need to look elsewhere. Also note that a few of the tools in this guide are bash (the Bourne-Again-SHell) specific, tools specific to other shells are not listed in this document. For some of the tools that are harder to use, or perform a more complex task, there are several mini-tutorials (or mini-guides; Chapter 20) within this document. Where a mini-guide was considered unncessary, detailed descriptions that explain in detail how a particular tool works, and some examples of how to use it are provided. Please note that the word "tool" is used interchangeably with the word "command", both have the same meaning (at least in this guide). For a more detailed explanation, read about the UNIX Tools Philosophy here: Chapter 3 or visit the links in the appendix, Section A.2.2.1. Tip To find out which tools are bash specific To find out which tools are bash specific you can type: enable -a _________________________________________________________________ 1.1. Formats This document is available in the following formats: * HTML (HyperText Markup Language), many HTML files, one per section, for viewing with any browser * HTML (HyperText Markup Language), one big HTML file * TXT (ASCII Text) * RTF (Rich Text Format) * PDF (Portable Document Format) * PS.GZ (Compressed Postscript) * SGML (Standard Generalized Markup Language)(with the Appendix and the Bibliography) * LYX (LaTeX frontend LyX)(with the Appendix and the Bibliography) Note RTF: Page numbers In order to get correct page numbers (and often to get the images working) in Microsoft Word, type the following after opening the document: 1. CTRL-END 2. CTRL-A 3. F9 In Word Viewer 97, you must instead do: 1. CTRL-END 2. ALT 3. V 4. N 5. ALT 6. V 7. P See The OpenJade RTF backend for more details. Important Important: Downloads for offline reading! If you want to download the HTML or RTF formats for offline reading, you will need to download the images as well - PNG for HTML and BMP for RTF, including the callouts! To save you the hassle, I have compiled the following gzipped tar archives for offline reading: * TAR.GZ (Compressed TAR Archive), many HTML files with images * TAR.GZ (Compressed TAR Archive), one big HTML file with images * TAR.GZ (Compressed TAR Archive), SGML file with images * TAR.GZ (Compressed TAR Archive), RTF file with images A tarball containing all the above is also available: * TAR.GZ (Compressed TAR Archive), All files _________________________________________________________________ 1.2. Who would want to read this guide? Anyone who is interested in learning about the tools (also known as commands) available to them when using their GNU/Linux based operating system. Why would you want to learn how to use the command line (and available tools)? The Command Line Interface (CLI), while difficult to learn, is the quickest and most efficient way to use a computer for many different tasks. The CLI is the normal method of use for most UNIX system administrators, programmers and some power users. While a GUI is better suited to some tasks, many operations are best suited to the CLI. The major motivation behind learning the GNU/Linux CLI is the authors idea that, with software in general, the more time spent learning something equals less time spent performing that particular task (authors opinion only). This guide is aimed at beginners to intermediate users who want to learn about the commandline tools available to them. Advanced users may wish to use it as a command reference, however this document aims to list commands of interest, as judged by the authors opinion, it is not designed to be completely comprehensive, see the appendix, Section A.2.1 for further information. Or if you are not looking for a command reference guide, but a more gentle introduction to GNU/Linux you may be interested in the Introduction to Linux guide authored by Machtelt Garrels. This guide could also be considered a summarised version of the Linux Cookbook. If you are looking for a book with more detailed descriptions of each tool have a look at the Linux Cookbook Homepage, also check out the command list from "Linux in a Nutshell 3rd Edition" for an index of 300+ commands and their explanations. _________________________________________________________________ 1.3. Who would not want to read this guide? Anyone who is not interested in the command line, or anyone looking for a detailed reference to all available GNU/Linux tools should look elsewhere. This is only a summary, while it does list many commands, it's not a complete listing (I don't think it's possible to make a complete listing anyway). This document would not be unlikely to be of interest to those who already have an expert knowledge of the command line interface and do require any reference information. Or those readers who require detailed lists of options for each command, the man pages are better suited to this purpose. _________________________________________________________________ 1.4. Availability of sources The modifiable sources of the original book (in english), are available in LyX format (LyX Document Processor) or Machine-translated SGML (SGML markup language). LyX is a completely free document processor based on LaTeX, downloadable from the LyX homepage.. See Section 1.1 for the modifiable sources of this document. These are the official versions. We (the translators and current maintainers) plan to continue work on this document and add new chapters and enhancements. If you want to see the version we are currently working on (the "bleeding edge" version), check the GNU/Linux Command-Line Tools Summary Homepage from time to time (kindly hosted by Chris Karakas). _________________________________________________________________ 1.5. Conventions used in this guide The following conventions are used within this guide: italic Anything appearing in italic, like this is either an executable command or emphasized text. Tools (executable commands) are in italics to prevent confusion. Some tools have names which are real english words, such as the "locate" tool. key combinations Are represented by using a '-' (dash sign) inbetween the key(s), which must be used in combination. All combinations are also printed in italics to improve clarity. For example CTRL-Z means hold down the Control key and press the z key. admonitions Admonitions are little pictures used to emphasize something of importance to the reader. The five types used are: Note This is a note Notes often give important information about a tool. Tip This is a tip This will offer a useful switch or useful way to use a tool. Important This is something important This is something that is considered very important. Consider it like a note with extra importance, they are usually there to save the reader time. Caution This is a caution This will inform you of something that you be careful about (because it could be harmful to your system). Warning This is a warning This will inform you of something that you shouldn't do (because it probably will break something within your system). code examples Code examples are shown for most commands. Below is an example of what code looks like: Hello World, I'm a code example. :) command syntax (or a similar phrase) simply shows how you would normally use the command. Often real examples are used instead of explaining the command syntax. The phrase " Command syntax" is always followed by the way you would type a command in a shell. The standard syntax for any tool is usually: command -options file Note Note Note that some tools do not accept options. wildcards Also note that most commands, even when not explicitly stated, will work with standard wildcards (or globbing patterns) such as *, [A-Z] and various other standard wildcards. Refer to Section 20.4.1 for further information. access keys Access keys enable navigation through the document, without relying on a mouse. The following keys have been given special meaning in this document: P Previous page. N Next page. H Home of the document (Table of Contents). U Up (takes you one level up the section hierarchy). If you also happen to be reading the document from its original location, then the following access keys can also be used: S Start (takes you to the author's start page). T The current ("This") page, without the Sitemenu on the left. M The current page in a frameset, where the left frame contains a Menu. To use the access keys, you have to simultaneously press a modifier key, which may vary from browser to browser. For example in NN6+/Mozilla, the modifier key is ALT, so you have to use ALT-N to go to the next page, and ALT-P to come back. In other browsers such as IE6, the access keys just give focus to the associated link, so the sequence becomes ALT-N Enter . Try it, you'll like it! [icon_smile.png] _________________________________________________________________ 1.6. Resources used to create this document To create the GNU/Linux Command-Line Tools Summary, I used LyX, the document processor. To convert the LyX files to DocBook SGML I used the LyX-to-X Scripts created by Chris Karakas. You may also want to check out the db2lyx package, created by Dr. B Guillion, which can be used to convert LyX files to XML DocBook and XML DocBook to LyX. I also had assistance from various The Linux Documentation Project volunteers (see the contributors section Section 1.8 for specific details). _________________________________________________________________ 1.7. Feedback Feedback is necessary for the advancement of this guide. Positive, constructive criticism is encouraged. If you have ideas, suggestions, advice, or problems with this guide, please send an email to the author Gareth Anderson. Important Contributions If you wish to make contributions it is recommended (if possible) to read the LyX file(s) for this document. They contain various notes which you can't see in the other versions. These notes highlight the areas that need contributions, certain tools which I cannot understand, tools which have not been added, or tools which were removed. These notes also explain some of the structure of this document. _________________________________________________________________ 1.8. Contributors As you may be able to see, parts of this guide are based off various advice columns on GNU/Linux, anything that has being directly quoted from an article can be found in the references, Bibliography, section of this document. The following is a list of people who have made a significant contribution to this document, in a rough chronological order. Chris Karakas: Chris allowed the use of his lyxtox scripts to convert the LyX file of the document to working DocBook SGML output (to learn how to use the lyxtox scripts yourself, see Document processing with LyX and SGML). + Chris provided useful suggestions and advice, and added an index listing for many of the commands. + Chris is also responsible for the great looking HTML file for this document (the CSS file and HTML customisations are completely his work). + Chris has also helped fix up problems in the document (many times), especially with docbook/sgml related problems and LyX related issues. + Chris has also improved the structure of the document by adding labels and fixing minor errors. William West: William provided a thorough review of the document as required by the Linux Documentation Project. He is responsible for a variety of improvements to the quality of this document. His contributions include: + Improvements to the readability of this document. + Improvements to the structure and consistency of this document. + Various grammar improvements throughout the document. + Repair of some minor technical errors. Tabatha Persad: Tabatha, as the Linux Documentation Project Review Coordinator (at the time) also gave a brief review of this document. Her general advice was used to improve the structure, language and grammar of the document. Rahul Sundaram: Rahul provided a brief review of this document for the Linux Documentation Project. Advice from his brief review was integrated into this document to improve readability and structure, several references were added as recommended by Rahual. David Lawyer: David's criticism of the document (via the TLDP discuss list) were listened to, and attempts to improve the document were made. A number of his criticisms were addressed and improved. George Harmon: George provided a second language review. His detailed review of the material allowed me to improve the general grammar of the document and some minor errors. Machtelt Garrels (tille): Machtelt provided tips in regard to referencing the correct LDP documents from this guide. As well as general advice on improvements to the guide. _________________________________________________________________ Chapter 2. Legal The legal chapter provides information about the disclaimer that applies to the entire document and the licensing information. _________________________________________________________________ 2.1. Disclaimer No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. There may be errors and inaccuracies, that may of course be damaging to your system. Although this is highly unlikely, you should proceed with caution. The author does not accept any responsibility for any damage incurred. All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark. Naming of particular products or brands should not be seen as endorsements. UNIX is a registered trademark of The Open Group. _________________________________________________________________ 2.2. License Copyright © 2003 - 2006 Gareth Anderson. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License,Documentation License} Version 1.1 or any later version published by the Free Software Foundation;Foundation} with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license can be found in the section called the GNU Free Documentation License or at the GNU Documentation License Site. _________________________________________________________________ Chapter 3. The Unix Tools Philosophy A tool is a simple program, usually designed for a specific purpose, it is sometimes referred to (at least throughout this document) as a command. The " Unix tools philosophy" emerged during the creation of the UNIX operating system, after the breakthrough invention of the pipe '|' (refer to Chapter 6 for information on using the pipe). The pipe allowed the output of one program to be sent to the input of another. The tools philosophy was to have small programs to accomplish a particular task instead of trying to develop large monolithic programs to do a large number of tasks. To accomplish more complex tasks, tools would simply be connected together, using pipes. All the core UNIX system tools were designed so that they could operate together. The original text-based editors (and even TeX and LaTeX) use ASCII (the American text encoding standard; an open standard) and you can use tools such as; sed, awk, vi, grep, cat, more, tr and various other text-based tools in conjunction with these editors. Using this philosophy programmers avoided writing a program (within their larger program) that had already been written by someone else (this is sometimes called code recycling). For example, command line spell checkers are used by a number of different applications instead of having each application create its own own spell checker. This philosophy lives on today in GNU/Linux and various other UNIX system-based operating systems (FreeBSD, NetBSD, OpenBSD, etc.). For further information (articles) on the UNIX tools philosophy please see the further reading section, here: Section A.2.2.1 _________________________________________________________________ Chapter 4. Shell Tips The shell tips chapter provides handy tricks that you may wish to use when you are using a GNU/Linux shell (the command line interface). This information includes handy shortcut key combinations, the shell's command history and information on virtual terminals. Tip If you can't boot into your system If your having problems booting into your system you may like to use a shell so you can boot into your system and attempt to fix things up again. To do this you need to pass the "init=/bin/sh" to your system before you boot up. If you don't know how to do this please see Chapter 14, the technique is the same except this time you pass "init=bin/sh" rather than "single". _________________________________________________________________ 4.1. General Shell Tips Automatic Command Completion Use the TAB key and BASH will attempt to complete the command for you automatically. You can use it to complete command (tool) names. You can also use it when working with the file-system, when changing directories, copying files et cetera. There are also other lesser known ways to use automatic command completion (for example completing user names):[1] ESC-Y (Y: special character) Will attempt to complete the command name for you. If it fails it will either list the possible completions (if they exist). If there are none it will simply beep (and/or) flash the screen. CTRL-X-Y (Y: special character) Lists the possible completions (it won't attempt to complete it for you) or beep if there are no possible completions. Special-characters: Use the following special characters combined with either ESC-Y or CTRL-X-Y , where Y is some special character. For example ESC-$ or CTRL-X-$ to complete an environment variable name. + ~ (tilde) complete a user name + @ (at sign) complete a machine name + $ (dollars sign) complete an environment variable name + ! (exclamation mark) a magic character for completing a command name or a file name (/). The ! special character has the same function as the TAB key. It works in some other situations; for example when completing man page names. alias The alias command will list your current aliases. You can use unalias to remove the alias (to disable it just for one command add a "\" (back-slash) before the command)... An alias allows one command to be substituted for another. This is used to make a command do something else or to automatically add certain options. This can be either be done during one session using the alias command (see below) or the information can be added to the .bashrc file (found in the users home directory). Below is an example of what an alias section (within your .bashrc file) might look like: # my personal aliases alias cp='cp -vi' #to prompt when copying if you want to overwrite and will tel l you where information is going alias rm='rm -i' #Prompts you if you really want to remove it. alias mv='mv -i' #Prompts you if you are going to overwrite something On any Mandrake/Mandriva Linux system the global aliases (for all users) are all in /etc/profile.d/alias.sh. The above listed commands already have aliases, as well as several other commonly used commands. set -x set is one of bash's inbuilt commands, try looking in the bash manual for its many usage options. Using set with the -x option will make bash print out each command it is going to run before it runs it. This can be useful to find out what is happening with certain commands such as things being quoted that contain wildcards or special symbols that could cause problems, or complex aliases. Use set +x to turn this back off. Examples After using set -x you can run the command: ls The output printed before the command runs (for example): + ls -F --color=auto Which means that the command is really an alias to run ls with the -F and --color=auto options. Use a "\" (backslash) before the command to run it without the alias. \ (backslash) The backslash escape character can be used before a shell command to override any aliases. For example if rm was made into an alias for rm -i then typing "rm" would actually run rm -i. However, typing \rm lets the shell ignore the alias and just run rm (its runs exactly what you type), this way it won't confirm if you want to delete things. Caution Using rm Please note that the alias for the remove command is there for a reason. Using it incorrectly could remove files which you don't want removed. Only use \rm if you know exactly what you are doing (recovering files is not easy, rm does not send things to a recycle bin). The "\" character can be used before special characters (such as a space or a wildcard...) to stop bash from trying to expand them (a directory name with a space in it is possible this way). For example you could type cd My\ Directory\ With\ Spaces which normally wouldn't work. The "\" character can also be used to stop bash from expanding certain symbols (as an alternative you could use single quotation marks, although you may need to use both). Tip The TAB Key Please note that using the TAB key (automatic-command-completion) will automatically use escapes for spaces (so you don't have to type them manually). script The "script" command creates a typescript, or "capture log" of a shell session - it writes a copy of your session to a file, including commands you type and their output. ~ (tilde character) The tilde character is used as an alias to a users home directory. For example, if your user-name was "fred", instead of typing cd /home/fred you could simply type cd ~. Or to get to fred's tmp directory (under his home directory) you could type cd ~/tmp. Tip Home directory shortcut ~ (tilde) can also be used as a shortcut to other users home directories, simply type: ~user_name and it will take you to the user's home directory. Note that you need to spell the username exactly correct, no wildcards. set bell-style none This particular set command will turn off the system bell from the commandline (use xset -b for X windows). If you want the bell to stay off pernamently (no audible bell) then you can add this command to your ".bashrc" or ".bash_profile" (just add it to the same one you have your alises in...). reset The reset command re-initializes your current terminal. This can be useful when the text from your terminal's text becomes garbled, simply type "reset" and this will fix your terminal. exit Closes your current terminal (with x-terminals) or logs-out. Also try CTRL-D . logout Logs out of a terminal, also try CTRL-D . echo A little command that repeats anything you type. Example: echo "hello world" Simply displays " hello world". Example: echo rm -R * This will output what will be passed to the rm command (and therefore what would be deleted), putting echo before a command renders it harmless (it just expands wildcards so you know what it will do). Also try using the -e option with echo. This will allow you to use the escape character sequences to format the output of a line. Such as '\t' for tab, '\n' for newline etc. Tip Using echo to prevent accidents Typing: echo command(s) could save you the trouble of accidentally doing something you didn't expect. Using echo allows you to expand the wildcards to understand what will happen before you actually run the command. _________________________________________________________________ 4.2. The command-line history Using the command history Use the up and down key's to scroll through previously typed commands. Press [Enter] to execute them or use the left and right arrow keys to edit the command first. Also see history (below). The history command The history command can be used to list Bash's log of the commands you have typed: This log is called the "history". To access it type: history n This will only list the last n commands. Type "history" (without options) to see the the entire history list. You can also type !n to execute command number n. Use !! to execute the last command you typed. !-n will execute the command n times before (ie. !-1 is equivalent to !!). !string will execute the last command starting with that "string" and !?string? will execute the last command containing the word "string". For example: !cd Will re-run the command that you last typed starting with "cd". " commandName !*" will execute the "commandName" with any arguments you used on your last command. This maybe useful if you make a spelling mistake, for example. If you typed: emasc /home/fred/mywork.java /tmp/testme.java In an attempt to execute emacs on the above two files this will obviously fail. So what you can do is type: emacs !* This will execute emacs with the arguments that you last typed on the command line. In other words this is equivalent to typing: emacs /home/fred/mywork.java /tmp/testme.java Searching through the Command History ( CTRL-R ) Use the CTRL-R key to perform a "reverse-i-search". For example, if you wanted to use the command you used the last time you used snort, you would type: CTRL-R then type "snort". What you will see in the console window is: (reverse-i-search)`': After you have typed what you are looking for, use the CTRL-R key combination to scroll backward through the history. Use CTRL-R repeatedly to find every reference to the string you've entered. Once you've found the command you're looking for, use [Enter] to execute it. Alternatively, using the right or left arrow keys will place the command on an actual command line so you can edit it. _________________________________________________________________ 4.3. Other Key combinations GNU/Linux shells have many shortcut keys which you can use to speed up your work, below is a rough list of some (also see CTRL-R the history section of the commands, over here, Section 4.2). The CTRL-D the "end-of-file" (EOF) key combination can be used to quickly log out of any terminal. CTRL-D is also used in programs such as "at" to signal that you have finished typing your commands (the EOF command). The CTRL-Z key combination is used to stop a process. It can be used to put something in the background temporarily. For example, if you were editing a file with vim or emacs just press CTRL-Z to regain control of the terminal do what you want and then type fg to bring it back. For further information please see Section 9.3. Tip If fg doesn't work If fg doesn't work you may need to type jobs and then fg job_name or fg job_number The CTRL-A and CTRL-E key combinations are used for going to the start and end of the line on the command line. Use CTRL-A to jump to the start of the line, and CTRL-E to jump to the end of the line. The CTRL-K key combination can be used to cut or delete what is currently in front of the cursor. The CTRL-Y key combination can be used to paste the last thing you deleted (using CTRL-K or CTRL-W ). The CTRL-W key combination can be used to cut or delete the entire line that has being typed. _________________________________________________________________ 4.4. Virtual Terminals and screen Using the key combination ALT-F* keys you may change to different virtual terminals. You will have several (usually 6) virtual terminals setup with shells. Number 7 is usually setup with X you need to use CTRL-ALT-F* to change to a terminal from within X (X as in the X windowing system). screen is a great program that allows you to switch between multiple virtual terminals on the one physical terminal that you are using. Its a command-line based window manager, clearly this isn't that useful if you do have virtual terminals, but its amazingly useful when you log into machines remotely, using ssh and similar, see Section 13.3. It works on key-combinations, you type screen On the command-line to begin. Now you start with one virtual terminal by default, but using the key combination CTRL-A and then hitting "C" you can create another virtual terminal to use. Use CTRL-N to go to the next virtual terminal and CTRL-P to go to the previous virtual terminal. Also try hitting CTRL-A to go backwards and forwards between two particular terminals. screen also has various other abilities that you can test out. The documentation and guides are well written so please feel free to read the manual page or try searching the internet. _________________________________________________________________ Chapter 5. Help The help chapter provides information on how you may access the documentation of the GNU/Linux system. There is normally a document describing every single tool you have installed, even if its only brief... man This command displays summary information on a program from an online manual. For example typing man man will bring up the manual page for man (the manual page viewer). Note: q is the quit key. Command syntax: man program_name Tip Also try Specifying the section of the manual page, sometimes the man page is different for the same tool in different sections, note sections are numbered 1 to 9. Use apropos to find which section number to look in. The syntax to look at a different section is: man section_number tool_name For example: man 2 time This will show you the man page called time in section 2, the equivalent page in section 1 is completely different man -K keyword Search the manual pages for a string, as in it will search all manual pages for a particular string within each individual man page, it will then prompt whether you would like to view each page it will find. Use double quotes " and " if there are spaces in the string you are typing. Caution Speed issue Please be warned that this method is going to be really, really slow. You are searching *all* man pages for a string man -f command This will list details associated with the command. The root user must run makewhatis (see below) before this command will work. Note Equivalent to whatis This command is the same as running whatis info Provides a more detailed hyper-text manual on a particular command, this only works for some commands. Command syntax: info program_name whatis Displays a one line description of what a program does. The string needs to be an exact match, otherwise whatis won't output anything. Relies on the whatis database (see below). Command syntax: whatis program_name makewhatis Make the whatis database for apropos, whatis and man -f. Note Root Privileges This takes some time and you require root privileges to do this. apropos Searches the whatis database for strings, similar to whatis except it finds and prints anything matching the string (or any part of the string). Also relies on the whatis database (see above). Command syntax: apropos string Note Equivalent to... apropos is the same as doing man -k (lowercase k). Note Please note You need to run makewhatis (as root) so whatis, man -f and apropos will work. Tip Also try Using a program with the -?, --h, --help, and the -h options, they will display very short summary information on the command usage options. _________________________________________________________________ Chapter 6. Directing Input/Output The directing input/output chapter explains how you can use a program and send its output to a file or to another command that you wish to use. This technique is very powerful and there are a number of ways of doing this. _________________________________________________________________ 6.1. Concept Definitions All three of the following definitions are called " File Streams." They hold information that is either received from somewhere or sent to somewhere. In a UNIX system, the keyboard input (standard input), information printed to the screen (standard output) and error output (also printed to the screen) are treated as separate File Streams. Standard output Standard output is the output from the program printed to the screen, not including error output (see below). Standard input Standard input is the input from the user. Normally the keyboard is used as the standard input device in a UNIX system. Standard error Standard error is error output from programs. This output is also sent to the screen and will normally be seen mixed in with standard output. The difference between standard output and standard error is that standard error is unbuffered (it appears immediately on the screen) and standard error is only printed when something goes wrong (it will give you details of what went wrong). _________________________________________________________________ 6.2. Usage > The greater than symbol is used to send information somewhere (for example a text file) Example: cat file1 file2 > file1_and_2.txt This will concatenate the files together into one big file named "file1_and_2.txt". Note that this will overwrite any existing file. < The less than symbol will insert information from somewhere (a text file) as if you typed it yourself. Often used with commands that are designed to get information from standard input only. For example (using tr): tr [A-Z] [a-z] < fileName.txt > fileNameNew.txt The example above would insert the contents of "fileName.txt" into the input of tr and output the results to "fileNameNew.txt". >> The >> symbol appends (adds) information to the end of a file or creates one if the file doesn't exist. << The << symbol is sometimes used with commands that use standard input to take information. You simply type << word (where word can be any string) at the end of the command. The command takes your input until you type "word", which causes the command to terminate and process the input. Using << is an alternative to using CTRL-D (EOF key), except it uses a string to perform the end-of-file function. For example type "cat" (with no options...) and it will work on standard input. To stop entering standard input you would normally hit CTRL-D . As an alternative you can type "cat << FINISHED", then type what you want. When you are finished, instead of hitting CTRL-D you could type "FINISHED" and it will end (the word FINISHED will not be recorded). 2> Redirects error output. For example, to redirect the error output to /dev/null, so you do not see it, simply append this to the end of another command... For example: make some_file 2> /dev/null Runs make on a file and send all error output to /dev/null | The "pipe" command allows the output of one command to be sent to the input of another. For example: cat file1.txt file2.txt | less Concatenates the files together, then runs less on them. If you are only going to look at a single file, you would simply use less on the file... tee Sends output of a program to a file and to standard output. Think of it as a T intersection...it goes two ways. For example: ls /home/user | tee my_directories.txt Lists the files (displays the output on the screen) and sends the output to a file: "my_directories.txt". &> Redirects standard output and error output to a specific location. For example: make &> /dev/null Sends both error output and standard output to /dev/null so you won't see anything... _________________________________________________________________ 6.3. Command Substitution Command substitution is basically another way to do a pipe, you can use pipes and command substitution interchangeably, it's up to you which one you find easier... Command substitution can be done in two distinct ways. Method One (back-quotes) Simply type: command_1 `command_2 -options` This will execute "command_2" and it's output will become the input to "command_1". Tip Backquote key The back-quote key is usually located at the same place as the tilde, above the [Tab] key. Method Two (dollars sign) Simply type: command_1 $(command_2) This will execute "command_2" and it's output will become the input to "command_1". Using the pipe instead You can of course use pipes to do the same thing, if you don't know what a pipe is, please see Section 6.2. For example instead of doing: less $cat file1.txt file2.txt You could do: cat file1.txt file2.txt | less And end up with exactly the same result, it's up to you which way you find easier. _________________________________________________________________ 6.4. Performing more than one command Executing the second command only if the first is successful To do this you would type: command1 && command2 command2 will be executed if command1 successfully completes (if command1 fails command2 won't be run). This is called a logical AND. Executing the second command only if the first fails To do this you would type: command1 || command2 command2 will be executed if command1 does not successfully completes (if command1 is successful command2 won't be run). This is called a logical OR. Executing commands sequentially To execute command sequentially regardless of the success/failure of the previous you simply type: command1; command2 command2 will execute once command1 has completed. Tip More than two commands You can continue to use ';' (semicolon) characters to do more and more commands on the one line. _________________________________________________________________ Chapter 7. Working with the file-system The working with the file-system chapter explains a number of commands that you use to move around the file system hierarchy and manipulate the files. Also explained are finding files and how to mass-rename files. _________________________________________________________________ 7.1. Moving around the filesystem cd Change directory. Use " cd .." to go up one directory. One dot '.' represents the current directory while two dots '..' represent the parent directory. " cd -" will return you to the previous directory (a bit like an "undo"). You can also use cd absolute path or cd relative path (see below): Absolute paths An " absolute path" is easily recognised from the leading /. The / means that you start at the top level directory and continue down. For Example to get to /boot/grub you would type: cd /boot/grub This is an absolute path because you start at the top of the hierarchy and go downwards from there (it doesn't matter where in the filesystem you were when you typed the command). Relative paths A " relative path" doesn't have a preceding slash. Use a relative path when you start from a directory below the top level directory structure. This is dependent on where you are in the filesystem. For example if you are in root's home directory and want to get to /root/music, you type: cd music Please note that there is no / using the above cd command. Using a / would cause this to be an absolute path, working from the top of the hierarchy downward. ls List files and directories. Typing "ls" will list files and directories, but will not list hidden files or directories that start with a leading '.'. Example options: + ls -l --- long style, this lists permissions, file size, modification date, ownership. + ls -a --- this means "show all", this shows hidden files, by default any file or directory starting with a '.' will not be shown. + ls -d --- list directories only (see example below) + ls -F --- append symbols to particular files, such as * (asterisk) for executable files. + ls -S --- sort the output of the command in decending order sorted by size. + ls -R --- (recursive) to list everything in the directories below as well as the current directory. Command syntax, either: ls -options This simply lists everything in the current directory, the options are not required (options such as -l, -a et cetera). ls -options string This lists files using a certain string. The string can contain standard wildcards to list multiple files, to learn more about standard wildcards please read Section 20.4.1 You can use ls -d to show directories that match an exact string, or use standard wildcards. Type " ls -d */" to list all subdirectories of the current directory. Depending on the setup of your aliases (see Chapter 4) you may simply be able to type lsd as the equivalent to ls -d */. Examples for ls -d: ls -d */ Lists all subdirectories of current directory. ls -d string* Lists directories that start with "string". ls -d /usr/*/*/doc Lists all directories that are two levels below the /usr/ directory and have a directory called "doc", this trick can come in quite handy sometimes. Tip You can also use Depending on how your aliases (see Chapter 4) are setup you can also use l, la (list all) and ll (list long) to perform the above commands pwd Print working directory. Print the absolute (complete) path to the directory the user is currently in. Command syntax: pwd This will tell you the full path to what directory you are in, for example it may output "/usr/local/bin" if you are currently in that directory. tree Outputs an ASCII text tree graph starting at a given directory (by default the current directory). This command recursively lists all files and all directories. In other words, it will list files within the directories below the current one, as well as all files in the current directory. tree has a large number of options, refer to the manual page for details. Command syntax: tree or tree -option(s) /optional/directory/to/list _________________________________________________________________ 7.1.1. Finding files find find is a tool which looks for files on a filesystem. find has a large number of options which can be used to customise the search (refer to the manual/info pages). Note that find works with standard wildcards, Section 20.4.1, and can work with regular expressions, Section 20.4.2. Basic example: finsd / -name file This would look for a file named "file" and start at the root directory (it will search all directories including those that are mounted filesystems). The `-name' option is case sensitive you can use the `-iname' option to find something regardless of case. Use the '-regex' and '-iregex' to find something according to a regular expression (either case sensitive or case insensitive respectively). The '-exec' option is one of the more advanced find operations. It executes a command on the files it finds (such as moving or removing it or anything else...). To use the -exec option: use find to find something, then add the -exec option to the end, then: command_to_be_executed (1) then '{}' (curly brackets) (2) then the argumen ts (for example a new directory) and finally a ';' (3) . See below for an example of use this command. (1) This is the tool you want to execute on the files find locates. For example if you wanted to remove everything it finds then you would use -exec rm -f (2) The curly brackets are used in find to represent the current file which has been found. ie. If it found the file shopping.doc then {} would be substituted with shopping.doc. It would then continue to substitute {} for each file it finds. The brackets are normally protected by backslashes (\) or single-quotation marks ('), to stop bash expanding them (trying to interpret them as a special command eg. a wildcard). (3) This is the symbol used by find to signal the end of the commands. It's usually protected by a backslash (\) or quotes to stop bash from trying to expand it. find / -name '*.doc' -exec cp '{}' /tmp/ ';' The above command would find any files with the extension '.doc' and copy them to your /tmp directory, obviously this command is quite useless, it's just an example of what find can do. Note that the quotation marks are there to stop bash from trying to interpret the other characters as something. Excluding particular folders with find can be quite confusing, but it may be necessary if you want to search your main disk (without searching every mounted filesystem). Use the -path option to exclude the particular folder (note cannot have a '/' (forward slash) on the end) and the -prune option to exclude the subdirectories. An example is below: find / -path '/mnt/win_c' -prune -o -name "string" -print This example will search your entire directory tree (everything that is mounted under it) excluding /mnt/win_c and all of the subdirectories under /mnt/win_c. When using the -path option you can use wildcards. Note that you could add more -path '/directory' statements on if you wanted. Find has many, many different options, refer to the manual (and info) page for more details. slocate slocate outputs a list of all files on the system that match the pattern, giving their full path name (it doesn't have to be an exact match, anything which contains the word is shown). Note Replaces locate Secure locate is a replacement for locate, both have identical syntax. On most distributions locate is an alias to slocate. Commmand syntax: slocate string Note This won't work unless You need to run either updatedb (as root) or slocate -u (as root) for slocate to work. whereis whereis locates where the binary, source, and manual page is for a particular program, it uses exact matches only, if you only know part of the name use slocate. Command syntax: whereis program_name which Virtually the same as whereis, except it only finds the executable (the physical program). It only looks in the PATH (environment variable) of a users shell. Use the -a option to list all occurances of the particular program_name in your path (so if theres more than one you can see it). Command syntax: which program_name _________________________________________________________________ 7.2. Working with files and folders mkdir Make a directory. Use mkdir -p to create subdirectories automatically. Note Directories are Folders Directories are sometimes called folders in other operating systems (such as Microsoft Windows) Examples: mkdir -p /home/matt/work/maths This would create the directories "work" and "maths" under matt's home directory (if matt's home directory didn't exist it would create that too). mkdir foo This would create a directory in the current path named "foo". rm Remove/delete a file(s) or directories(s). You can use standard wildcards with this command Section 20.4.1. Command syntax: rm -options file_or_folder You can of course use standard wildcards to delete multiple files or multiple directories and files. Use the -R or -r option to remove recursively, this removes everything within subdirectories. Also try the -f option to force removal (useful when you don't want to be prompted). To remove files with special characters, for example, "da*d" type: rm -- da*d After the "--" it uses everything literally instead of using them as standard wildcards. This may be useful when you need to delete a filename containing weird symbols (this also works on other tools...). Tip Disabling Aliases (per execution) On some systems such as Mandrake an alias will send rm to rm -i (prompting you for every file you wish to delete). To override this use: \rm -R directory (using the \ disables the alias for this run only) rmdir Remove an empty directory. If you want to remove a directory with files in it type " rm -R directory", read above for information on rm -R Command syntax: rmdir directory This will only remove directory if it's empty otherwise it will exit with an error message. mv Move a file or a directory to a new location or rename a file/directory. Rename example: mv filename1 filename2 Renames filename1 to filename2. To move a file or directory, simply type: mv original_file_or_folder new_location Note that this command can use standard wildcards Section 20.4.1 to move files (not for renaming). Tip Move and rename Note that you can also move and rename a file in a single command. The difference is with the destination (right hand side) you change the filename to the new name of the file. For example typing: mv /etc/configuration.txt /home/joe/backupconfig This would move the file "configuration.txt" to /home/joe/ and rename it "backupconfig" cp Copy a file. Has a number of useful options, such as -R (or -r) which recursively copies directories and subdirectories. Command syntax: cp -options file_or_files new_location Examples: cp file1 file2 Simply copy file1 to file2 (in the same directory). cp /tmp/file1 ~/file2 /mnt/win_c Where the last option is the directory to be copy to. Copies two files from different areas of the hard disk to /mnt/win_c cp -R directory_and_or_files new_location This command will copy directories (and all subdirectories) and/or files to new_location Note that this command can use standard wildcards Section 20.4.1 to copy multiple files. You may also like to try the "-u" when moving large directories around, this copies only if the source file is newer than the destination to where you are copying to, or if the destination file does not exist at all. ln Create a link to a file. There are two types of links: Hard links Hard links are considered pointers to a file (the number is listed by typing ls -l). Each hard-link is a reference to a file. The file itself only goes away when all hard-links are deleted. If you delete the original file and there are hard links to it the original file will remain. Example: ln target_name link_name Will create a "hard link" to target_name called link_name, you need to delete both of these to remove the file. Symbolic links Symbolic links are created by typing " ln -s". When you remove the original file the symbolic link becomes broken, a symbolic link is similar to a windows "short-cut". The advantage of symbolic links is that they can be to something on another file-system, while hard-links can only exist on the same file-system. For example: ln -s target_name link_name This creates a symbolic link to "target_name" called "link_name", if you delete the original file the symbolic link won't work (it becomes a broken link). shred Securely remove a file by overwriting it first. Prevents the data from being recovered by software (and even by most hardware), please be very careful when using shred as you may never be able to retrieve the data you have run the application on. For example: shred -n 2 -z -v /dev/hda1 What this tells shred, is to overwrite the partition 2 times with random data (- n 2) then finish it up by writing over it with zeroes (-z) and show you its prog ress (-v). Of course, change /dev/hda1 to whatever your partition is. Each pass can take some time, which is why I set it to only do 2 random passes instead of the default 25. You can adjust this number, of course, to your particular level of paranoia and the amount of time you have. Since shred writes on such a low-level, it doesn't actually matter what kind of filesystem is on the partition--everything will be unrecoverable. Once shred is finished, you can shut down the machine and sell or throw away the drive with peace of mind. ...However, even shre dding devices is not always completely reliable. For example, most disks map out bad sectors invisibly to the application; if the bad sectors contain sensitiv e data, `shred' won't be able to destroy it. [ shred info page ].[2] Note Shredding files doesn't work with all filesystems Please note that as mentioned in the shred manual page (please see the manual and preferably info pages for more information). shred does not work correctly on log-structured or journaled filesystems, such as JFS, ReiserFS, XFS, Ext3 and many other modern filesystems Tip Alternatives to using shred shred has its disadvantages when run on a filesystem. First of all since it has to be installed you cannot run shred on your operating systems filesystem, you also cannot use shred on a windows machine easily since you cannot install shred on this machine. You may like to try alternatives such as the DBAN project that create self-booting floppy disks that can completely erase a machines hard disk. You may also like to see how chattr can assist you in shredding files once they are removed (it has similar problems to shred, only ext2 and ext3 style filesystems...), please see Section 14.2. du Displays information about file size. Use du filename to display the size of a particular file. If you use it on directories it will display the information on the size of the files in the directory and each subdirectory. Options for du (use du -option(s)): + -c -- this will make du print a grand total after all arguments have being processed. + -s -- summarises for each argument (prints the total). + -h -- prints things in " human readable" mode; for example printing 1M (megabyte) rather than 1,024,000 (bytes). Using the -hs options on a directory will display the total size of the directory and all subdirectories. Command syntax: du -options file_directory_or_files Example: du -hs * This command will list the size of all files in the current directory and it will list the size of subdirectories, it will list things in human-readable sizes using 1024 Kb is a Megabyte, M for megabyte, K for kilobyte etc. file Attempts to find out what type of file it is, for example it may say it's: binary, an image file (well it will say jpeg, bmp et cetera), ASCII text, C header file and many other kinds of files, it's a very useful utility. Command syntax: file file_name stat Tells you detailed information about a file, including inode number creation/access date. Also has many advanced options and uses. For simple use type: stat file dd Copies data on a very low level and can be used to create copies of disks Section 20.3 and many other things (for example CD image files). dd can also perform conversions on files and vary the block size used when writing the file. Command syntax, note the block size and count are optional and you can use files instead of devices... Note Please note dd is an advanced and difficult to use command. Its also very powerful, so be careful what you do with it Command syntax: dd if=/dev/xxx of=/dev/xxx bs=xxxx count=x Warning Warning The command dd is used to work on a very low level. It can be used to overwrite important information such as your master-boot record or various important sections of your hard-disk. Please be careful when using it (especially when working with devices instead of files). touch This command is used to create empty files, simply do touch file_name. It is also used to update the timestamps on files. touch can be used to change the time and/or date of a file: touch -t 05070915 my_report.txt[3] This command would change the time stamp on my_report.txt so that it would look like you created it at 9:15. The first four digits stand for May 7th (0507), in MM-DD (American style), and the last four (0915) the time, 9:15 in the morning. Instead of using plain numbers to change the time, you can use options similar to that of the date tool. For example: touch -d '5 May 2000' some_file.txt You can also use --date= instead of -d. Also have a look at the date command under Section 8.1 for examples on using -d and --date= (the syntax for the date part is exactly the same when using -d or --date). split Splits files into several smaller files. Use the -b xx option to split into xx bytes, also try -k for kilobytes, and -m for megabytes. You can use it to split text files and any other files... you can use cat to re-combine the files. This may be useful if you have to transfer something to floppy disks or you wish to divide text files into certain sizes. Command syntax: split -options file This will split the input file into 1000 lines of input each (thats the default...), and output (using the above example), with the input name file, "fileaa" (1st part of file), "fileab" (2nd part of file), "fileac" (3rd part of file) etc. until the there is no more of the file left to split. _________________________________________________________________ 7.3. Mass Rename/copy/link Tools There are a few different ways to perform mass renaming of files in GNU/Linux (yes, mass renaming is possible!). There is also a perl script that renames the extentions on files, see Chapter 19. Below are three ways to perform mass renaming of files, using the commands mmv, rename (a perl script) or some bash shell scripting. mmv mmv is a mass move/copy/renaming tool that uses standard wildcards to perform its functions. mmv's manual page is quite difficult to understand, I have only a limited understandi ng of this tool. However mmv supports some standard wildcards. According to the manual the ';' wildcard is useful for matching files at any depth in the directory tree (ie it will go below the current directory, recursively). An example of how to use mmv is shown below: mmv \*.JPG \#1.jpg The first pattern matches anything with a ".JPG" and renames each file (the "#1" matches the first wildcard) to ".jpg". Each time you use a \(wildcard) you can use a #x to get that wildcard. Where x is a positive number starting at 1. Tip mmv Homepage You can find mmv on the web here. Also be aware that certain options used with mmv are also applicable to other tools in the suite, these include mcp (mass copy), mad (mass append contents of source file to target name), mln (mass link to a source file). Tip Tip: A Java alternative to mmv which runs on both GNU/Linux and Windows is available, Esomaniac rename rename is a perl script which can be used to mass rename files according to a regular expression. An example for renaming all ".JPG" files to ".jpg" is: rename 's/\.JPG$/.jpg/' *.JPG Note Finding rename You can get rename from various places. I would recommend trying CPAN Search Site, I found the script here Rename Script Version 1.4 Bash scripting Bash scripting is one way to rename files. You can develop a set of instructions (a script) to rename files. Scripts are useful if you don't have mmv or rename... One way to this is shown below: for i in *.JPG; do mv $i `basename $i JPG`jpg; done Note that the above came script came from a usenet post. Unfortunately I do not know the author's name. The first line says find everything with the ".JPG" extension (capitals only, because unix is case sensitive). The second line uses basename (type man basename for more details) with the '$i' argument. The '$i' is a string containing the name of the file that matches. The next portion of the line removes the JPG extension from the end and adds the jpg extention to each file. The command mv is run on the output. An alternative is: for i in *.JPG; do mv $i ${i%%.JPG}.jpg; done The above script renames files using a built-in bash function. For more information on bash scripting you may like to see the advanced bash scripting guide, authored by Mendel Cooper. _________________________________________________________________ Chapter 8. Finding information about the system time If you are looking for how to change the time please refer to date here: Section 8.1. time is a utility to measure the amount of time it takes a program to execute. It also measures CPU usage and displays statistics. Use time -v (verbose mode) to display even more detailed statistics about the particular program. Example usage: time program_name options /proc The files under the /proc (process information pseudo file-system) show various information about the system. Consider it a window to the information that the kernel uses. For example: cat /proc/cpuinfo Displays information about the CPU. less /proc/modules To view information on what kernel-modules are loaded on your system. dmesg dmesg can be used to print (or control) the " kernel ring buffer". dmesg is generally used to print the contents of your bootup messages displayed by the kernel. This is often useful when debugging problems. Simply type: dmesg df Displays information about the space on mounted file-systems. Use the .h option to have df list the space in a 'human readable' format. ie. if there are 1024 kilobytes left (approximately) then df will say there is 1MB left. Command syntax: df -options /dev/hdx The latter part is optional, you can simply use df with or without options to list space on all file-systems. who Displays information on which users are logged into the system including the time they logged in. Command syntax: who w Displays information on who is logged into the system and what they are doing (ie. the processes they are running). It's similar to who but displays slightly different information. Command syntax: w users Very similar to who except it only prints out the user names who are currently logged in. (Doesn't need or take any options). Command syntax: users last Displays records of when various users have logged in or out. This includes information on when the computer was rebooted. To execute this simply type: last lastlog Displays a list of users and what day/time they logged into the system. Simply type: lastlog whoami Tells the user who they are currently logged in as. Doesn't need or take any options. Simply type: whoami free Displays memory statistics (total, free, used, cached, swap). Use the -t option to display totals of everything and use the -m to display memory in megabytes. Example: free -tm This will display the memory usage including totals in megabytes. uptime Print how long the computer has been "up", how long the computer has been running. It also displays the number of users and the processor load (how hard the CPU has been worked...). Note: the w command displays uptime's output as the top line of its output when it is executed (ie. you could use w instead...). uname uname is used to print information on the system such as OS type, kernel version et cetera. Some uname options: + -a --- print all the available information + -m --- print only information related to the machine itself. + -n --- print only the machine hostname. + -r --- print the release number of the current kernel. + -s --- print the operating system name + -p --- print the processor type. Command syntax: uname -options xargs Note that xargs is an advanced, confusing, yet powerful command. xargs is a command used to run other commands as many times as necessary, this way it prevents any kind of overload... When you run a command then | xargs command2. The results of command1 will be passed to command2. Understanding xargs tends to be very difficult and my explanation is not the best. Refer to the examples below or try [6] of the Bibliography for another xargs tutorial. Note Alternatives to using xargs Please note that the below explanation of xargs is not the strongest (at the time of writing I could not find anything better :()). Alternatives may include writing a simple bash script to do the job which is not the most difficult task in the world. Examples: ls | xargs grep work The first command is obvious, it will list the files in the current directory. For each line of output of ls, xargs will run grep on that particular line and look for the string "work". The output have the each time grep is executed on a new line, the output would look like: file_name: results_of_grep If grep didn't find the word then there would be no output if it had an error then it will output the error. Obviously this isn't very useful (you could just do grep *, its just an example... xargs also takes various options: + -nx --- will group the first x commands together + -lx --- xargs will execute the command for every x number of lines of input + -p --- prompt whether or not to execute this particular string + -t --- (tell) be verbose, echo each command before performing it + -i --- will use substitution similar to find's -exec option, it will execute certain commands on something. Example: ls dir1 | xargs -i mv dir1/'{}' dir2/'{}' The {} would be substituted for the current input (in this example the current file/directory) listed within the directory. The above command would move every file listed in dir1 to dir2. Obviously this command won't be too useful, it would be easier to go to dir1 and type mv * ../dir2 Here is a more useful example: \ls *.wav | xargs -i lame -h '{}' '{}'.mp3 This would find all wave files within the current directory and convert them to mp3 files (encoded with lame) and append a .mp3 to the end of the file, unfortunately it doesn't remove the .wav and so its not too useful...but it works. _________________________________________________________________ 8.1. Date/Time/Calendars There is one command to change both the date and time on a UNIX like system, date, there is also a simple calendar utility, cal. If you are looking to change the time-stamps on files please see Chapter 8 date Tells you the date (and the time) and is also used to set the date/time. To set the date, type date MM:DD:YYYY (American style date) where MM is month, DD is the number of days within the month and YYYY is the year. For example to set the date to the 1st January 2000 you would type: date 01:01:2000 To set the time (where the -s option is to set a new time), type: date -s hh:mm:ss Another useful option you can use is --date="string" (or -d "string") option to display a date from x days ago or in x days (or x weeks, months, years et cetera). See the examples below. Examples: date --date="3 months 1 day ago" Will print the date 3 months and 1 day ago from the current date. Note that --date="x month x day ago" and -d "x month x day ago" are equivalent. date -d "3 days" The above command will print the date 3 days forward in time from now. cal Typing cal will give you the calendar of the present month on your screen, in the nice standard calendar format. There are various options to customise the calendar, refer to the info/man page. Example: cal -y year Will display a calendar for a specific year, simply use cal -y to print the calendar for the current year. cal 2 2004 This will display the calendar for February 2004 _________________________________________________________________ 8.2. Finding information about partitions There are a number of ways to find out information on your hard disk drives, for information on mounted partitions also try df in Chapter 8 Using the proc filesystem You can look through the information in the relevant area of the proc filesystem, under the directory of either /proc/ide/ or /proc/ide?/hd? where the first question mark is a number and the second is a letter (starting with 'a'). For example: cd /proc/ide0/hda Under this directory there will be various information on the hard drive or cdrom connected. Using fdisk Using fdisk with the -l option will output information on any hard drives connected to the system and information on their partitions (for example, the type of partition). Information relating to using fdisk to partition hard disks can be found in your distributions documentation, the fdisk manual page or online. Note Root Access Required This command needs root access to work _________________________________________________________________ Chapter 9. Controlling the system The controlling the system chapter details commands that you may wish to use to interact with devices on your system and then details how to control processes and services/daemons. eject eject simply tells a device to open (eject). Useful for cdrom/DVD drives. For example the command below would eject the cdrom-drive (if your cdrom is linked to /dev/cdrom): eject /dev/cdrom Note This won't work unless This will only work if the user has permission to mount the partition. Please see the tip in Section 9.1 for more information. _________________________________________________________________ 9.1. Mounting and Unmounting (Floppy/CDROM/Hard-drive Partitions) Tip Allowing Users to mount partitions By default a UNIX system will allow normal users to unmount partitions. However unless given permission by the super-user, users will not be allowed to mount partitions. The commands listed below will not work for normal users unless users have permission to mount that device. If your particular distribution is setup not to allow users to mount partitions its not very hard to change this, simply edit the /etc/fstab file (as root) and: Replace the word "defaults" with "user" or Add "user" to the end of the options list for the particular partition(s). mount Mount a device. Attach the device to the file-system hierarchy (the tree ( / )). This needs to be done so you can access the drive (see below, Section 9.1 for an example). umount 'Unmount' a device. The command umount (no 'n') unmount's a device. It removes it from the file-system hierarchy (the tree ( / )). This needs to be done before you remove a floppy/CDROM or any other removable device (see below, Section 9.1 for an example). smbmount //wincomp/c /mnt/win Where "win" would be the place you want it mounted and "wincomp" is the IP address or name (the name will only work if it's listed in /etc/hosts) of your windows computer. smbmount is a tool from the samba package, it can mount a remote windows file system onto your current computer. Un-mounting uses the same syntax as 'umount', as listed above, or you may like to use: smbumont /mountpoint Examples of how to mount a file-system: mount -t ext2 /dev/fd0 /mnt/floppy (1) mount -t iso9660 /dev/hdb /mnt/cdrom (2) mount -t iso /tmp/image_file /mnt/iso_file/ -o loop (3) (1) The windows filesystem is known as vfat (standard on Windows 9x) or NFTS (standard on Windows 2000 and XP). (2) for CDROM's (3) This will mount an image file (usually a CD image file) so you can view/change the files (it will appear to be like any other device). Note The -t option On any system running a newer version of the Linux kernel the -t option is not always necessary and can be left out. Examples of how to unmount a file-system (necessary before you eject/remove disk): umount /mount_point An example unmount point could be "/mnt/floppy" or "/mnt/cdrom" _________________________________________________________________ 9.2. Shutting Down/Rebooting the System shutdown now Shutdown the computer immediately (don't power down). Note that in UNIX systems this kind of shutdown means to go to " single-user mode". Single user mode is a mode where only the administrator (root) has access to the computer, this mode is designed for maintenance and is often used for repairs. For example this would take you to single user mode shutdown now shutdown -h now Shutdown (-h = halt) the computer immediately. It begins the shutdown procedure, press CTRL-C (break-key) to stop it. After the end of the command you can also leave a message in quotation marks which will be broad-casted to all users, for example: shutdown -h now "Warning system malfunction, self-destruct imminent" This would halt the system and send the message to anyone who was logged in. Note you can put a time instead of "now" or "+x minutes" (any number of minutes is appropriate) or you can set an exact time. For example to shutdown at 11:50 type: shutdown -h 11:50 Note Shutdown -h vs poweroff On some systems, shutdown -h and halt do not actually poweroff the system. On systems that do not power off with these commands use the poweroff command halt Same as above, doesn't take any options, just shuts down immediately. shutdown -r now Shutdown ( -r = reboot) the computer immediately. It begins the reboot procedure, press CTRL-C (break-key) to stop it. After the end of the command you can also leave a message in quotation marks which will be broad-casted to all users, for example: shutdown -r now "Warning system rebooting, all files will be destroyed" This would reboot the system and send the message to anyone who was logged in. Note you can put a time instead of "now" or "+x minutes" (any number of minutes is appropriate) or you can set an exact time. For example to reboot at 11:50 type: shutdown -r 11:50 reboot Same as above, doesn't take any options, reboots immediately. CTRL-ALT-DEL (key-combination) May be used from a terminal to reboot or shutdown, it depends on your system configuration. Note that this doesn't work from an xterminal. CTRL-ALT-DEL begins the reboot/shutdown immediately, the user does not have to be logged in. Tip You can change the behaviour of CTRL-ALT-DEL from rebooting To disable CTRL-ALT-DEL from rebooting your computer (or to have it do something different), you can edit the /etc/inittab file (as root). Here is how it looks on a Mandrake/Mandriva Linux system: # Trap CTRL-ALT-DEL ca::ctrlaltdel:/sbin/shutdown -t3 -r now Note that the # means a comment (and is not used). If you simply put a # (hash) before the command it would disable it (it would become a comment). You could also change the command it runs for example if you changed the -r to a -h the computer would turn off instead of rebooting, or you could have it do anything you want. It's up to your creativity to make it do something interesting. _________________________________________________________________ 9.3. Controlling Processes ps Will give you a list of the processes running on your system. With no options, ps will list processes that belong to the current user and have a controlling terminal. Example options include: + -aux --- list all running processes (by all users with some information). + -a --- list all processes from all users. + -u --- list more information including user names, %cpu usage, and %mem usage et cetera. + -x --- list processes without controlling terminals. + -l --- display different information including UID and nice value. + --forest --- this makes it easier to see the process hierarchy, which will give you an indication of how the various processes on your system interrelate (although you should also try pstree). For example to list all running processes with additional information, simply type: ps -aux pstree Displays the processes in the form of a tree structure (similar to how tree does it for directories). Use the -p option to show process id's. Example: pstree -p This would list all processes and their ids. pgrep This command is useful for finding the process id of a particular process when you know part of its name. Use the -l option to list the name of the process as well and the -u option to search via a particular user(s). Normally pgrep will only return the pid number; this way you can use it with other commands. Examples: kill $(pgrep mozilla) This would kill any process name that starts with mozilla. Note that this is the same as using pkill (see below). If you are unfamiliar with the $( ) part of this command, please refer to Section 6.4. To list processes id's and names type: pgrep -l process_name top Displays the 'top' (as in CPU usage) processes, provides more detail than ps. top also provides an updated display, it has many option that make it fully customisable, refer to the manual or info page for details. kill To kill processes on your system, you will need their pid's or id's . Use ps or pstree to find out the process id's (pids), or use jobs to find out id's. Tip killall and pkill - kill a process by name pkill and killall can be a lot easier to use than kill. pkill allows you to type part of the name of a process to kill it, while killall requires the full process name. See below for more information. Examples: kill pid Simply kill a process (allow it time to save it's files and exit) kill %id Same as above, except it uses an id instead of a pid, you need to use a % (percent) when using an id to kill. kill -kill pid Force a process to be killed (won't allow files to be saved or updated); only use when necessary because all data that the program had will be lost. There are also many other kill options such as kill -HUP (hangup)... refer to the manual/info pages for more information. killall Kill a process by it's name, uses names instead of process id's (pids). Use -v to have killall report whether the kill was successful or not and -i for interactive mode (will prompt you before attempting to kill). Tip pkill - a little like a killall with regular expressions pkill is another command that allows processes to be killed but does so using regular expressions. See below for more information. For example: killall -iv mozilla Would kill anything named "mozilla" and prompt you before each kill and report whether the kill was successful or not. Unfortunately you need to get the name exactly right for killall to work, you would need to use "mozilla-bin" to kill the mozilla browser. If you want something where you don't need to know the exact name try pkill (below). pkill pkill is used to kill processes according to an extended regular expression. Use the -u option to kill using a user name(s) and process name (for example to only kill a process of a certain user). pkill can also send specific signals to processes. For normal usage simply type: pkill process_name Note that the "process_name" doesn't have to be an exact match... Or to kill the "process_name" of only the users "fred" and "anon" type: pkill -u fred anon process_name skill skill is used to send a command/username/tty a particular signal. skill has a number of options available to ensure correct interpretation (otherwise it just guesses what it is), simply type skill -option(s) + -L --- list the various signals that can be sent + -u --- specify a username; this is obviously followed by the user name or a space-seperated list of usernames. + -p --- process id (followed by the process id) + -c --- command name (this is the same as killall) + -t --- (tty number) + -v --- verbose mode + -i --- interactive mode. skill can be used to stop, continue, or kill processes using the username, command name or process id (or send them any variety of signals you like). Useful example: skill -STOP abusive_user_name The above command will stop all of that users processes, this will cause his screen to freeze until you type: skill -CONT abusive_user_name This would tell that all processes may continue as before. Note that this would only work if you are root. Also note you can list more than one user name with the command so it will apply to multiple users. CTRL-C The break key, will kill (break, stop) something that's running on your terminal. jobs Prints currently running jobs, as in processes you have executed within the shell. bg Backgrounds a process. To start a program in the background (so it doesn't take over the terminal) use an "&" (ampersand) sign at the end of the command. You usually use CTRL-Z to suspend something you are currently using. You can simply use bg to resume in the background the last job suspended... Command syntax: bg job_number or bg job_name fg Bring a process to the foreground, so you can interact with it. The process will use your current terminal. Note simply use fg to foreground the last job number suspended... You can bring jobs to the foreground by name or by number (use jobs to find the number). Command syntax: fg job_number or fg job_name nice Sets the priority for a process. nice -20 is the maximum priority (only administrative users can assign negative priorities), nice 20 is the minimum priority. You must be root to give a process a higher priority, but you can always lower the priority of your own processes... Example: nice -20 make Would execute make and it would run at maximum priority. renice Changes the priority of an existing command. You may use the options -u to change the priorities of all processes for a particular user name and -g to change priorities for all processes of a particular group. The default is to change via the process id number. Example: renice +20 2222 This would change the priority of process 2222 to +20 (minimum priority). snice snice works very similarly to skill, only it changes the priority of the process(es). Its function is similar to that of renice. To use options (to ensure correct interpretation) you simply type snice -option(s): + -u --- specify a username; this is obviously followed by the user name or a space-seperated list of usernames. + -p --- process id (followed by the process id) + -c --- command name (this is the same as killall) + -t --- (tty number) + -v --- verbose mode + -i --- interactive mode. Example: snice -10 -u root This would increase the priority of all root's processes. _________________________________________________________________ 9.4. Controlling services Concept Definitions UNIX systems use scripts to control "daemons" which provide "services" (for example your sound output) to run a UNIX system. UNIX systems consist of a variety of services (daemons). A "daemon" is a system process which runs in the background (zero interaction) performing a particular task. Daemons normally have a "d" on the end of their name and either listen for certain events or perform a system task, for example sshd listens for secure shell requests to the particular machine and handles them when they occur. Daemons usually perform critical system tasks such as control swap-space, memory management and various other tasks. service service is a shell script available on Mandrake/Mandriva and Redhat systems which allows you to perform various tasks on services. + Use the -s option to print the status of all services available + Use the -f option followed by a service name to restart that particular service. + Use the -R option to restart all services (note that this will kill any current services running, including X). For example to restart the daemon sshd you would type: service -f sshd Using the script directly You may also execute the shell script directly from /etc/init.d. Simply go to that directory then type ./script_name. Executing the script should return the options it can take, by default they will be: + restart --- this will make the service stop and then start again. + start --- this option will start a service (assuming its not running). + stop --- this option will stop a service (assuming its running). + status --- this option will tell you about the service _________________________________________________________________ Chapter 10. Managing users su username (Switch User), change to a different user. Use su - to switch to root or su username, to switch to a different username. Tip Using sudo Its often considered better practice to use the sudo command rather than switch to the root user The sudo command allows you to perform actions as root but logs the actions you take (so you can trace anything that was done to the system by yourself or others). sudo has a very good manual page which provides plenty of information about it. You use sudo similar to how you execute a normal command with sudo prepended to it, for example: sudo rpm -U myrpm.i386.rpm This would allow you to install a rpm even if you have the correct sudo access Note that if you want to return to your original user you don't use su again, type exit or press CTRL-D . Simply typing su will give you some root privileges, but there are minor complications relating to environment variables. It's generally considered better practice to use su - because it has no restrictions. root The superuser. This user has power over everything and all, and can do anything with the system (including destroy it, and of course fix it :)). This user is used to perform most administration functions on the system. _________________________________________________________________ 10.1. Users/Groups All user information is normally listed in the "/etc/passwd" file and the group information in the "/etc/groups" file. If you need to edit either file it is recommended that you use vipw to edit the password file and vigr to edit the group file. These particular commands take care of any processing and locking of the files before and after editing them. There is a lot of information about adding/removing/controlling users and groups, this information is only the minimal information required. chsh Used to change your login shell. To list the shells available type: chsh --list-shells Simply type chsh then [Enter], then type the name of the shell you would like to use every time you login. chfn Change finger information. The information this command changes is reflected in the /etc/passwd file, use this utility to update your real name, office and home phone numbers (if they exist). Use the -f option to change a user's full name. Use this tool as either chfn or chfn user_name (usable by root only). Command syntax: chfn user_name passwd Changes the password of a user. You will need to be root if you want to change other user's passwords. Simply type passwd to change your own password or to change another user's password type: passwd username _________________________________________________________________ Chapter 11. Text Related Tools The text related tools chapter is the largest in this guide, most of the time on a GNU/Linux machine you will spend time interacting with text. This chapter briefly covers text editors and goes into more depth on viewing text, using tools to manipulate text, finding text within files and changing text formats between windows based systems and GNU/Linux based systems. _________________________________________________________________ 11.1. Text Editors vi A traditional UNIX system text editor, should be on any UNIX system. It is fairly difficult to learn but quite powerful, it is also quite small and uses minimal amounts of resources. Note vim vim - vi improved. A newer version of the vulnerable vi editor. Many systems use vim rather than vi. emacs More than just a text editor. This text editor has a steep learning curve but is also very powerful, it is both advanced and quite large. Emacs can do anything, surf the internet, chat, play games and many other tasks. Others There are too many different text editors to list here. Have a look on the internet, either search for them you will find many of them at Sourceforge or Freshmeat. _________________________________________________________________ 11.2. Text Viewing Tools head With no options it shows the first ten lines of a text file. Use head -n x (where "x" is a number) to display the first x lines. Try head -F to use a continually updated version of head (if the file changes it will be reloaded and displayed), please note that using this option will run head is a continuous loop so you'll need to use CTRL-C to exit. For example: head -n 20 somelog.txt Will display the top 20 entries of the file "somelog.txt". tail With no options it shows the last ten lines of a file. Use tail -n x (where "x" is a number) to display the last x lines. Try tail -F to use a continually updated version of tail (if the file changes it will be reloaded and displayed), please note that using this option will run tail is a continuous loop so you'll need to use CTRL-C to exit. For example: tail -n 20 somelog.txt Will display the last 20 entries of the file "somelog.txt". less Views text, can scroll backwards and forwards. Has many different options which are all described in the manual page. When less is already running, use :n and :p (type a colon then the character) to move to the next and previous files (when there are multiple open files). Command syntax: less filename.txt Or using a tool (in this example cat): cat file.txt | less more Displays text, one page full at a time, more limited than less. In this case less is better than more. more filename.txt Or using a tool (is this example cat): cat file.txt | more cat Combines (concatenates) multiple documents into one document. Can be used on individual files as well. Some useful options: + -b --- number all non-blank lines + -n --- number all lines. Also try using nl to number lines (it can do more complex numbering), find it under under this section, Section 11.4 Example: cat filepart1 filepart2 filepart3 > wholefile.txt Would combine (concatenate) filepart1, filepart2 and filepart3 into the single file "wholefile.txt". tac Combines (concatenates) multiple documents into one document and outputs them in reverse. Can also be used on individual files. Notice that tac is cat written backwards. Example: tac filepart1 filepart2 filepart3 > wholefile.txt Would combine (concatenate) filepart1, filepart2 and filepart3 into the single file but have each of the files written in reverse. z* commands Many commands can be prefixed with a "z" to read/work within a gzip compressed file. Some examples are zcat, zless, zmore, zgrep, zcmp, zdiff. There are many utilities for working with text within compressed files without trying to manually de-compress them somewhere first...most begin with a "z". bz* commands There are also a few commands that prefixed with a "bz" to read/work within a file compressed with bzip2. The tools are bzcat, bzless, bzgrep _________________________________________________________________ 11.3. Text Information Tools wc Word count, count how many words you have in a text document. Can also be used to count the lines or bytes within the file. Use the options -w for words, -l for lines and -c for bytes. Or simply run wc with no options to get all three. Command syntax: wc -option file.txt style To run various readability tests on a particular text file. Will output scores on a number of different readability tests (with no options). Command syntax: style -options text_file Note Find style in the diction package This command is part of the diction package and does not appear to be used too often these days cmp Determines whether or not two files differ, works on any type of file. Very similar to diff only it compares on the binary level instead of just the text. diff Compares two text files and output a difference report (sometimes called a "diff") containing the text that differs between two files. Can be used to create a 'patch' file (which can be used by patch). Example: diff file1.txt file2.txt diff will output a '>' (followed by the line) for each line that isn't in the first file but is in the second file, and it will output a '<' (followed by the line) for each line that is in the first file but not in the second file. sdiff Instead of giving a difference report, it outputs the files in two columns, side by side, separated by spaces. diff3 Same as diff except for three files. comm Compares two files, line-by-line and prints lines that are unique to file1 (1st column), unique to file 2 (2nd column) and common to both files (3rd column). Use comm with the -1, -2, or -3 to suppress the printing of those particular lines. Simply run comm to have all three listed (ie. unique to files 1 and 2 and common to both). Command syntax: comm file1 file2 look To output a list of words in the system dictionary that begin with a given string -- this is useful for finding words that begin with a particular phrase or prefix. Give the string as an argument; it is not case sensitive. Command syntax: look string _________________________________________________________________ 11.4. Text manipulation tools Tip Also see Also see tac, and cat over in this section, Section 11.2, as they can perform text modification too sort Sorting text with no options the sort is alphabetical. Can be run on text files to sort them alphabetically (note it also concatenates files), can also be used with a pipe '|' to sort the output of a command. Use sort -r to reverse the sort output, use the -g option to sort 'numerically' (ie read the entire number, not just the first digit). Examples: cat shoppinglist.txt | sort The above command would run cat on the shopping list then sort the results and display them in alphabetical order. sort -r shoppinglist.txt The above command would run sort on a file and sort the file in reverse alphabetical order. Advanced sort commands: sort is a powerful utility, here are some of the more hard to learn (and lesser used) commands. Use the -t option to use a particular symbol as the separator then use the -k option to specify which column you would like to sort by, where column 1 is the first column before the separator. Also use the -g option if numeric sorting is not working correctly (without the -g option sort just looks at the first digit of the number). Here is a complex example: sort -t : -k 4 -k 1 -g /etc/passwd | more This will sort the "/etc/passwd" file, using the colon ':' as the separator. It will sort via the 4th column (GID section, in the file) and then sort within that sort using the first (name) if there are any ties. The -g is there so it sorts via full numbers, otherwise it will have 4000 before 50 (it will just look at the first digit...). join Will put two lines together assuming they share at least one common value on the relevant line. It won't print lines if they don't have a common value. Command syntax: join file1 file2 cut Prints selected parts of lines (of a text file), or, in other words, removes certain sections of lines. You may wish to remove things according to tabs or commas, or anything else you can think of... Options for cut: + -d --- allows you to specify another delimiter, for example ':' is often used with /etc/passwd: cut -d ':' (and probably some more options here) /etc/passwd + -f --- this option works with the text by columns, separated according to the delimiter. For example if your file had lines like "result,somethingelse,somethingelse" and you only wanted result you would use: cut -d ',' -f 1 /etc/passwd This would get you only the usernames in /etc/passwd + "," (commas) --- used to separate numbers, these allow you to cut particular columns. For example: cut -d ':' -f 1,7 /etc/passwd This would only show the username and the shell that each person is setup for in /etc/passwd. + "-" (hyphen) --- used to show from line x to line y, for example 1-4, (would be from lines 1 to line 4). cut -c 1-50 file1.txt This would cut (display) characters (columns) 1 to 50 of each line (and anything else on that line is ignored) + -x --- where x is a number to cut from line 1 to "x" and use x- (where x is a number) to cut from "x" to the end. cut -5, 20-, 8 file2.txt This would display ("cut") characters (columns) 1 to 5, 8 and from 20 to the end. ispell/aspell To spell check a file interactively, prompts for you to replace word or continue. aspell is said to be better at suggesting replacement words, but its probably best to find out for yourself. aspell example: aspell -c FILE.txt This will run aspell on a particular file called "FILE.txt", apsell will run interactively and prompt for user input. ispell example: ispell FILE.txt This will run ispell on a particular file called "FILE.txt" ispell will run interactively and prompt for user input. chcase Is used to change the uppercase letters in a file name to lowercase (or vice versa). You could also use tr to do the same thing... cat fileName.txt | tr [A-Z] [a-z] > newFileName.txt The above would convert uppercase to lowercase using the the file "fileName.txt" as input and outputting the results to "newFileName.txt". cat fileName.txt | tr [a-z] [A-Z] > newFileName.txt The above would convert lowercase to uppercase using the the file "fileName.txt" as input and outputting the results to "newFileName.txt". chcase (a perl script) can be found at the chcase homepage. fmt (format) a simple text formatter. Use fmt with the -u option to output text with "uniform spacing", where the space between words is reduced to one space character and the space between sentences is reduced to two space characters. Example: fmt -u myessay.txt Will make sure the amount of space between sentences is two spaces and the amount of space between words is one space. paste Puts lines from two files together, either lines of each file side by side (normally separated by a tab-stop but you can have any symbols(s) you like...) or it can have words from each file (the first file then the second file) side by side. To obtain a list of words side by side, the first word from the first file on the left side separated by a tab-stop then the first word from the second file you would type: paste file1.txt file2.txt To have the list displayed in serial, first word from first file, [Tab], second word from first file, then third and fourth until the end of the first file type: paste --serial file1.txt file2.txt expand Will convert tabs to spaces and output it. Use the option -t num to specify the size of a "tapstop", the number of characters between each tab. Command syntax: expand file_name.txt unexpand Will convert spaces to tabs and output it. Command syntax: unexpand file_name.txt uniq Eliminates duplicate entries from a file and it sometimes greatly simplifies the display. uniq options: + -c --- count the number of occurances of each duplicate + -u --- list only unique entries + -d --- list only duplicate entries For example: uniq -cd phone_list.txt This would display any duplicate entries only and a count of the number of times that entry has appeared. tr (translation). A filter useful to replace all instances of characters in a text file or "squeeze" the white space. Example: cat some_file | tr 3 5 > new_file This will run the cat program on some file, the output of this command will be sent to the tr command, tr will replace all the instances of 3 with 5, like a search and replace. You can also do other things such as: cat some_file | tr [A-Z] [a-z] > new_file This will run cat on some_file and convert any capital letters to lowercase letters (you could use this to change the case of file names too...). Tip Alternatives You can also do a search and replace with a one line Perl command, read about it at the end of this section. nl The number lines tool, it's default action is to write it's input (either the file names given as an argument, or the standard input) to the standard output. Line numbers are added to every line and the text is indented. This command can do take some more advanced numbering options, simply read the info page on it. These advanced options mainly relate to customisation of the numbering, including different forms of separation for sections/pages/footers etc. Also try cat -n (number all lines) or cat -b (number all non-blank lines). For more info on cat check under this section: Section 11.2 There are two ways you can use nl: nl some_text_file.txt The above command would add numbers to each line of some_text_file. You could use nl to number the output of something as shown in the example below; grep some_string some_file | nl Perl search and replace text To search and replace text in a file is to use the following one-line Perl command[4]: $ perl -pi -e "s/oldstring/newstring/g;" filespec [RET] In this example, oldstring is the string to search, newstring is the string to replace it with, and filespec is the name of the file or files to work on. You can use this for more than one file. Example: To replace the string `helpless' with the string `helpful' in all files in the current directory, type: $ perl -pi -e "s/helpless/helpful/g;" * [RET] Also try using tr to do the same thing (see further above in this section). Tip If these tools are too primitive If these text tools are too simple for your purposes then you are probably looking at doing some programming or scripting. If you would like more information on bash scripting then please see the advanced bash scripting guide, authored by Mendel Cooper. sed and awk are traditional UNIX system tools for working with text, this guide does not provide an explanation of them. sed works on a line-by-line basis performing substitution and awk can perform a similar task or assist by working on a file and printing out certain information (its a programming language). You will normally find them installed on your GNU/Linux system and will find many tutorials all over the internet, feel free to look them up if you ever have to perform many similar operations on a text file. _________________________________________________________________ 11.5. Text Conversion/Filter Tools Filters (UNIX System/dos formats) The following filters allow you to change text from Dos-style to UNIX system style and vice-versa, or convert a file to other formats. Also note that many modern text editors can do this for you... Why use filters? Because UNIX systems and Microsoft use two different standards to represent the end-of-line in an ASCII text file. This can sometimes causes problems in editors or viewers which aren't familiar with the other operating systems end-of-line style. The following tools allow you to get around this difference. Whats the difference? The difference is very simple, on a Windows text file, a newline is signalled by a carriage return followed by a newline, '\r\n' in ASCII. On a UNIX system a newline is simply a newline, '\n' in ASCII. dos2unix This converts Microsoft-style end-of-line characters to UNIX system style end of line characters. Simply type: dos2unix file.txt fromdos This does the same as dos2unix (above). Simply type: fromdos file.txt fromdos can be obtained from the from/to dos website. unix2dos This converts UNIX system style end of line characters to Microsoft-sty le end-of-line characters. Simply type: unix2dos file.txt todos This does the same as unix2dos (above). Simply type: todos file.txt todos can be obtained from the from/to dos website. antiword This filter converts Microsoft word documents into plain ASCII text documents. Simply type: antiword file.doc You can get antiword from the antiword homepage. recode Converts text files between various formats including HTML and dozens of different forms of text encodings. Use recode -l for a full listing. It can also be used to convert text to and from Windows and UNIX system formats (so you don't get the weird symbols). Caution Warning By default recode overwrites the input file, use '<' to use recode as a filter only (and to not overwrite the file). Examples: UNIX system text to Windows text: recode ..pc file_name Windows text to UNIX system text: recode ..pc/ file_name UNIX system Text to Windows Text without overwriting the original file (and creating a new output file): recode ..pc < file_name > recoded_file tr (Windows to UNIX system style conversion only). While tr is not specifically designed to convert files from Windows-format to UNIX system format by doing: tr -d '\r' < inputFile.txt > outputFile.txt The -d switch means to simply delete any occurances of the string. Since we are looking for '\r', carriage returns it will remove any it finds, making the file a UNIX system text file... _________________________________________________________________ 11.5.1. Conversion tools enscript Converts text files to postscript, rtf, HTML (use ghostview to view the postscript file). enscript has a large number of options which can be used to customise the output. Examples:[5] enscript --language=html input_file.txt -o output_file.html This will take some file and output it as a html file. enscript --help-highlight Display help on using the highlight feature (list all different types of highlighting available) -E[lang] Highlight using the lang (pretty print), example: enscript -E --color --language=html --toc -pfoo.html *.h *.c Add all the files with a .h and a .c (C source and header files) into a file called foo.html, use colour and add a table of contents For further options refer to the well written manual page of enscript. figlet Used to create ASCII "art". Figlet can create several different forms (fonts) of ASCII art, its one of the more unusual programs around. _________________________________________________________________ 11.6. Finding Text Within Files grep Looks for text within files. For example: grep this_word this_file.txt Example options: + -v --- this option is used to display lines which do not contain the string. + -n --- this option displays the line numbers + -w --- this option makes grep match the whole word + -A x or -B x (where x is a number) --- display "x" lines After or Before the section where the particular word is found. + -r or rgrep --- search for text within files recursively. This command uses regular expressions, for more information please see, Section 20.4.2. For example, this command would look in the file "rpmlist.txt" for anything starting with "rpm": grep rpm rpmlist.txt Or you could use it like this, to search through the output of another file: rpm -qa | grep ogg The first command lists all RPM's installed on your system, the second finds any containing the string "ogg" and outputs them. rgrep A "recursive" version of grep (this is a different program to grep). This will search all the files in the current directory and all it's subdirectories and print the names of the files and the matching line. Follows similar syntax to grep (see above). You could also use grep with the -r option to achieve the same affect. fgrep This version of grep calls grep with the -F option. This will look for literal strings only, it won't use or expand any kind of regular expression. For example you could type: fgrep a$*b? file.txt And fgrep would look for the string "a$*b?" in the file "file.txt". Tip Other Versions There are various versions of grep which are designed to do different things try searching for them on the internet or within your distribution. _________________________________________________________________ Chapter 12. Mathematical tools Tip num-utils homepage The "num-utils" homepage, Num Utils, contains a variety of command line programs that could be useful when performing maths on your GNU/Linux machine. units Convert units of measurement between different scales. For example, centimeters to inches, litres to gallons. Simply run the program, I recommend running it as foll