Quick reference guide Scientific Linux for DESY

Previous Contents Index Next 

9. File manipulation

awk [-f progfile] [-Fsep] ['program'] [file[s]] F
Pattern scanning and processing language. Process file[s] or STDIN according to program (or progfile) (setting the record separator to sep). Useful to extract information from tabular data, e.g. count number of shells in use:
Example: awk -F: '{print 7}' /etc/passwd | sort | uniq -c
Documentation
bunzip2 [file[s]] F
Decompresses all file[s]. Same as bzip2 -d.  
bzcat [file[s]] F
Decompress file[s] to STDOUT. Same as bzip2 -cd.  
bzip2 [-hcdzl9] [file[s]] F
Compress or expand file[s]. See a complete list of options with -h
-c write on standard output
-d force decompression of file[s]
-z force compression of file[s]
-l list compressed file contents
-9 set maximum compression block size
 
cat [-entv] [file[s]] F
Concatenate file[s] and print on the standard output. Display line numbers with -n, line ends with -E, tabs with -T, other nonprinting chars with -v. Documentation
cmp [-l] [-s] file1 file2 f
Compares two (binary) files. Display the first (or with -l all) difference(s). Documentation
cpio -p [-adlmuvLV] [-R id] directory f
Copy files to and from archives. Use normally the more flexible tar command. Useful to copy directory hierarchies from current to new dir:
find . -depth -print | cpio -pdlmv new
 
csplit [options] file pattern[s]
Split a file into sections determined by context lines. See also split. Documentation
cut [-b | -c | -f] list [-d sep] [file[s]] F
Remove sections from each line of file[s]. Cut out columns specified by a comma separated list of ranges n1[-n2], where n is given as byte count (-b), character count (-c) or field number (-f). Fields are separated by TAB or by character sep. Documentation
diff [-biw] [-ceu] [-r] name1 name2
Compare the contents of files name1 and name2, the contents of file name1 and the same file in directory name2 or all files in directories name1 and name2. Use cmp to compare binary files. The more common options are:
-b ignore trailing blanks, treat strings of blanks as equivalent
-i ignore the case of letters
-w ignore whitespace in comparisons
-c produce context diffs
-e produce a script for ed that creates name2 from name1
-u produce unified context diffs
-r descend recursively directories
Documentation
grep [-E|-F] [-chilnqvx] [-e]pattern [-fpatternfile] [--color] [file[s]] F
egrep [-chilnqvx] [-e]pattern [-fpatternfile] [--color] [file[s]] F
fgrep [-chilnqvx] [-e]pattern [-fpatternfile] [--color] [file[s]] F
Print lines matching a pattern (optionally from patternfile). Interpret pattern as regular or (with egrep and grep -E) extended regular expressions or (with fgrep and grep -F) as fixed strings . Print the matched strings in color with --color.
-c print a count of matching lines only
-h do not print file names, if used with multiple files
-i ignore the case of letters
-l print only file names with matching lines
-n precede each output line by its relative line number in the file
-q do not produce output, return with exit status 0 if match was found
-v print only lines that do not match
-x print only lines that exactly match the whole line
 
gpg [-c] file
Encrypt (-c) or decrypt a file using the OpenPGP standard.  
groff [-mmacro] [-nnum] [-orange] [-Tdev] [file[s]] F
Format documents for printer (default) or display (-Tascii) using a macro package (usually an or andoc). Output only pages in range if -o given.  
gunzip [-clrtv] [-S suffix] [file[s]] F
Expand files. Equivalent to gzip -d  
gzip [-cdlrtv9] [-S suffix] [file[s]] F
Compress or expand file[s]. See a complete list of options with -h
-c write on standard output, keep original files unchanged
-d decompress (expand) file[s]
-l list compressed file contents
-r operate recursively on directories
-S use suffix on compressed files
-v display file name and compression rate for each file
-9 use best (slowest) compression algorithm
Documentation
head [-n] [-n|-c n] [file[s]] F
Output the first n lines (default: 10) or the first n bytes (-c) of file[s] Documentation
iconv [-f encoding] [-t encoding] [-o file] [inputfile[s]] F
iconv --list  
{} Convert the encoding of given inputfile[s] from one encoding (-f) to another (-t), optionally write output to a file. The list of encodings is displayed using --list.  
md5sum [-c] file[s] F
Compute and check MD5 (128 bit) message digest. The output can be used to check (-c) whether the file integrity is still ok.
Trivial test: md5sum file | md5sum -c
Documentation
nl [options] [file[s]] F
Write file[s] to standard output, with line numbers added. More complex numbering schemes are possible than using cat -n. Documentation
paste [-d separator[s]] [file[s]] F
Merge lines of files line by line into single lines using TAB or separator as delimiter, output goes to STDOUT. Documentation
sed [-n] [-i[suffix]] [-e script] [-f scriptfile] [file[s]] F
Stream editor. Copies file[s] or STDIN to STDOUT by modifying the input according to script. Default output is suppressed with -n.
To replace all occurrences of old in file by new in newfile use
sed -e 's/old/new/g' file > newfile
Can also be done in place using the -i option:
sed -i -e 's/old/new/g' file or even with perl:
perl -i -pe 's/old/new/g' file
 
sort [-cmu] [-t sep] [-bdfinr] [-k n1[,n2]] [file[s]] F
Sort, merge (-m) or sequence check (-c) text file[s]. The basic options are
-b ignore leading blanks in each line
-d consider only blanks and alphanumeric characters in keys
-f sort case insensitively
-i ignore non printing characters in sorting
-n treat field elements to be sorted as numbers
-r reverse the sorting direction
-t sep Use sep as field delimiter
-u output only the first of unique lines
The sort fields can be given with -k. In the most general case it has the syntax
start_field[.first_char][bdfgiMnr][,end_field[.last_char][bdfgiMnr]]
Documentation
split [-lines] [-l lines] [-b bytes[km]] [file [prefix]] f
Split a file into pieces. Store a number of lines (default 1000) or bytes (where k is 1024 and m is 1048576) in each outputfile (named prefixaa,prefixab,..., default xaa,xab,...). Documentation
strings [-a] [-n] [-n n] [file[s]] f
Print the strings of printable characters in binary file[s]. With -a look in all sections of executables for strings. Minimum string length is 4 (or n). Documentation
tail [-f] [-n] [-n|-c n] [file[s]] F
Output the last n lines (default: 10) or the last n bytes(-c) of file[s]. Wait for further records from the input file, if -f given. Documentation
tar [-]mode[options] [archive] [file[s]] F
Archiving utility. The modes (for a complete list refer to the man page)
-c create a new archive
-r append files to the end of an archive
-t list the contents of an archive
-u only append files newer than copy in archive
-x extract files from an archive
can be combined with options
-N date only store files newer than date
-O extract files to standard output
-X file exclude files listed in file
-f [host:]file use archive file (on host)
-j filter the archive through bzip2 (usually autodetected)
-l stay in local file system when creating archive
-p extract all protection information of files
-v verbosely list files processed
-z filter the archive through gzip (usually autodetected)
For even more options see tar --help. The remote access (-f host:file) works only if host based authentication is enabled.
Documentation
tee [-ai] [file[s]] F
Pass STDIN to STDOUT and make a copy to file[s] (append with -a) Documentation
tkdiff [-h] file1 file2 P
Compare and edit two files using a GUI. See a short help with -h.  
tr [-csd] [string1 [string2]] F
Translate or delete chars. Chars in string1 are mapped to chars in string2 or deleted (-d) or repeated chars found in string2 are squeezed to single chars (-s). With -c all chars not in string1 are used instead. For string1 and string2 a range of chars can be given. To convert all chars in file to uppercase chars in newfile use tr a-z A-Z < file > newfile Documentation
uniq [-cdu] [-f fields] [-s chars] [infile [outfile]] F
Remove duplicate lines from a sorted file (by ignoring the first fields or the first chars in comparisons). Only print unique lines (-u), duplicate lines (-d) or precede the output lines with a repetition count (-c). Documentation
unzip [-opts[modifiers]] file[.zip] [list] [-x xlist] [-d exdir]
List, test and extract compressed files in a ZIP archive. See unzip -h or man unzip for details. Provided for interoperability with DOS.  
vimdiff file1 file2 [file3]
Edit two or three versions of a file with vim and show differences. There is also gvimdiff based on the GUI version of vim.  
wc [-clw] [file[s]] f
Print the number of bytes (-c), words (-w), and lines (-l) in file[s] Documentation
xargs [-pt] [-e[eofstr]] [-i[repstr]] [cmd [args]] F
Build and execute command lines from STDIN. Each input line is appended to cmd [args] and then executed. More efficient than find -exec.
-t list (trace) executed commands
-p prompt before each command
-i [repstr] replace each occurrence of repstr (default {}) with full input line
For more options see the man page. Move .log files to .logold files with
ls *.log | xargs -ti mv {} {}old
Documentation
zcat [file[s]] F
Uncompress file[s] (or STDIN) to STDOUT. Is the same as gunzip -c.  
zip [-options] [-b path] [-t mmddyy] [-n suffixes] [zipfile list] [-xi list] PF
Package and compress (archive) files in a ZIP archive. See zip -h or man zip for details. Provided for interoperability with DOS.  

Previous Contents Index Next