Database Reference
In-Depth Information
Replacing and Deleting Values
You can use the command-line tool
tr
, which stands for
translate
, to replace individ‐
ual characters. For example, spaces can be replaced by underscores as follows:
$
echo
'hello world!'
| tr
' '
'_'
hello_world!
If more than one character needs to be replaced, then you can combine that:
$
echo
'hello world!'
| tr
' !'
'_?'
hello_world?
tr
can also be used to delete individual characters by specifying the
-d
option:
$
echo
'hello world!'
| tr -d -c
'[a-z]'
helloworld
Here, we've actually used two more features. First, we've specified a set of characters
(all lowercase letters). Second, we've indicated with the
-c
option that the comple‐
ment should be used. In other words, this command only retains lowercase letters.
We can even use
tr
to convert our text to uppercase:
$
echo
'hello world!'
| tr
'[a-z]'
'[A-Z]'
HELLO WORLD!
$
echo
'hello world!'
| tr
'[:lower:]'
'[:upper:]'
HELLO WORLD!
The latter command is preferable because that also handles non-ASCII characters. If
you need to operate on more than individual characters, then you may find
sed
use‐
ful. We've already seen an example of
sed
with extracting the chapter headings from
Alice in Wonderland
. Extracting, deleting, and replacing is actually all the same opera‐
tion in
sed
. You just specify different regular expressions. For example, to change a
word, remove repeated spaces, and remove leading spaces:
$
echo
' hello world!'
| sed -re
's/hello/bye/;s/\s+/ /g;s/\s+//'
bye world!
The
g
flag stands for
global
, meaning that the same part can be applied more than
once on the same line. We do not need that with the second part, which removes
leading spaces. Note that regular expressions of the first and the last parts could have
been combined into one regular expression.
Working with CSV
Bodies and Headers and Columns, Oh My!
The command-line tools that we've used to scrub plain text, such as
tr
and
grep
, can‐
not always be applied to CSV. The reason is that these command-line tools have no
notion of headers, bodies, and columns. What if we wanted to filter lines using
grep
