I love shell pipelines, and flatter myself that I'm pretty good at them. But a discussion last week on the Linuxchix Techtalk mailing list on finding added lines in a file turned up a terrific bash/zsh shell redirection trick I'd never seen before:
join -v 2 <(sort A.txt) <(sort B.txt)
I've used backquotes, and their cognate $(), plenty. For instance,
you can do things like
to set your prompt to the current hostname: the shell runs the
hostname command, takes its output, and substitutes that output in place
of the backquoted or parenthesized expression.
But I'd never seen that <(...) trick before, and immediately saw how useful it was. Backquotes or $() let you replace arguments to a command with a program's output -- they're great for generating short strings for programs that take all their arguments on the command line. But they're no good for programs that need to read a file, or several files. <(...) lets you take the output of a command and pass it to a program as though it was the contents of a file. And if you can do it more than once in the same command -- as in Little Girl's example -- that could be tremendously useful.
Playing with it to see if it really did what it looked like it did, and what other useful things I could do with it, I tried this (and it worked just fine):
$ diff <(echo hello; echo there) <(echo hello; echo world) 2c2 < there --- > worldIt acts as though I had two files, which each have "hello" as their first line; but one has "there" as the second line, while the other has "world". And diff shows the difference. I don't think there's any way of doing anything like that with backquotes; you'd need to use temp files.
Of course, I wanted to read more about it -- how have I gone all these years without knowing about this? -- and it looks like I'm not the only one who didn't know about it. In fact, none of the pages I found on shell pipeline tricks even mentioned it.
It turns out it's called "process substitution" and I found it documented in Chapter 23 of the Advanced Bash-Scripting Guide.
I tweeted it, and a friend who is a zsh master gave me some
similar cool tricks. For instance, in zsh
echo hi > >(cat) > >(cat -n) lets you pipe
the output of a command to more than one other command.
That's zsh, but in bash (or zsh too, of course), you can use >() and
tee to do the same thing:
echo hi | tee >(cat) | cat -n
If you want a temp file to be created automatically, one you can both read and write, you can use =(foo) (zsh only?)
Great stuff! Some other pages that discuss some of these tricks:
[ 18:23 Aug 24, 2013 More linux/cmdline | permalink to this entry | comments ]