/dev/stdin? What are they useful for?
amongst the disks, TTYs,
urandoms, and other pseudo-devices,
are the files
What are they, and what are they good for?
You may have heard of standard in, standard out and standard error,
the three “standard streams” which each UNIX process usually has attached to it on start.
cat run with no arguments
reads from standard in and writes to standard out (achieving nothing):
$ echo hello | cat | tr '[a-z]' '[A-Z]' HELLO
The standard streams don’t seem like “devices”, but there they are in
files which any process can open, read from, and write to, just like other files.
When I write to
/dev/stdout from my shell, it’s printed to my terminal:
$ echo hello > /dev/stdout hello
/dev/stdout means different things to different processes.
/dev/stdout refers to the standard out of my shell process,
which is my terminal,
and that’s why
hello printed to my screen.
But if I tell
cp to write to
it writes to its own standard out stream,
not to my shell’s standard out:
$ echo hello | cp /dev/stdin /dev/stdout | tr '[a-z]' '[A-Z]' HELLO
Look at that:
normally used for copying files,
to behave just like
cat, which copies streams!
We were able to do this by telling
cp to use its standard streams instead of files.
When building shell pipelines, this can be very useful. The following pipeline grabs a JPEG from the web, converts it to a monochrome PNG, then posts the result to another web server, without creating any temporary files:
curl https://upload.wikimedia.org/wikipedia/en/5/52/Testcard_F.jpg \ | convert -monochrome /dev/stdin png:/dev/stdout \ | curl -X POST --data @/dev/stdin -H 'Content-Type: image/png' http://requestbin.fullcontact.com/13rcezq1
Processes interact with files via file descriptors.
When a process opens a file,
the OS gives the process a file descriptor (a number)
which the process can use to read or write from the open file.
We can see this with
$ cat foo.txt hello $ strace cat foo.txt ... open("foo.txt", O_RDONLY) = 3 ... read(3, "hello\n", 65536) = 6 ...
cat foo.txt called
and the OS returned the file descriptor
cat then read from this with
The standard streams in, out and error are at file descriptors 0, 1 and 2 respectively.
The standard streams are pre-attached; the process does not have to open them.
cat run with no arguments
reads from file descriptor
0 and writes to file descriptor
In a shell pipeline, these standard file descriptors are references to anonymous pipes.
When we run
cp /dev/stdin /dev/stdout,
the process opens the files
/dev/stdout as normal,
and gets back file descriptors for them.
Which numbers do you think it gets back?
open("/dev/stdin", O_RDONLY) = 3 open("/dev/stdout", O_WRONLY|O_TRUNC) = 4
No, it doesn’t get back 0 and 1; it gets two new file descriptors 3 and 4. But those new file descriptors both refer to the same underlying anonymous pipe.
More by Jim
- The inception bar: a new phishing method
- The hacker hype cycle
- Project C-43: the lost origins of asymmetric crypto
- How Hacker News stays interesting
- My parents are Flat-Earthers
- The dots do matter: how to scam a Gmail user
- The sorry state of OpenSSL usability
- I hate telephones
- The Three Ts of Time, Thought and Typing: measuring cost on the web
- Granddad died today
- Your syntax highlighter is wrong
Tagged . All content copyright James Fisher 2018. This post is not associated with my employer. Found an error? Edit this page.