Jay Taylor | programmer notes

TAG | python

So, I got this weird exception when I ran the command:

python script_that_prints_output.py | head
..[some output, in fact, just the amount that head wanted(!)]..
Traceback (most recent call last):
File "./script_that_prints_output.py", line 86, in
print out
IOError: [Errno 32] Broken pipe

I was not sure why this was happening, because when I ran the command without piping it to “head”, it ran just fine, displaying the expected output with no exceptions or errors raised.

I found the explanation for this behavior, and thought it was interesting enough to warrant a post (that way if I encounter it again at some point in the future, the odds that I’ll remember are probably greater ;):

> IOError: [Errno 32] Broken pipe
> Anyone know why this is happening?

That’s normal, at least with Unix. When the program on the receiving end
of a pipe decides to close its end for some reason, Unix sends the signal
‘SIGPIPE’ to the sending end. Python catches this and turns it into an
IOError exception.
The only way around this (that I can think of) is to
catch the exception and exit the program gracefully. If you try to send
more data, you will get more IOErrors, since your program has nowhere left
to send data.

I actually thought of anohter way around it, albeit not efficient in any way (the whole point of the SIGPIPE signal is to stop the program from completing, since no further output is required from it).  This is definitely a rather ugly hack, but sometimes that is preferable to adding the exception handling, I guess.

python script_that_prints_output.py > tmpfile ; head tmpfile; rm tmpfile

Update 2009-12-17:  Useful information about broken pipes in Python at StackOverflow

, , , , , Hide

I first looked at Python’s dom and minidom modules last Friday.  There was an initial wave of shock for me after looking over the apparently complex API.  Eventually I remembered that all I wanted to do was create a straightforward RSS feed document from a MySQL database.   I had to go to class so I didn’t make much progress on that attempt.

Today I revisited the project and randomly happened upon a 3rd party Python XML library called ‘lxml‘.  This module is purported to be an order of magnitude faster than Python’s dom/minidom (rendering them quite slow, painfully slow, even, by comparison).  Not that execution speed is a particularly important to me in this instance.  Anyways, after 4 minutes to compile the module from their Subversion repository (my server is slow, I guess..), 2 minutes of the tutorial, and approximately 10 minutes coding, I had the script doing what I had originally set out to do.  The lxml API seems much more practical, at least in this case where the task and goal were pretty simple (naturally, I can’t account for other scenarios).  At any rate, I guess this time around its “Vive lxml”.

At this point, the Python dom and minidom modules do have a feeling of circumstantial familiarity.  They remind me of the my days of having to troubleshoot and develop fixes to address protocol incompatibilities between the Java SOAP API implementation with PHP Pear SOAP’s API.  Shudder.

This is the source that pointed me to lxml.

, , Hide

Find it!