bash: pipe error: Too many open files in system

So I came across the following error the other day while trying to run the find command on my Mac:

bash: pipe error: Too many open files in system  

I also then went into one of those 'must close all the things' scenarios where all the applications started screwing up and I in the end resorted to rebooting - something I have hardly ever had to do since starting to use the Mac.

Long story short, it ended up being a bad update to BetterTouchTool (BTT) which I use to enable keyboard commands to move windows to using the left or right half of a screen, between monitors and the like. The author of BTT had put out a bad update, and as I had auto-updates turned on I got some bad code which didn't crash the application, but after further diagnosis was opening handles and leaving them open whilst opening new handles and so forth - after ~7000 handles in use by the one application the system ran out (total limit I think is 10,000) and crashes commenced with catastrophic consequences for all running applications.

Finding out which app was causing the issue turns out not to be as obvious as you might think. Activity Monitor, while an excellent system application does not actually show this info in an easy to digest (or compare) method - you can double-click an application in one of its views and see which files are open for that application, but you cannot easily show which is the application using the most file handles.

A quick search on Google pointed me at either lsof (a command-line tool accessed via bash) and a pretty neat looking application called Sloth. Sloth is also a fine application - but it also does not give a nice clear listing of just the number of handles per application.

The solution in my case was to use lsof, combine it with awk, uniq, sort and head and get a nice clear listing of the top applications hogging all the handles.

The command is nice and simple right? :)

lsof | awk '{print $1}' | uniq -c | sort -rn | head  

This generates the following output:

user@desktop ~/AllTheFiles  
$lsof | awk '{print $1}' | uniq -c | sort -rn | head
3271 com.apple  
2978 Google  
 914 Atom\x20H
 505 Skype
 476 Mail
 375 Atom\x20H
 304 Finder
 292 Dock
 277 Atom\x20H
 270 Atom\x20H

com.apple in the above example is actually Safari - Google is the reference to Chrome.

It would be entirely possible to automate polling for an application going above 6000 handles and terminating the process (hidden behind the output is the pid for each process) - I didn't go that far myself, and thankfully the BTT author updated the app within a couple of days so I'm back to normal.