Non-blocking command-line I/O in Ruby using nio4r and Open3
If you're writing an integration with external tooling using command line, you probably want to have control over how long the CLI command is going to run. You can of course do like some do - spawn another thread and keep a counter there while monkey-patching a core Ruby class in unrelated gem because why the hell not...
You can also use included Ruby library called Timeout - that is of course if you're not worried about crashes or stopping your code in the middle of doing something else.
And of course you can use asynchronous non-blocking evented I/O. If you know how to do it, obviously.
While experimenting with implementing my own library for communication with ffmpeg I was looking for a solution that didn't involve either Timeout module or spawning another Ruby thread for no good reason. I came across nio4r - nonblocking low-level I/O gem based on libev / Java NIO that is being used as a building block for larger software packages like Celluloid or ActionCable. Imagine my surprise when I couldn't find anything - really, anything! - about how to use this library to do something other than listening for incoming TCP connections.
Something like monitoring I/O of a spawned process for example, with the ability to timeout it if it doesn't respond in time while at the same time keeping the amount of threads to minimum?
require 'nio' require 'open3' selector = NIO::Selector.new stdin, stdout, stderr, thread = Open3.popen3("ffmpeg -i file.mp4 file.mp3") monitor_stdout = selector.register(stdout, :r) monitor_stderr = selector.register(stderr, :r) monitor_stdout.value = proc { puts "Got some data: #{monitor_stdout.io.read_nonblock(4096)}" } monitor_stderr.value = proc { puts "Got some error: #{monitor_stderr.io.read_nonblock(4096)}" } timeout = 30 # seconds loop do begin ready = selector.select(timeout) raise 'Command timeout' if ready.nil? ready.each { |m| m.value.call } rescue EOFError break end end
There, isn't that better?
There is one important caveat to consider while using this approach. While using synchronous read/write with Open3, your callback / block of code will yield on every read line. Unfortunately this approach does not wait patiently for the IO to pipe anything but rather uses your system polling mechanism (epoll/kqueue/inotify etc.). This means if the CLI tool that you're using is not playing it nice and doesn't buffer the output (ffmpeg for example doesn't) then your callback my be yielded with partial chunk of data rather than something more tangible, like a single line of output. Be wary and test often.
If you need more than just a code example to get you going, consider reading excellent post by Tony Arcieri, A gentle introduction to nio4r: low-level portable asynchronous I/O for Ruby.
Hope I could help!