Groovy: Line-by-line process output - stdout and stderr

It is very easy to start up command line processes with Groovy.   Just use the execute() method on a string, which returns a Java Process object:

def proc = "./some-shell-script.sh".execute()

But what if we want to capture the stdout and stderr of the process?  Turns out this is also very easy:

def proc = "./some-shell-script.sh".execute()
def rc = proc.waitFor()
def output = proc.text

This is all well and good, but like anything it has a few limitations.   The output is buffered and not available until the process completes.   That makes it not as useful for say, logging to the console in real time as you might want to do in a Jenkins job.   Also, it might be handy to prefix each line of output as it comes from the process with a timestamp or something like that.   The good news is that Groovy comes with some things that can be used for this purpose.

Groovy extends the Process object with some interesting methods that make it easy to handle stdout and stderr very easily.   We can also use the ANT LineOrientedOutputStream to give us line-by-line behavior:

  class LineOutput extends LineOrientedOutputStream
  {
    String prefix
    List lines

    @Override
    protected void processLine(String line) throws IOException
    {
      lines.add(line)
      println "${new Date().format('yyyy-MM-dd HH:mm:ss.SSS')} ${prefix} : ${line}"
    }
  }

We can start threads for stdout and stderr on the process like this:

def outLines = []
def errorLines = []
def proc = "./some-shell-script.sh".execute()
def outThread = proc.consumeProcessOutputStream(new LineOutput(prefix: "out", lines: outLines))
def errThread = proc.consumeProcessErrorStream(new LineOutput(prefix: "err", lines: errLines))

The threads will automatically start, and begin recording and echoing every line as soon as it happens. The we can wait for the process to terminate and clean up:

try { outThread.join(); } catch (InterruptedException ignore) {}
try { errThread.join(); } catch (InterruptedException ignore) {}
try { proc.waitFor(); } catch (InterruptedException ignore) {}

All of this can be combined into a class for convenient access, adding some configuration options to make it more useful:

class ShellCommand
{
  private final String cmd
  private final boolean echo
  private final Process proc
  private final Thread outThread
  private final Thread errThread
  private final List outLines = []
  private final List errLines = []

  private class LineOutput extends LineOrientedOutputStream
  {
    boolean echo
    String prefix
    List lines

    @Override
    protected void processLine(String line) throws IOException
    {
      lines.add(line)
      if (echo)
        println "${new Date().format('yyyy-MM-dd HH:mm:ss.SSS')} ${prefix} : ${line}"
    }
  }


  ShellCommand(String cmd, boolean echo = false,String outPrefix = "stdout", String errPrefix = "stderr")
  {
    this.cmd = cmd
    this.echo = echo
    // Start the process.
    this.proc = cmd.execute()
    // Start the stdout, stderr spooler threads
    outThread = proc.consumeProcessOutputStream(new LineOutput(echo: echo, prefix: outPrefix, lines: outLines))
    errThread = proc.consumeProcessErrorStream(new LineOutput(echo: echo, prefix: errPrefix, lines: errLines))
  }

  def waitForOrKill(int millis)
  {
    proc.waitForOrKill(millis)
    _done()
  }

  private void _done()
  {
    try { outThread.join(); } catch (InterruptedException ignore) {}
    try { errThread.join(); } catch (InterruptedException ignore) {}
    try { proc.waitFor(); } catch (InterruptedException ignore) {}
    proc.closeStreams()
  }

  def getRc()
  {
    def rc = null
    try { rc = proc.exitValue() } catch (IllegalThreadStateException e) {}
    return rc
  }
}

This also adds the waitForOrKill(millis) method, which is useful when running shell commands that are expected to complete in a reasonable amount of time (or something is wrong).