python - redirecting shell output using subprocess -

i have python script calls lot of shell functions. script can run interactively terminal, in case i'd display output right away, or called crontab, in case i'd email error output.

i wrote helper function calling shell functions:

import subprocess import shlex import sys  def shell(cmdline, interactive=true):     args = shlex.split(cmdline.encode("ascii"))     proc = subprocess.popen(args, stdout=subprocess.pipe,         stderr=subprocess.pipe)     val = proc.communicate()     if interactive true:         if proc.returncode:             print "returncode " + str(proc.returncode)             print val[1]             sys.exit(1)         else:             print val[0]     else:         if proc.returncode:             print ""             # send email val[0] + val[1]  if __name__ == "__main__":     # example of command produces non-zero returncode     shell("ls -z") 

the problem i'm having two-fold.

1) in interactive mode, when shell command takes while finish (e.g. few minutes), don't see until command done since communicate() buffers output. there way display output comes in, , avoid buffering? need way check returncode, why i'm using communicate().

2) shell commands call can produce lot of output (e.g. 2mb). documentation communicate() says "do not use method if data size large or unlimited." know how large "large"?

1) when use communicate, capture output of subprocess nothing sent standard output. reason why see output when subprocess finished because print yourself.

since want either see runs , not capture or capture , @ end, can change way works in interactive mode leaving stdout , stderr none. makes subprocess use same streams program. you'll have replace call communicate call wait:

if interactive true:     proc = subprocess.popen(args)     proc.wait()     if proc.returncode:         print "returncode " + str(proc.returncode)         sys.exit(1) else:     proc = subprocess.popen(args, stdout=subprocess.pipe,         stderr=subprocess.pipe)     val = proc.communicate()     if proc.returncode:         print ""         # send email val[0] + val[1] 

2) large "too large store in memory", depends on lot of factors. if storing temporarily 2mb of data in memory fine in situation, there's nothing worry about.