Twisted] 10. Thread and Subprocess
Twisted doesn't make the code automatically non-asynchronous or non-blocking code. Twisted provides common networking, file system, timer activity non-blocking primitives and wrapping non-blocking api provided by OS.
Twisted is event-driven based. That means using callback and different architecture with synchronous program. It provides abstraction called 'deffered' helping manage callback.
However, sometimes it needs to use thread or process. 1) Thread There are some cases which cannot use callback and deffered. For example, it’s using 3rd party API which calls blocking during executing. So use it in thread. - callInThread: Execute in separated private thread (Preferred for using United Interface) - deferToThread: Same and the result returns to Deferred.
https://github.com/AstinCHOI/book_twisted/blob/master/ch10_thread_and_subprocess/blocking.py
in order to wait, not to shutdown for reactor before deferToThread finished, write reactor.stop of callback chain. https://github.com/AstinCHOI/book_twisted/blob/master/ch10_thread_and_subprocess/blocking_revised.py
- Others callFromThread callMultipleInThread blockingCallFromThread 2) Subprocess Twisted provides independent API to execute subprocess with non-blocking through reactor. 2-1) Execute a subprocess and get a result Example) remote man page https://github.com/AstinCHOI/book_twisted/blob/master/ch10_thread_and_subprocess/manpage_server.py # Terminal 1 $ python manage_server.py
# Terminal 2 $ telnet localhost 8000 > man ls - getProcessOutput: spawn subprocess and return a firing deffered when process finished. getProcessOutput(executable, args=(), env={}, path=None, reactor=None, errortoo=False) - getProcessValue(executable, args=(), env={}, path=None, reactor=None)
2-2) Custom process protocol See IProcessProtocol > ProcessProtocol - spawnProcess method (similar to getProcessOutput, but has richer syntax)
Example) Execute an echo application using EchoProcessProtocol and kill server after 10 sec. https://github.com/AstinCHOI/book_twisted/blob/master/ch10_thread_and_subprocess/twisted_spawnecho.py
# Terminal 1 $ python twisted_spawnprocess.py # Terminal 2 $ telnet localhost 8000
- ConnectionMade: Process starts as soon as it is called and transport is set to communicate this process self.transport.write method: write to stdin self.transport.writeToChild: appoint file descriptor. - outReceived: calls when data arrived at stdout and connected pipe. data is being buffering and arrived by chunk. so it’s appropriate to stack data until processEnded would be called. errReceived method is similar to this without stderr - inConnectionLost, outConnectionLost, errConnectionLost When stdin, stdout, stderr is closed, these methods are called. - processExited and processEnded: When all file descriptors are closed, processExited is callback called. ProcessEnded is appropriate place to end reactor. In Example: SIGTERM signal : graceful end KILL signal: - 3) Others http://twistedmatrix.com/documents/current/core/howto/threading.html http://twistedmatrix.com/documents/current/core/howto/gendefer.html http://twistedmatrix.com/documents/current/core/howto/process.html http://twistedmatrix.com/documents/current/core/examples/ https://launchpad.net/ampoule