线程池版的SocketServer

python的SocketServer模块提供了一个开发socket server的捷径。但是其自带的异步handler ThreadingMixIn每次处理请求是都会产生一个新的线程，在负载较重的时候会有一些问题。需要一个使用线程池的TCPServer. 

具体的代码如下： 

from threading import Thread from SocketServer import ThreadingMixIn, TCPServer, BaseRequestHandler from Queue import Queue, Empty class PooledThreadTCPServer(TCPServer, object): def __init__(self, server_address, RequestHandlerClass, thread_num): super(PooledThreadTCPServer, self).__init__(server_address, RequestHandlerClass) self.__thread_num = thread_num self.__queue = Queue() def __thread_main(self): while self.__serving: try: request, client_address = self.__queue.get(True, 0.5) except Empty: continue self.process_request_thread(request, client_address) db.close() def process_request_thread(self, request, client_address): try: self.finish_request(request, client_address) except: self.handle_error(request, client_address) finally: self.close_request(request) def process_request(self, request, client_address): self.__queue.put((request, client_address)) def serve_forever(self): self.__serving = True for i in range(self.__thread_num): t = Thread(target=self.__thread_main) t.setDaemon(True) t.start() super(PooledThreadTCPServer, self).serve_forever()  

 

继承自TCPServer和object。 
继承object的原因是需要使用super调用父类的方法，需要使用new style的类。 
process_request方法在有请求到来的时候被调用。 
它将request和客户端地址放到一个队列里。

 

配合一个RequestHandler，就可以轻松的实现一个线程池版的TCPServer 

class EchoRequestHandler(BaseRequestHandler): def handle(self): data = self.request.recv(1024) if data == "quit": self.server.shutdown() self.request.send(data) if __name__ == "__main__": addr = ("localhost", 12345) server = PooledThreadTCPServer(addr, EchoRequestHandler, 10) server.serve_forever()