Python发送WEB请求，并对WEB内容进行解析 - lpxuan151009的专栏 - 博客频道 - CSDN.NET

Python发送WEB请求，并对WEB内容进行解析 - lpxuan151009的专栏 - 博客频道 - CSDN.NET

def getRelateKeywords(requestUrl):

    url = requestUrl.replace(' ', '%20');

    page_encode = "gbk"

   

    request = urllib2.Request(url)

    request.add_header("Accept-encoding", "gzip")

   

    usock = urllib2.urlopen(request)

    page = usock.read()

   

    if usock.headers.get('content-encoding', None) == 'gzip':

        page = gzip.GzipFile(fileobj=StringIO.StringIO(page)).read()



    if not isinstance(page, unicode):

        page = unicode(page, page_encode)



    #print(page)