Python3.7 Scrapy crawl 运行出错解决方法

笔记本安装的是Python3.7,装上依赖包和scrapy后运行爬虫命令出错

from scrapy import cmdline

cmdline.execute('scrapy crawl xxx'.split(' '))

主要错误如下:

 File "D:\Python37\lib\site-packages\scrapy\extensions\telnet.py", line 12, in 
    from twisted.conch import manhole, telnet
  File "D:\Python37\lib\site-packages\twisted\conch\manhole.py", line 154
    def write(self, data, async=False):
                              ^
SyntaxError: invalid syntax

Process finished with exit code 1

解决方法:

    def write(self, data, shark=False):
        self.handler.addOutput(data, shark)

    def addOutput(self, data, shark=False):
        if shark:
            self.terminal.eraseLine()
            self.terminal.cursorBackward(len(self.lineBuffer) + len(self.ps[self.pn]))

        self.terminal.write(data)

        if shark:
            if self._needsNewline():
                self.terminal.nextLine()

            self.terminal.write(self.ps[self.pn])

将源码manhole.py中的async参数更改为shark(注意更换全部)
可以直接点击错误跳转
也可以通过文件路径查找
D:\Python37\Lib\site-packages\twisted\conch\manhole.py

改完再运行就OK了

你可能感兴趣的:(Python3.7 Scrapy crawl 运行出错解决方法)