Scrapy安装和问题处理

安装 scrapy

windows 安装

pip install Scrapy

结果报错:

building 'twisted.test.raiser' extension
error: [WinError 2] 系统找不到指定的文件。

+++ Command “d:\python\python.exe -u -c “import setuptools, tokenize;file=‘C:\Users\Tuteng\AppData\Local\Temp \pip-build-l7ra5rw5\Twisted\setup.py’;f=getattr(tokeniz e, ‘open’, open)(file);code=f.read().replace(’\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” install –record C:\Users\Tuteng\AppData\Local\Temp\pip-yon9xl5j-record\ins tall-record.txt –single-version-externally-managed –compile” failed with error code 1 in C:\Users\Tuteng\AppData\Local\Temp\pip-build-l7ra5rw5\Twis ted

因为scrapy框架是基于twisted 的, 所以又要去上面那个网站: http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml

搜Twisted

Twisted, an event-driven networking engine.
Twisted, an event-driven networking engine.
  ● Twisted‑17.9.0‑cp27‑cp27m‑win32.whl
  ● Twisted‑17.9.0‑cp27‑cp27m‑win_amd64.whl
  ● Twisted‑17.9.0‑cp34‑cp34m‑win32.whl
  ● Twisted‑17.9.0‑cp34‑cp34m‑win_amd64.whl
  ● Twisted‑17.9.0‑cp35‑cp35m‑win32.whl
  ● Twisted‑17.9.0‑cp35‑cp35m‑win_amd64.whl
  ● Twisted‑17.9.0‑cp36‑cp36m‑win32.whl
  ● Twisted‑17.9.0‑cp36‑cp36m‑win_amd64.whl

选择对于的版本进行下载:cp35代表的是Python版本是多少,这里我的版本是python35,操作系统是win64位的,所以我这里进行Twisted‑17.5.0‑cp35‑cp35m‑win_amd64.whl 文件的下载

输入命令:

pip install [whell]
注意:这里的为whell还是文件的绝对路径,并且带上文件格式.whl、
pip install E:\Download\Twisted-17.9.0-cp35-cp35m-win_amd64.w hl
Processing e:\download\twisted-17.9.0-cp35-cp35m-win_amd64.whl
Requirement already satisfied: constantly>=15.1 in d:\python\lib\site-packages (from Twisted==17.9.0)
Requirement already satisfied: hyperlink>=17.1.1 in d:\python\lib\site-packages (from Twisted==17.9.0)
Requirement already satisfied: zope.interface>=4.0.2 in d:\python\lib\site-packages (from Twisted==17.9.0)
Requirement already satisfied: Automat>=0.3.0 in d:\python\lib\site-packages (from Twisted==17.9.0)
Requirement already satisfied: incremental>=16.10.1 in d:\python\lib\site-packages (from Twisted==17.9.0)
Requirement already satisfied: setuptools in d:\python\lib\site-packages (from zope.interface>=4.0.2->Twisted==17.9.0)
Requirement already satisfied: six in d:\python\lib\site-packages (from Automat>=0.3.0->Twisted==17.9.0)
Requirement already satisfied: attrs in d:\python\lib\site-packages (from Automat>=0.3.0->Twisted==17.9.0)
Installing collected packages: Twisted
Successfully installed Twisted-17.9.0

最后再安装Scrapy

pip install Scrapy
Collecting Scrapy
  Using cached Scrapy-1.4.0-py2.py3-none-any.whl
======================================
.....................................................................................
Installing collected packages: PyDispatcher, Scrapy
Successfully installed PyDispatcher-2.0.5 Scrapy-1.4.0

哈哈,最后安装成功 。