先上代码
import requests
import gevent
from gevent.pool import Pool
from gevent import monkey
# monkey.patch_all()
monkey.patch_socket()
monkey.patch_ssl()
contents = []
pool = Pool(20)
def req(url):
res = requests.get(url)
contents.append(res.status_code)
urls = [
"http://www.baidu.com",
"http://www.iplaysoft.com",
"http://www.cupfox.com",
"http://www.opbear.com",
"http://www.bing.ren"
]
gtasks = [pool.spawn(req, url) for url in urls]
gevent.joinall(gtasks)
print(contents)
就这段代码用的是最简单的 gevent 协程请求 requests 如果只用 monkey.patch_socket()是完全没有问题的 但凡加上了 monkey.patch_ssl()就会报错 或者加上 monkey.patch_all()也会报错
Traceback (most recent call last):
File "/usr/local/python3.6/lib/python3.6/site-packages/gevent/greenlet.py", line 536, in run
result = self._run(*self.args, **self.kwargs)
File "test.py", line 14, in req
res = requests.get(url)
File "/usr/local/python3.6/lib/python3.6/site-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/python3.6/lib/python3.6/ssl.py", line 459, in options
super(SSLContext, SSLContext).options.set(self, value)
[Previous line repeated 316 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object`
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/python3.6/lib/python3.6/site-packages/gevent/hub.py", line 866, in switch
switch(value)
File "/usr/local/python3.6/lib/python3.6/site-packages/gevent/_tblib.py", line 425, in dump_traceback
return dumps(tb)
Tue Sep 19 10:28:11 2017 <built-in method switch of Greenlet object at 0x10c6c4f20> failed with RecursionError<br/>
Traceback (most recent call last):
File "test.py", line 25, in <module>
gevent.joinall(gtasks)
File "/usr/local/python3.6/lib/python3.6/site-packages/gevent/greenlet.py", line 649, in joinall
return wait(greenlets, timeout=timeout, count=count)
File "/usr/local/python3.6/lib/python3.6/site-packages/gevent/hub.py", line 1037, in wait
return list(iwait(objects, timeout, count))
gevent.hub.LoopExit: ('This operation would block forever',<Hub at 0x10c6c4af8 select default pending=0 ref=0 resolver=<gevent.resolver_thread.Resolver at 0x10c7905f8 pool=<ThreadPool at 0x10c790668 0/2/10>> threadpool=<ThreadPool at 0x10c790668 0/2/10>>)
今天的错误代码长这样。 因为错误代码很长很乱 我就删掉了中间的一些过程 把头尾都留下来了。 gevent 版本 1.2.2 requests 版本 2.18.4
有没有大大帮忙解释下出现错误的原因以及 monkey.patch_ssl()所做的事情
1
nightstart OP 求不沉.... 这个问题困扰我好久了
|
2
NoAnyLove 2017-09-19 11:46:48 +08:00 1
这种问题,翻一下官方 repo 的 issue 就知道了。Python 3 下不知道什么问题造成的,解决方法是,把下面几行内容移动到最开头:
from gevent.pool import Pool from gevent import monkey # monkey.patch_all() monkey.patch_socket() monkey.patch_ssl() |
3
nightstart OP @NoAnyLove 可以问一下是在哪里看的吗。
|
4
qs 2017-09-19 14:34:55 +08:00
同样的问题 我本地跑会报这个错误 测试服务器跑和线上服务器跑没这个错误
官方 repo 的 issue 里说是 gevent 和 requests 的问题 gevent 版本升级到 1.21 也都是在 3.6 上出现的 可以尝试升级下 gevent |
5
fy 2017-09-19 15:01:25 +08:00
老哥 python3 还用 gevent ? asyncio 不好吗
|
6
janxin 2017-09-19 15:04:19 +08:00
|
7
calease 2017-09-19 16:37:37 +08:00
gevent 文档里写的很清楚了一定要先 from gevent import monkey; monkey.patch_all()之后才能 import 别的 library。
http://www.gevent.org/intro.html#monkey-patching When monkey patching, it is recommended to do so as early as possible in the lifetime of the process. If possible, monkey patching should be the first lines executed. |
8
NoAnyLove 2017-09-19 22:15:10 +08:00
|
9
nightstart OP @qs 我这边也发生了这个情况 我的自己上跑有问题 我小伙伴上的就没问题
|
10
nightstart OP @fy gevent 简直小白必备 之前去看了下 asyncio 发现哇 gevent 还是好简单于是就用回来了
|