Friday, September 27, 2013

Python requests hook returns a value to cause exception

The Documentation for python requests module says for hooks that “If the callback function returns a value, it is assumed that it is to replace the data that was passed in. If the function doesn’t return anything, nothing else is effected”

Now i am trying to return a value(int in my case) from my hook function & it throws an exception. This will be valid in all the cases when the return value is an object that DOESNOT have the raw() method defined for it.

Here is some code

def hook(resp,**kwargs):    print resp.url    return 1def main()    s = requests.Session()    s.hooks = {"response":hook}    r = s.get("http://localhost/index.html")

And here is the exception:

http://localhost/index.htmlTraceback (most recent call last): File "/home/talha/ws/test.py", line 85, in <module>   main() File "/home/talha/ws/test.py", line 72, in main   r = s.get("http://localhost/index.html") File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 347, in get   return self.request('GET', url, **kwargs) File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 335, in request   resp = self.send(prep, **send_kwargs) File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 446, in send   extract_cookies_to_jar(self.cookies, request, r.raw) AttributeError: 'int' object has no attribute 'raw'

The code in sessions.py @line 446 is trying to extract cookies after the dispatch_hook..From source

    # Response manipulation hooks    r = dispatch_hook('response', hooks, r, **kwargs)    # Persist cookies    extract_cookies_to_jar(self.cookies, request, r.raw)

Either the documentation needs to alter or the handling needs to be re-worked. What is the best way to handle this ?

[update]

Based on the comments I tried to return the base response object. Turns out it cannot be used in that manner moreover since some of its fields are initialized to None.

Newer code:

def hook(resp, **kwargs):    obj = requests.Response()    return obj

Exception thrown now:

Traceback (most recent call last):File "/home/talha/ws/test.py", line 88, in <module>   main()File "/home/talha/ws/test.py", line 75, in main   r = s.get("http://localhost/index.html")File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 347, in get   return self.request('GET', url, **kwargs)File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 335, in request   resp = self.send(prep, **send_kwargs)File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 446, in send   extract_cookies_to_jar(self.cookies, request, r.raw)File "/usr/lib/python2.7/site-packages/requests/cookies.py", line 108, in extract_cookies_to_jar    res = MockResponse(response._original_response.msg)AttributeError: 'NoneType' object has no attribute '_original_response'

What seems is that i will have to implement a full pseudo response?

If the callback function returns a value, it is assumed that it is to replace the data that was passed in. If the function doesn’t return anything, nothing else is effected.

This means that whatever you return is expected to take the place of the response object you were passed.

Nothing in the documentation states that you can return just anything. What did you expect to happen instead?

If you wanted to return a response that has different data, return something that acts like a response still. This means that either you need to subclass the requests response object, or implement something that provides the same API:

from requests.moduls import Responseclass MyIntResponse(Response):    def __init__(self, integer):        super(MyIntResponse, self).__init__()        self._content_consumed = True        self._content = integerdef hook(resp,**kwargs):    print resp.url    newresp = MyIntResponse(1)    newresp.raw = resp.raw  # copy across original HTTP response object

You may want to copy over some of the other attributes from the original response; check the documentation on what attributes Response objects have.

No comments:

Post a Comment