Transforming Curl into Python using Urllib with Sentiment140 API

I am trying to use Python to request data from the Sentiment140 API.
The API is using a Bulk Classification Service (JSON). Within terminal it is working fine

curl -d "{'data': [{'text': 'I love Titanic.'}, {'text': 'I hate Titanic.'}]}" http://www.sentiment140.com/api/bulkClassifyJson

leading to the following response:

{"data":[{"text":"I love Titanic.","polarity":4,"meta":{"language":"en"}},{"text":"I hate Titanic.","polarity":0,"meta":{"language":"en"}}]}

I thought I could just use urllib to obtain the same response from my python code. I tried:

import urllibimport urllib2url = 'http://www.sentiment140.com/api/bulkClassifyJson'values = {'data': [{'text': 'I love Titanic.'}, {'text': 'I hate Titanic.'}]}data = urllib.urlencode(values)response = urllib2.urlopen(url, data)page = response.read()

The code works yet it does not donate me any results.
Am I missing something?

I think you need to use json here.

Try to do:

data = json.dumps(values) # instead of urllib.urlencode(values)response = urllib2.urlopen(url, data)page = response.read()

and on the top

import json 

Python regexp multiple expressions with grouping

I’m trying to match the output given by a Modem when asked approximately the network info, it looks like this:

Network survey started...For BCCH-Carrier:arfcn: 15,bsic: 4,dBm: -68For non BCCH-Carrier:arfcn: 10,dBm: -72arfcn: 6,dBm: -78arfcn: 11,dBm: -81arfcn: 14,dBm: -83arfcn: 16,dBm: -83

So I’ve two types of expressions to match, the BCCH & non BCCH. the following code is almost working:

match = re.findall('(?:arfcn: (\d*),dBm: (-\d*))|(?:arfcn: (\d*),bsic: (\d*),dBm: (-\d*))', data)

But it seems that BOTH expressions are being matched, & not found fields left blank:

>>> match[('', '', '15', '4', '-68'), ('10', '-72', '', '', ''), ('6', '-78', '', '', ''), ('11', '-81', '', '', ''), ('14', '-83', '', '', ''), ('16', '-83', '', '', '')]

May anyone help? Why such behaviour? I’ve tried changing the order of the expressions, with no luck.

Thanks!

That is how capturing groups work. Since you have five of them, there will always be five parts returned.

Based on your data, I think you could simplify your regex by making the bsic part optional. That way each row would return three parts, the middle one being empty for non BCCH-Carriers.

match = re.findall('arfcn: (\d*)(?:,bsic: (\d*))?,dBm: (-\d*)', data)

Sum a multidimensional list in python

How can i compute this :

[["toto", 3], ["titi", 10], ["toto", 2]]

to obtain this:

[["toto", 5], ["titi", 10]]

thanks

You can use collections.defaultdict

>>> from collections import defaultdict>>> d = defaultdict(list)>>> for i, j in L:...     d[i].append(j)... >>> [[i, sum(j)] for i, j in d.items()][['titi', 10], ['toto', 5]]

---

Thanks @raymonad for the alternate, cleaner, solution:

>>> d = defaultdict(int)>>> L = [["toto", 3], ["titi", 10], ["toto", 2]]>>> for i, j in L:...     d[i] += j... >>> d.items()[('titi', 10), ('toto', 5)]

Update method in Python dictionary

I was trying to update values in my dictionary, I came across 2 ways to do so:

product.update(map(key, value))product.update(key, value)

What is the difference between them?

The difference is that the second method does not work:

>>> {}.update(1, 2)Traceback (most recent call last):  File "<stdin>", line 1, in <module>TypeError: update expected at most 1 arguments, received 2

dict.update() expects to find a iterable of key-value pairs, keyword arguments, or another dictionary:

Update the dictionary with the key/value pairs from other, overwriting existing keys. Return None.

update() accepts either another dictionary object or an iterable of key/value pairs (as tuples or other iterables of length two). If keyword arguments are specified, the dictionary is then updated with those key/value pairs: d.update(red=1, blue=2).

map() is a built-in method that produces a sequence by applying the elements of the second (and subsequent) arguments to the first argument, which must be a callable. Unless your key object is a callable & the value object is a sequence, your first method will fail too.

Demo of a working map() application:

>>> def key(v):...     return (v, v)... >>> value = range(3)>>> map(key, value)[(0, 0), (1, 1), (2, 2)]>>> product = {}>>> product.update(map(key, value))>>> product{0: 0, 1: 1, 2: 2}

Here map() just produces key-value pairs, which satisfies the dict.update() expectations.

Http Download with Python

Simple download

import urlliburllib.urlretrieve ("http://www.example.com/songs/mp3.mp3", "mp3.mp3")

Download with progress

 import urllib2url = "http://download.thinkbroadband.com/10MB.zip"file_name = url.split('/')[-1]u = urllib2.urlopen(url)f = open(file_name, 'wb')meta = u.info()file_size = int(meta.getheaders("Content-Length")[0])print "Downloading: %s Bytes: %s" % (file_name, file_size)file_size_dl = 0block_sz = 8192while True:    buffer = u.read(block_sz)    if not buffer:        break    file_size_dl += len(buffer)    f.write(buffer)    status = r"%10d  [%3.2f%%]" % (file_size_dl, file_size_dl * 100. / file_size)    status = status + chr(8)*(len(status)+1)    print status,f.close()

Connect to mysql database using python with MySQLdb cursor

import sysimport MySQLdbimport MySQLdb.cursorsconn = MySQLdb.Connect(    host='localhost', user='vroom',    passwd='vroom', db='vroom',compress=1,    cursorclass=MySQLdb.cursors.DictCursor)cursor = conn.cursor()cursor.execute("SELECT * FROM posts")rows = cursor.fetchall()cursor.close()conn.close()for row in rows:    print row['title'], row['create_date']