Pete’s Python cookbook

Attempting to ensure that Python 2.7 uses UTF-8 encoding internally and in all output:

#!/usr/bin/python
# -*- coding: utf-8 -*-

import sys
reload(sys)
sys.setdefaultencoding("utf-8")
import codecs

# Open a text file for writing:
outfile = codecs.open("out.txt", 'w', 'utf-8')

 

Sort a dictionary (aka an indexed list, aka a hash) by value in descending order and write them to a file, one line per key:value, space-delimited:

import operator

things = {'bob': 3, 'jane': 4, 'alice': 2}

sortedItems = sorted(things.items(), key=operator.itemgetter(1))
sortedItems.reverse()
outfile = open('outfile.txt', 'w')

for item in sortedItems:
  outfile.write(str(item[0]) + " " + str(item[1]) + "\n")

 

Download and cache a webpage if it hasn’t been downloaded and cached before
NOTE: Requires the existence of a folder named ‘cachedPages’!

import requests
import pickle
import slugify # Needs to be installed via 'pip install slugify'

def cacheQuery(query, forceUncache=False):
  queryFile = 'cachedPages/' + slugify(query)
  if ((not forceUncache) and os.path.isfile(queryFile)):
    data = pickle.load(open(queryFile, 'rb'))
  else:
    time.sleep(1) # Waiting 1 second is the minimum level of "politeness"
    r = requests.get(query)
    if (r.status_code == 200):
      data = r.text
      pickle.dump(data, open(queryFile, 'wb')) 
    else:
      data = None
 return data