A couple of days ago, a friend sent me a link to Google’s “killer-robots.txt” easter-egg. I was curious to see if there was more to it than just the file (a tribute to both the robots.txt file and to the Terminator movies), and started trying to mess around with the user agent I was sending to google.com. I had started by using Burp Suite, which allows you to intercept and alter your requests, but it was a drag having to repeatedly intercept and alter the User-Agent string; consequently, I ended up writing the python code below.
It’s the first time I use Twisted to do anything – I have to say, I like its simplicity.
Like the header says, I did end up using a Firefox plugin to change the user agent; but I thought that the code would be useful to have around for those of you that might wish to intercept and alter a whole bunch of packets. I use regex to identify the user-agent string and replace it with whatever I want – this method grants you quite a bit of flexibility in finding and replacing elements of your request.
# Written by redparanoid (@Inf0Junki3), Friday 11.07.2014 # ReplacingProxy: A proxy that takes incoming requests, substitutes the user agent with another # value, and passes the request on. I wrote this script because I was curious about the # killer-robots.txt easter egg on google, and wanted to see if fiddling with the user-agent would # change the content of the pages. Sadly, it doesn't. # Note that I ended up using Firefox's User Agent Switcher to force the agent, because all google # traffic is over SSL and I didn't want to mess around with HTTPS. I shall someday fix this, when # I am bored again. In the meantime, this is still a neat proof of concept 🙂 # References: # https://wiki.python.org/moin/Twisted-Examples # http://stackoverflow.com/questions/9063583/python-twisted-proxy-how-to-intercept-packets from twisted.web import proxy, http from twisted.internet import reactor from twisted.python import log import sys import re from termcolor import colored #log to the standard output. log.startLogging(sys.stdout) class ReplacingProxy(proxy.Proxy): """ The class used to intercept the data received and print it to the std output. """ def dataReceived(self, data): match = re.compile(r"User-Agent: .*$", re.MULTILINE) new_user_agent = "User-Agent: T-1000" if "User-Agent: " in data: old_user_agent = match.search(data).group(0) print colored( "Substituting %s with %s" % (old_user_agent, new_user_agent), "blue") new_data = match.sub(new_user_agent, data) print colored( "OUTPUT: %s" % new_data, "green") return proxy.Proxy.dataReceived(self, new_data) #end def dataReceived #end class class ProxyFactory(http.HTTPFactory): """ The factory class for the proxy. """ protocol = ReplacingProxy #end class reactor.listenTCP(8080, ProxyFactory()) reactor.run()