Replacing your user agent with a Twisted proxy

A couple of days ago, a friend sent me a link to Google’s “killer-robots.txt” easter-egg. I was curious to see if there was more to it than just the file (a tribute to both the robots.txt file and to the Terminator movies), and started trying to mess around with the user agent I was sending to google.com. I had started by using Burp Suite, which allows you to intercept and alter your requests, but it was a drag having to repeatedly intercept and alter the User-Agent string; consequently, I ended up writing the python code below.

It’s the first time I use Twisted to do anything – I have to say, I like its simplicity.

Like the header says, I did end up using a Firefox plugin to change the user agent; but I thought that the code would be useful to have around for those of you that might wish to intercept and alter a whole bunch of packets. I use regex to identify the user-agent string and replace it with whatever I want – this method grants you quite a bit of flexibility in finding and replacing elements of your request.

Enjoy!


# Written by redparanoid (@Inf0Junki3), Friday 11.07.2014

# ReplacingProxy: A proxy that takes incoming requests, substitutes the user agent with another
# value, and passes the request on. I wrote this script because I was curious about the
# killer-robots.txt easter egg on google, and wanted to see if fiddling with the user-agent would
# change the content of the pages. Sadly, it doesn't.
# Note that I ended up using Firefox's User Agent Switcher to force the agent, because all google
# traffic is over SSL and I didn't want to mess around with HTTPS. I shall someday fix this, when
# I am bored again. In the meantime, this is still a neat proof of concept ūüôā

# References:
#   https://wiki.python.org/moin/Twisted-Examples
#   http://stackoverflow.com/questions/9063583/python-twisted-proxy-how-to-intercept-packets

from   twisted.web         import proxy, http
from   twisted.internet    import reactor
from   twisted.python      import log
import sys
import re
from   termcolor import colored

#log to the standard output.
log.startLogging(sys.stdout)

class ReplacingProxy(proxy.Proxy):
  """
  The class used to intercept the data received and print it to the std output.
  """
  def dataReceived(self, data):

  match = re.compile(r"User-Agent: .*$", re.MULTILINE)

  new_user_agent = "User-Agent: T-1000"

  if "User-Agent: " in data:
    old_user_agent = match.search(data).group(0)
    print colored(
        "Substituting %s with %s" % (old_user_agent, new_user_agent),
        "blue")
    new_data = match.sub(new_user_agent, data)
    print colored(
        "OUTPUT: %s" % new_data,
        "green")

    return proxy.Proxy.dataReceived(self, new_data)
  #end def dataReceived
#end class

class ProxyFactory(http.HTTPFactory):
  """
  The factory class for the proxy.
  """
  protocol = ReplacingProxy
#end class

reactor.listenTCP(8080, ProxyFactory())
reactor.run()