monotai

consist3ntly inconsistEnt

How to Manipulate a Remote Website (that is not necessarily yours)

My current tasks regularly involve fiddling around with the live website of a rather large eCommerce venture. Sometimes I need to review or research the behaviour of a third party script, sometimes it’s about analyzing timing or writing automated tests or implementing crazy A/B Tests.

To make things a little bit more interesting, neither do I have access to the code, nor to a testing environment, so I cannot inject JavaScript or play around with the templates in order to add my tests. So I’ve built myself a little toolset, that allows me to do a lot of fancy things, without knowing anything more than what the website is showing to the outside world (which turns out to be quite a lot).

Please note, all this is, of course, limited to the local browser and it’s not always perfect when it comes to timing and sync vs. async. operations, but still...!

As a result I even more love hate love hate love hate love hate love hate fear JavaScript and I completely stopped trusting any browser extension!

The Goal

So why would I want to play around with a website and what is it that I need to achive? It’s simple, the aim is to add, change, replace and manipulate as many aspects of the frontend as possible. This includes

The Tools

I’m using Google’s Chrome browser for this kind of task and I assume that you’re already aware of the amazing developer tools this browser has. If not, please start by having a look at the console (ok, that’s obvious) and the network tab (especially for analyzing headers, responses and http response codes).

Screenshot

If you want to dive deeper, some of these tricks and tools might come handy. I strongly suggest to create a browser profile especially dedicated to this setup. It happened more than once to me, that I forgot to remove my work and weird things happened which led to unnecessary long hours on my side!

CJS – Inject Custom JavaScript

The CJS extension is incredibly helpful if you want to inject some JavaScript into the target site without having to type it into the console every time you reload the page (which often is not sufficient due to your manually typed commands come way too late).

Screenshot

There are two modes: injecting external scripts or simply adding commands as a script tag in the header of the target site. The later is basically what you want – even if you do nothing more than using it to inject script src tags into the site, because:

Please be Careful!

This extension is everything but stable! The injection of external scripts works, or not (depending on criterias, that are unkown to me) and the extension is enabled or disabled for a host (including all subdomains – which may not be what you would expect). Plus there is no visual indicator, if it is active. So I always add a console.log('cjs enabled‘);.

Injecting Custom Styles

There are a few extensions that allow you to inject styles (and of course once more Chrome’s developer tools are doing a great job), but I haven’t found anything that works for me. This is sad, because injecting styles is a really important. Especially if you want to play around with things like A/B Testing you need to be able to hide site elements before they are even loaded, so elem.style.display = none; is not an option here.

After some back and forth, I’m happy using CJS to inject a link tag into the head of the site.

var head = document.getElementsByTagName('head')[0];
var link = document.createElement('link');
link.rel = 'stylesheet';
link.type = 'text/css';
link.href = '//127.0.0.1/path/to/my/external.css';
link.media = 'screen';
link.onload = function () {
  externalStylesLoaded = true;
}
head.appendChild(link);

Please note the onload handler. I use it as a sort of poor man’s promise here, allowing me to wait with certain operations (e.g. showing a modified site element) until the external styles are loaded.

If this is too much, simply inject a style tag into the head and add the necessary styles inline:

var sheet = (function() {
  // Create the <style> tag
  var style = document.createElement('style');
  style.appendChild(document.createTextNode(''));
  document.head.appendChild(style);
  return style.sheet;
})();

sheet.insertRule('#selector { display:none; }', 0);

thx to David Walsh for describing this technique

Switcheroo – Replace a Script at Runtime

Switcheroo allows you to completely replace scripts at runtime in the context of the current website. All you have to do is inserting the URL of a script and the URL of the script you want to replace it with and Switcheroo stops loading the original script. – This also works for any other resource you might want to replace.

Although it works like a charm and allows you to do lots of amazing things, I find it a bit limited because it only allows you to replace existing resources and most of the time this is simply not enough for me.

Screenshot

My Workflow

Most of the times I start with downloading the script I want to replace and unminify it using a service like jsbeautifier.org.

Next thing you need is a local server because Switcheroo does not deliver files locally. The easiest way to do this is using something like PHP’s development server, so simply cd into the directory, where your script is located and run:

> php -S 127.0.0.1:8000 .

This command starts a local server listening on port 8000 (or failing if this port is already taken on your machine) and delivering all files in that directory.

http-server – A Local Server for Static Files

Switcheroo and CJS are great, but if your target website is running on HTTPS (which it should these days), you’re in trouble because Chrome simply refuses to load any of your external scripts.

Here comes http-server to the rescue! http-server is a little node app, that acts as a http server (like a lot of other apps do as well). The amazing thing about it is the -S switch. This enables SSL and allows you to serve files via HTTPS!

Generate Keys

All http-server needs is a key and a certificate. Default are key.pemand cert.pem located in the same directory the server is started. This is a command to generate excatly these (self-signed, not secure, not ready for production) certificates:

> openssl req -x509 -newkey rsa:4096 -nodes -keyout key.pem -out cert.pem -days 365

Copy the files into the target dir, cd into it and run:

> npm install -g http-server
> http-server -S
Starting up http-server, serving ./ through https
Available on:
  https://127.0.0.1:8080
  https://192.168.56.1:8080
Hit CTRL-C to stop the server