The Plusnet Hub One and webscraping for fun and profit

Your external IP can be a useful thing to know.
Perhaps you want to SSH into a box within your network, access a web server or some other service, or even update a domain’s DNS to point to your home IP address.

Normally, this can be gotten in plain-text format using a few services which return just the IP that you are visiting from, such as http://icanhazip.com and http://ipecho.net/plain using cURL or wget. However, if you plan on regular checking, you have to a) ensure that the external site is going to be up, and b) avoid hitting it with too many requests (as Major Hayden asks on his FAQ)

On some routers which have SNMP enabled, then the External IP Address is actually available within the network, and can be obtained using a command such as:

snmpwalk -v2c -c public your_router_here ip | grep ipAdEntAddr

Or some variation thereof.

If you have a Plusnet Hub One, however:Then you are not quite so lucky. Your external IP address is available within your network, but only on one of the Broadband settings pages. (results partially obscured below, but it is there)

Problem is… this page sits behind the Advanced Settings page, which for security reasons is password protected. Visiting the page and reading off the External IP address is fine, but we want to use it in some other way, so we want it programmatically so we can use it for other things, right?

Right.

The login page has some interesting functions and hidden form fields:

• md5.js – provides some md5 hashing implementations within JavaScript
• md5_pass – calculated in Javascript and then sent to the router
• post_token – a 64-character string that changes every session
• request_id – a 10-character string that changes every non-cached page access
• auth_key – a 9-character string that changes every session
• password_xxxxxxxxxx – the name of the password field changes every page access, along with request_id

It’s interesting to note that the password is never sent to the router – the JavaScript on the page does the following:

function SendPassword()
{
 var tmp;
 document.form_contents.elements['md5_pass'].value=document.form_contents.elements['password_1020071056'].value+document.form_contents.elements['auth_key'].value
 tmp=hex_md5(document.form_contents.elements['md5_pass'].value);
 document.form_contents.elements['md5_pass'].value=tmp;
 document.form_contents.elements['password_1020071056'].value="";
 mimic_button('submit_button_login_submit: ..', 1);
}

This takes the password you’ve entered, salts it with the auth_key value, and then runs it through the hex_md5 function from md5.js. This md5 hash is then the value that is passed back from the clientside browser to the server on the router. At least they’re not sending the password in plaintext in a a URL querystring. the hex_md5 function gives an output identical to running the salted password through

echo -n $pass |openssl dgst -md5|awk '{print $2}

I confirmed this by running a few test strings through both functions and getting identical results.

So we have our method for calculating the md5_pass in bash. What we need now is to get a copy of the login page, capture the cookie (rg_cookie_session_id) and parse the page for the fields that we will need to send back to the router to authenticate.

page=`curl -Ls 'http://$routerip/index.cgi?active_page=9148' -H 'Cookie: rg_cookie_session_id=' -H 'Content-Type: application/x-www-form-urlencoded' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Cache-Control: max-age=0' -H 'DNT: 1' --data 'active_page=9121' --cookie-jar cookies.txt`

Here’s the first step, we grab the contents of the login page, ensuring we save the cookie along the way, so we can pick it up later.

We can then parse for the needed variables using grep and awk to strip away the extras that we don’t need:

posttoken=`echo $page |grep post_token |awk 'BEGIN { FS = "\"post_token\" value=\"" } ; {print $2}'|awk 'BEGIN { FS = "\"" } ; {print $1}'|xargs`

requestid=`echo $page |grep request_id |awk 'BEGIN { FS = "\"request_id\" value=\"" } ; {print $2}'|awk 'BEGIN { FS = "\"" } ; {print $1}'|xargs`

authkey=`echo $page |grep auth_key |awk 'BEGIN { FS = "\"auth_key\" value=\"" } ; {print $2}'|awk 'BEGIN { FS = "\"" } ; {print $1}'|xargs`

passwordid=`echo $page |grep password_ |awk 'BEGIN { FS = "\"password_" } ; {print $2}'| awk 'BEGIN { FS = "\"" } ; {print $1}'|xargs`

The final results are pushed through xargs as a decent/quick/hacky way to strip whitespace at the start and ends of the strings.

Next we build our POST data:

postvars="request_id=$requestid&active_page=9148&active_page_str=bt_login&mimic_button_field=submit_button_login_submit%3A+..&button_value=&post_token=$posttoken&password_$passwordid=&md5_pass=$passhash&auth_key=$authkey"

and then push it using cURL to the login page:

curl -Ls 'http://$routerip/index.cgi'  -H 'Content-Type: application/x-www-form-urlencoded' -H 'Cache-Control: max-age=0' -H 'Referer: http://192.168.1.254/index.cgi?active_page=9121' -H 'Connection: keep-alive' -H 'DNT: 1' --data "$postvars" --cookie cookies.txt >/dev/null

The -L flag on cURL is to get it to follow 302 redirects, as the login page throws one of these in as it validates.

Finally we can make the request to the page we actually need, and pull the External IP address! Yay!

IP=`curl -s  'http://$routerip/index.cgi?active_page=9121&nav_clear=1' -H 'DNT: 1' -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language: en-GB,en-US;q=0.8,en;q=0.6' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Connection: keep-alive' -H 'Cache-Control: max-age=0' --cookie cookies.txt --compressed | grep -o "[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}" |head -n 1`

Here’s the listing in full:

#!/bin/bash
routerip='192.168.1.254'
pass='<PASSWORD>'
page=`curl -Ls 'http://$routerip/index.cgi?active_page=9148' -H 'Cookie: rg_cookie_session_id=' -H 'Content-Type: application/x-www-form-urlencoded' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Cache-Control: max-age=0' -H 'DNT: 1' --data 'active_page=9121' --cookie-jar cookies.txt`
posttoken=`echo $page |grep post_token |awk 'BEGIN { FS = "\"post_token\" value=\"" } ; {print $2}'|awk 'BEGIN { FS = "\"" } ; {print $1}'|xargs`
requestid=`echo $page |grep request_id |awk 'BEGIN { FS = "\"request_id\" value=\"" } ; {print $2}'|awk 'BEGIN { FS = "\"" } ; {print $1}'|xargs`
authkey=`echo $page |grep auth_key |awk 'BEGIN { FS = "\"auth_key\" value=\"" } ; {print $2}'|awk 'BEGIN { FS = "\"" } ; {print $1}'|xargs`
passwordid=`echo $page |grep password_ |awk 'BEGIN { FS = "\"password_" } ; {print $2}'| awk 'BEGIN { FS = "\"" } ; {print $1}'|xargs`
pass+=$authkey
passhash=`echo -n $pass |openssl dgst -md5|awk '{print $2}'`
postvars="request_id=$requestid&active_page=9148&active_page_str=bt_login&mimic_button_field=submit_button_login_submit%3A+..&button_value=&post_token=$posttoken&password_$passwordid=&md5_pass=$passhash&auth_key=$authkey"

curl -Ls 'http://$routerip/index.cgi' -H 'Content-Type: application/x-www-form-urlencoded' -H 'Cache-Control: max-age=0' -H 'Referer: http://$routerip/index.cgi?active_page=9121' -H 'Connection: keep-alive' -H 'DNT: 1' --data "$postvars" --cookie cookies.txt >/dev/null

IP=`curl -s 'http://$routerip/index.cgi?active_page=9121&nav_clear=1' -H 'DNT: 1' -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language: en-GB,en-US;q=0.8,en;q=0.6' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Connection: keep-alive' -H 'Cache-Control: max-age=0' --cookie cookies.txt --compressed | grep -o "[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}" |head -n 1`

echo $IP