Mercurial > code > home > repos > homeauto

annotate service/wifi/scrape_unmaintained.py @ 1463:2452dd87aa9c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
release 0.9.0 Ignore-this: cd9ac6c22e35bce9d647edd85ab75acd darcs-hash:ee94646423f37369ac319ecce7a49cb108ecada4
author drewp <drewp@bigasterisk.com>
date Sun, 24 Nov 2019 00:02:42 -0800
parents 7de8f0cd3392
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1226
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
1
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
2 @inlineCallbacks
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
3 def loadUvaData():
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
4 import lxml.html.soupparser
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
5
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
6 config = json.load(open("priv-uva.json"))
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
7 headers = {'Authorization': ['Basic %s' % config['userPass'].encode('base64').strip()]}
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
8 resp = yield fetch('http://10.2.0.2/wlstationlist.cmd', headers=headers)
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
9 root = lxml.html.soupparser.fromstring(resp.body)
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
10 byMac = {}
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
11 for tr in root.cssselect('tr'):
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
12 mac, connected, auth, ssid, iface = [td.text_content().strip() for td in tr.getchildren()]
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
13 if mac == "MAC":
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
14 continue
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
15 connected = connected.lower() == 'yes'
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
16 byMac[mac] = dict(mac=mac, connected=connected, auth=auth == 'Yes', ssid=ssid, iface=iface)
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
17
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
18 resp = yield fetch('http://10.2.0.2/DHCPTable.asp', headers=headers)
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
19 for row in re.findall(r'new AAA\((.*)\)', resp.body):
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
20 clientHostname, ipaddr, mac, expires, iface = [s.strip("'") for s in row.rsplit(',', 4)]
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
21 if clientHostname == 'wlanadv.none':
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
22 continue
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
23 byMac.setdefault(mac, {}).update(dict(
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
24 clientHostname=clientHostname, connection=iface, ipaddr=ipaddr, dhcpExpires=expires))
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
25
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
26 returnValue(sorted(byMac.values()))
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
27
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
28 @inlineCallbacks
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
29 def loadCiscoData():
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
30 config = json.load(open("priv-uva.json"))
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
31 headers = {'Authorization': ['Basic %s' % config['userPass'].encode('base64').strip()]}
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
32 print(headers)
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
33 resp = yield fetch('http://10.2.0.2/', headers=headers)
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
34 print(resp.body)
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
35 returnValue([])
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
36
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
37
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
38 def jsValue(js, variableName):
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
39 # using literal_eval instead of json parser to handle the trailing commas
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
40 val = re.search(variableName + r'\s*=\s*(.*?);', js, re.DOTALL).group(1)
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
41 return ast.literal_eval(val)
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
42
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
43
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
44 def _parseZyxel(self, data, routerName):
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
45 import lxml.html.soupparser
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
46
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
47 root = lxml.html.soupparser.fromstring(data)
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
48 for tr in root.cssselect('tr'):
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
49 mac, assoc, uth, ssid, iface = [td.text_content().strip() for td in tr.getchildren()]
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
50 if mac == "MAC":
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
51 continue
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
52 assoc = assoc.lower() == 'yes'
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
53 yield dict(router=routerName, mac=mac, assoc=assoc, connected=assoc)
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
54
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
55 def _parseTomato(self, data, routerName):
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
56 for iface, mac, signal in jsValue(data, 'wldev'):
7de8f0cd3392 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp <drewp@bigasterisk.com>
parents:
diff changeset
57 yield dict(router=routerName, mac=mac, signal=signal, connected=bool(signal))