Mercurial > code > home > repos > homeauto

annotate service/tomatoWifi/tomatoWifi.py @ 980:2f1cb8b5950a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
rewrites for better graph export, removal of dhcp reader Ignore-this: ecc5280b15d66020412f82ad84862074 darcs-hash:20150504002120-312f9-85bcb342f5bdae78d7b6d9083929944d4a467001
author drewp <drewp@bigasterisk.com>
date Sun, 03 May 2015 17:21:20 -0700
parents 2489d111f4f1
children fc184ea1416d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
1 #!/usr/bin/python
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
2 """
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
3 scrape the tomato router status pages to see who's connected to the
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
4 wifi access points. Includes leases that aren't currently connected.
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
5
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
6 Returns:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
7 json listing (for magma page)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
8 rdf graph (for reasoning)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
9 activity stream, when we start saving history
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
10
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
11 Todo: this should be the one polling and writing to mongo, not entrancemusic
967
2489d111f4f1 whitespace
drewp <drewp@bigasterisk.com>
parents: 966
diff changeset
12
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
13 """
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
14 from __future__ import division
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
15 import sys, cyclone.web, json, traceback, time, pystache, datetime, logging
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
16 import web.utils
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
17 from cyclone.httpclient import fetch
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
18 sys.path.append("/home/drewp/projects/photo/lib/python2.7/site-packages")
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
19 from dateutil import tz
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
20 from twisted.internet import reactor, task
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
21 from twisted.internet.defer import inlineCallbacks
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
22
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
23 from pymongo import Connection, DESCENDING
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
24 from rdflib import Namespace, Literal, URIRef
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
25 sys.path.append("/my/site/magma")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
26 from stategraph import StateGraph
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
27 from wifi import Wifi
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
28
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
29 sys.path.append("/my/proj/homeauto/lib")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
30 from cycloneerr import PrettyErrorHandler
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
31 from logsetup import log
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
32
841
ddc2cdbfde50 rdflib and jsonlib api updates
drewp <drewp@bigasterisk.com>
parents: 806
diff changeset
33
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
34 DEV = Namespace("http://projects.bigasterisk.com/device/")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
35 ROOM = Namespace("http://projects.bigasterisk.com/room/")
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
36 reasoning = "http://bang:9071/"
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
37
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
38 class Index(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
39 def get(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
40
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
41 age = time.time() - self.settings.poller.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
42 if age > 10:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
43 raise ValueError("poll data is stale. age=%s" % age)
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
44
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
45 self.set_header("Content-Type", "text/html")
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
46 self.write(open("index.html").read())
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
47
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
48 def whenConnected(mongo, macThatIsNowConnected):
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
49 lastArrive = None
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
50 for ev in mongo.find({'address': macThatIsNowConnected.upper()},
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
51 sort=[('created', -1)],
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
52 max_scan=100000):
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
53 if ev['action'] == 'arrive':
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
54 lastArrive = ev
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
55 if ev['action'] == 'leave':
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
56 break
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
57 if lastArrive is None:
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
58 raise ValueError("no past arrivals")
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
59
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
60 return lastArrive['created']
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
61
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
62 def connectedAgoString(conn):
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
63 return web.utils.datestr(
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
64 conn.astimezone(tz.tzutc()).replace(tzinfo=None))
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
65
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
66 class Table(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
67 def get(self):
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
68 def rowDict(row):
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
69 row['cls'] = "signal" if row.get('connected') else "nosignal"
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
70 if 'name' not in row:
879
691aa1333073 handle wifi users with no clientHostname
drewp <drewp@bigasterisk.com>
parents: 867
diff changeset
71 row['name'] = row.get('clientHostname', '-')
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
72 if 'signal' not in row:
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
73 row['signal'] = 'yes' if row.get('connected') else 'no'
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
74
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
75 try:
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
76 conn = whenConnected(self.settings.mongo, row.get('mac', '??'))
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
77 row['connectedAgo'] = connectedAgoString(conn)
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
78 except ValueError:
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
79 row['connectedAgo'] = 'yes' if row.get('connected') else ''
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
80 row['router'] = row.get('ssid', '')
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
81 return row
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
82
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
83 self.set_header("Content-Type", "application/xhtml+xml")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
84 self.write(pystache.render(
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
85 open("table.mustache").read(),
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
86 dict(
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
87 rows=sorted(map(rowDict, self.settings.poller.lastAddrs),
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
88 key=lambda a: (not a.get('connected'),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
89 a.get('name'))))))
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
90
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
91
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
92 class Json(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
93 def get(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
94 self.set_header("Content-Type", "application/json")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
95 age = time.time() - self.settings.poller.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
96 if age > 10:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
97 raise ValueError("poll data is stale. age=%s" % age)
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
98 self.write(json.dumps({"wifi" : self.settings.poller.lastAddrs,
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
99 "dataAge" : age}))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
100
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
101 class GraphHandler(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
102 def get(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
103 g = StateGraph(ctx=DEV['wifi'])
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
104
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
105 # someday i may also record specific AP and their strength,
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
106 # for positioning. But many users just want to know that the
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
107 # device is connected to some bigasterisk AP.
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
108 aps = URIRef("http://bigasterisk.com/wifiAccessPoints")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
109 age = time.time() - self.settings.poller.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
110 if age > 10:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
111 raise ValueError("poll data is stale. age=%s" % age)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
112
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
113 for dev in self.settings.poller.lastAddrs:
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
114 if not dev.get('connected'):
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
115 continue
966
cce0107a78b4 scan dhcpd.leases to get more info about networked devices
drewp <drewp@bigasterisk.com>
parents: 879
diff changeset
116 uri = URIRef("http://bigasterisk.com/mac/%s" % dev['mac'].lower())
cce0107a78b4 scan dhcpd.leases to get more info about networked devices
drewp <drewp@bigasterisk.com>
parents: 879
diff changeset
117 g.add((uri, ROOM['macAddress'], Literal(dev['mac'].lower())))
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
118
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
119 g.add((uri, ROOM['connected'], aps))
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
120 if 'clientHostname' in dev:
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
121 g.add((uri, ROOM['wifiNetworkName'], Literal(dev['clientHostname'])))
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
122 if 'name' in dev:
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
123 g.add((uri, ROOM['deviceName'], Literal(dev['name'])))
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
124 if 'signal' in dev:
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
125 g.add((uri, ROOM['signalStrength'], Literal(dev['signal'])))
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
126 try:
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
127 conn = whenConnected(self.settings.mongo, dev['mac'])
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
128 except ValueError:
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
129 pass
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
130 else:
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
131 g.add((uri, ROOM['connectedAgo'],
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
132 Literal(connectedAgoString(conn))))
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
133 g.add((uri, ROOM['connected'], Literal(conn)))
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
134
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
135 self.set_header('Content-type', 'application/x-trig')
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
136 self.write(g.asTrig())
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
137
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
138 class Poller(object):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
139 def __init__(self, wifi, mongo):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
140 self.wifi = wifi
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
141 self.mongo = mongo
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
142 self.lastAddrs = []
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
143 self.lastWithSignal = []
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
144 self.lastPollTime = 0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
145
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
146 def assertCurrent(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
147 dt = time.time() - self.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
148 assert dt < 10, "last poll was %s sec ago" % dt
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
149
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
150 @inlineCallbacks
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
151 def poll(self):
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
152
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
153 connectedField = 'connected'
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
154
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
155 # UVA mode:
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
156 addDhcpData = lambda *args: None
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
157
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
158 try:
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
159 newAddrs = yield self.wifi.getPresentMacAddrs()
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
160 addDhcpData(newAddrs)
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
161
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
162 newWithSignal = [a for a in newAddrs if a.get('connected')]
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
163
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
164 actions = self.computeActions(newWithSignal)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
165 for action in actions:
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
166 log.info("action: %s", action)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
167 action['created'] = datetime.datetime.now(tz.gettz('UTC'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
168 mongo.save(action)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
169 try:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
170 self.doEntranceMusic(action)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
171 except Exception, e:
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
172 log.error("entrancemusic error: %r", e)
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
173
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
174 self.lastWithSignal = newWithSignal
855
87111270eacf ping reasoning
drewp <drewp@bigasterisk.com>
parents: 841
diff changeset
175 if actions: # this doesn't currently include signal strength changes
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
176 fetch(reasoning + "immediateUpdate",
857
1168b0cd90af fix fetch calls in tomatowifi
drewp <drewp@bigasterisk.com>
parents: 856
diff changeset
177 timeout=2,
1168b0cd90af fix fetch calls in tomatowifi
drewp <drewp@bigasterisk.com>
parents: 856
diff changeset
178 headers={'user-agent': ['tomatoWifi']}).addErrback(log.warn)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
179 self.lastAddrs = newAddrs
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
180 self.lastPollTime = time.time()
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
181 except Exception, e:
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
182 log.error("poll error: %r\n%s", e, traceback.format_exc())
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
183
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
184 def computeActions(self, newWithSignal):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
185 actions = []
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
186
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
187 def makeAction(addr, act):
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
188 d = dict(sensor="wifi",
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
189 address=addr.get('mac').upper(), # mongo data is legacy uppercase
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
190 name=addr.get('name'),
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
191 networkName=addr.get('clientHostname'),
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
192 action=act)
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
193 if act == 'arrive' and 'ip' in addr:
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
194 # this won't cover the possible case that you get on
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
195 # wifi but don't have an ip yet. We'll record an
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
196 # action with no ip and then never record your ip.
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
197 d['ip'] = addr['ip']
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
198 return d
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
199
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
200 for addr in newWithSignal:
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
201 if addr['mac'] not in [r['mac'] for r in self.lastWithSignal]:
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
202 actions.append(makeAction(addr, 'arrive'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
203
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
204 for addr in self.lastWithSignal:
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
205 if addr['mac'] not in [r['mac'] for r in newWithSignal]:
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
206 actions.append(makeAction(addr, 'leave'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
207
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
208 return actions
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
209
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
210
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
211 # these need to move out to their own service
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
212 def doEntranceMusic(self, action):
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
213 import restkit, jsonlib
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
214 dt = self.deltaSinceLastArrive(action['name'])
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
215 log.debug("dt=%s", dt)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
216 if dt > datetime.timedelta(hours=1):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
217 hub = restkit.Resource(
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
218 # PSHB not working yet; "http://bang:9030/"
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
219 "http://slash:9049/"
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
220 )
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
221 action = action.copy()
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
222 del action['created']
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
223 del action['_id']
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
224 log.info("post to %s", hub)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
225 hub.post("visitorNet", payload=jsonlib.dumps(action))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
226
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
227 def deltaSinceLastArrive(self, name):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
228 results = list(self.mongo.find({'name' : name}).sort('created',
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
229 DESCENDING).limit(1))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
230 if not results:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
231 return datetime.timedelta.max
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
232 now = datetime.datetime.now(tz.gettz('UTC'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
233 last = results[0]['created'].replace(tzinfo=tz.gettz('UTC'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
234 return now - last
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
235
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
236
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
237 if __name__ == '__main__':
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
238 config = {
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
239 'servePort' : 9070,
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
240 'pollFrequency' : 1/5,
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
241 }
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
242 from twisted.python import log as twlog
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
243 #twlog.startLogging(sys.stdout)
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
244 #log.setLevel(10)
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
245 #log.setLevel(logging.DEBUG)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
246
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
247 mongo = Connection('bang', 27017, tz_aware=True)['visitor']['visitor']
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
248
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
249 wifi = Wifi()
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
250 poller = Poller(wifi, mongo)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
251 task.LoopingCall(poller.poll).start(1/config['pollFrequency'])
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
252
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
253 reactor.listenTCP(config['servePort'],
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
254 cyclone.web.Application(
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
255 [
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
256 (r"/", Index),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
257 (r'/json', Json),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
258 (r'/graph', GraphHandler),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
259 (r'/table', Table),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
260 #(r'/activity', Activity),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
261 ],
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
262 wifi=wifi,
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
263 poller=poller,
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
264 mongo=mongo))
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
265 reactor.run()