Mercurial > code > home > repos > homeauto

annotate service/tomatoWifi/tomatoWifi.py @ 1188:afd19bf8b499

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
various docker setups and build fixes Ignore-this: adb559337fa1bd058818d1fc99d80ac1 darcs-hash:9aa24b30a2d3eab3676b85d2aa686d6ef24b5717
author drewp <drewp@bigasterisk.com>
date Fri, 28 Dec 2018 02:13:15 -0800
parents 0ba7a29a1886
children 10ec2744fe8a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
1 #!/usr/bin/python
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
2 """
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
3 scrape the tomato router status pages to see who's connected to the
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
4 wifi access points. Includes leases that aren't currently connected.
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
5
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
6 Returns:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
7 json listing (for magma page)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
8 rdf graph (for reasoning)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
9 activity stream, when we start saving history
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
10
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
11 Todo: this should be the one polling and writing to mongo, not entrancemusic
967
2489d111f4f1 whitespace
drewp <drewp@bigasterisk.com>
parents: 966
diff changeset
12
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
13 """
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
14 from __future__ import division
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
15 import sys, cyclone.web, json, traceback, time, pystache, datetime, logging
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
16 import web.utils
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
17 from cyclone.httpclient import fetch
1188
afd19bf8b499 various docker setups and build fixes
drewp <drewp@bigasterisk.com>
parents: 1145
diff changeset
18
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
19 from dateutil import tz
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
20 from twisted.internet import reactor, task
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
21 from twisted.internet.defer import inlineCallbacks
994
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
22 import docopt
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
23 from influxdb import InfluxDBClient
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
24 from pymongo import Connection, DESCENDING
1145
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
25 from rdflib import Namespace, Literal, URIRef, ConjunctiveGraph
1188
afd19bf8b499 various docker setups and build fixes
drewp <drewp@bigasterisk.com>
parents: 1145
diff changeset
26
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
27 from stategraph import StateGraph
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
28 from wifi import Wifi
1188
afd19bf8b499 various docker setups and build fixes
drewp <drewp@bigasterisk.com>
parents: 1145
diff changeset
29
1145
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
30 from patchablegraph import PatchableGraph, CycloneGraphEventsHandler, CycloneGraphHandler
1188
afd19bf8b499 various docker setups and build fixes
drewp <drewp@bigasterisk.com>
parents: 1145
diff changeset
31
1145
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
32 from rdfdb.patch import Patch
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
33
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
34 from cycloneerr import PrettyErrorHandler
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
35 from logsetup import log
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
36
841
ddc2cdbfde50 rdflib and jsonlib api updates
drewp <drewp@bigasterisk.com>
parents: 806
diff changeset
37
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
38 DEV = Namespace("http://projects.bigasterisk.com/device/")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
39 ROOM = Namespace("http://projects.bigasterisk.com/room/")
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
40 reasoning = "http://bang:9071/"
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
41
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
42 class Index(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
43 def get(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
44
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
45 age = time.time() - self.settings.poller.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
46 if age > 10:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
47 raise ValueError("poll data is stale. age=%s" % age)
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
48
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
49 self.set_header("Content-Type", "text/html")
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
50 self.write(open("index.html").read())
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
51
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
52 def whenConnected(mongo, macThatIsNowConnected):
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
53 lastArrive = None
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
54 for ev in mongo.find({'address': macThatIsNowConnected.upper()},
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
55 sort=[('created', -1)],
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
56 max_scan=100000):
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
57 if ev['action'] == 'arrive':
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
58 lastArrive = ev
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
59 if ev['action'] == 'leave':
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
60 break
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
61 if lastArrive is None:
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
62 raise ValueError("no past arrivals")
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
63
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
64 return lastArrive['created']
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
65
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
66 def connectedAgoString(conn):
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
67 return web.utils.datestr(
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
68 conn.astimezone(tz.tzutc()).replace(tzinfo=None))
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
69
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
70 class Table(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
71 def get(self):
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
72 def rowDict(row):
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
73 row['cls'] = "signal" if row.get('connected') else "nosignal"
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
74 if 'name' not in row:
879
691aa1333073 handle wifi users with no clientHostname
drewp <drewp@bigasterisk.com>
parents: 867
diff changeset
75 row['name'] = row.get('clientHostname', '-')
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
76 if 'signal' not in row:
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
77 row['signal'] = 'yes' if row.get('connected') else 'no'
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
78
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
79 try:
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
80 conn = whenConnected(self.settings.mongo, row.get('mac', '??'))
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
81 row['connectedAgo'] = connectedAgoString(conn)
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
82 except ValueError:
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
83 row['connectedAgo'] = 'yes' if row.get('connected') else ''
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
84 row['router'] = row.get('ssid', '')
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
85 return row
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
86
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
87 self.set_header("Content-Type", "application/xhtml+xml")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
88 self.write(pystache.render(
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
89 open("table.mustache").read(),
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
90 dict(
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
91 rows=sorted(map(rowDict, self.settings.poller.lastAddrs),
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
92 key=lambda a: (not a.get('connected'),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
93 a.get('name'))))))
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
94
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
95
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
96 class Json(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
97 def get(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
98 self.set_header("Content-Type", "application/json")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
99 age = time.time() - self.settings.poller.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
100 if age > 10:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
101 raise ValueError("poll data is stale. age=%s" % age)
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
102 self.write(json.dumps({"wifi" : self.settings.poller.lastAddrs,
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
103 "dataAge" : age}))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
104
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
105 class Poller(object):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
106 def __init__(self, wifi, mongo):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
107 self.wifi = wifi
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
108 self.mongo = mongo
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
109 self.lastAddrs = []
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
110 self.lastWithSignal = []
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
111 self.lastPollTime = 0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
112
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
113 def assertCurrent(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
114 dt = time.time() - self.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
115 assert dt < 10, "last poll was %s sec ago" % dt
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
116
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
117 @inlineCallbacks
1145
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
118 def poll(self):
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
119 connectedField = 'connected'
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
120 now = int(time.time())
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
121
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
122 # UVA mode:
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
123 addDhcpData = lambda *args: None
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
124
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
125 try:
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
126 newAddrs = yield self.wifi.getPresentMacAddrs()
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
127 addDhcpData(newAddrs)
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
128
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
129 newWithSignal = [a for a in newAddrs if a.get('connected')]
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
130
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
131 actions = self.computeActions(newWithSignal)
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
132 points = []
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
133 for action in actions:
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
134 log.info("action: %s", action)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
135 action['created'] = datetime.datetime.now(tz.gettz('UTC'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
136 mongo.save(action)
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
137 points.append(
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
138 self.influxPoint(now, action['address'].lower(),
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
139 1 if action['action'] == 'arrive' else 0))
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
140 try:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
141 self.doEntranceMusic(action)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
142 except Exception, e:
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
143 log.error("entrancemusic error: %r", e)
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
144
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
145 if now // 3600 > self.lastPollTime // 3600:
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
146 log.info('hourly writes')
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
147 for addr in newWithSignal:
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
148 points.append(self.influxPoint(now, addr['mac'].lower(), 1))
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
149
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
150 influx.write_points(points, time_precision='s')
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
151 self.lastWithSignal = newWithSignal
855
87111270eacf ping reasoning
drewp <drewp@bigasterisk.com>
parents: 841
diff changeset
152 if actions: # this doesn't currently include signal strength changes
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
153 fetch(reasoning + "immediateUpdate",
1084
3a9f20f881d1 immediate mode needs a PUT request
drewp <drewp@bigasterisk.com>
parents: 994
diff changeset
154 method='PUT',
857
1168b0cd90af fix fetch calls in tomatowifi
drewp <drewp@bigasterisk.com>
parents: 856
diff changeset
155 timeout=2,
1168b0cd90af fix fetch calls in tomatowifi
drewp <drewp@bigasterisk.com>
parents: 856
diff changeset
156 headers={'user-agent': ['tomatoWifi']}).addErrback(log.warn)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
157 self.lastAddrs = newAddrs
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
158 self.lastPollTime = now
1145
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
159
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
160 self.updateGraph(masterGraph)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
161 except Exception, e:
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
162 log.error("poll error: %r\n%s", e, traceback.format_exc())
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
163
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
164 def influxPoint(self, now, address, value):
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
165 return {
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
166 'measurement': 'presence',
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
167 'tags': {'sensor': 'wifi', 'address': address,},
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
168 'fields': {'value': value},
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
169 'time': now,
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
170 }
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
171
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
172 def computeActions(self, newWithSignal):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
173 actions = []
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
174
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
175 def makeAction(addr, act):
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
176 d = dict(sensor="wifi",
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
177 address=addr.get('mac').upper(), # mongo data is legacy uppercase
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
178 name=addr.get('name'),
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
179 networkName=addr.get('clientHostname'),
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
180 action=act)
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
181 if act == 'arrive' and 'ip' in addr:
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
182 # this won't cover the possible case that you get on
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
183 # wifi but don't have an ip yet. We'll record an
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
184 # action with no ip and then never record your ip.
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
185 d['ip'] = addr['ip']
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
186 return d
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
187
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
188 for addr in newWithSignal:
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
189 if addr['mac'] not in [r['mac'] for r in self.lastWithSignal]:
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
190 actions.append(makeAction(addr, 'arrive'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
191
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
192 for addr in self.lastWithSignal:
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
193 if addr['mac'] not in [r['mac'] for r in newWithSignal]:
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
194 actions.append(makeAction(addr, 'leave'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
195
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
196 return actions
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
197
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
198
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
199 # these need to move out to their own service
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
200 def doEntranceMusic(self, action):
1188
afd19bf8b499 various docker setups and build fixes
drewp <drewp@bigasterisk.com>
parents: 1145
diff changeset
201 import restkit, json
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
202 dt = self.deltaSinceLastArrive(action['name'])
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
203 log.debug("dt=%s", dt)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
204 if dt > datetime.timedelta(hours=1):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
205 hub = restkit.Resource(
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
206 # PSHB not working yet; "http://bang:9030/"
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
207 "http://slash:9049/"
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
208 )
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
209 action = action.copy()
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
210 del action['created']
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
211 del action['_id']
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
212 log.info("post to %s", hub)
1188
afd19bf8b499 various docker setups and build fixes
drewp <drewp@bigasterisk.com>
parents: 1145
diff changeset
213 hub.post("visitorNet", payload=json.dumps(action))
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
214
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
215 def deltaSinceLastArrive(self, name):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
216 results = list(self.mongo.find({'name' : name}).sort('created',
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
217 DESCENDING).limit(1))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
218 if not results:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
219 return datetime.timedelta.max
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
220 now = datetime.datetime.now(tz.gettz('UTC'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
221 last = results[0]['created'].replace(tzinfo=tz.gettz('UTC'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
222 return now - last
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
223
1145
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
224 def updateGraph(self, masterGraph):
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
225
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
226 g = ConjunctiveGraph()
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
227 ctx = DEV['wifi']
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
228
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
229 # someday i may also record specific AP and their strength,
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
230 # for positioning. But many users just want to know that the
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
231 # device is connected to some bigasterisk AP.
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
232 age = time.time() - self.lastPollTime
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
233 if age > 10:
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
234 raise ValueError("poll data is stale. age=%s" % age)
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
235
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
236 for dev in self.lastAddrs:
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
237 if not dev.get('connected'):
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
238 continue
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
239 uri = URIRef("http://bigasterisk.com/mac/%s" % dev['mac'].lower())
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
240 g.add((uri, ROOM['macAddress'], Literal(dev['mac'].lower()), ctx))
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
241
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
242 g.add((uri, ROOM['connected'], {
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
243 'wireless': URIRef("http://bigasterisk.com/wifiAccessPoints"),
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
244 '2.4G': URIRef("http://bigasterisk.com/wifiAccessPoints"),
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
245 '5G': URIRef("http://bigasterisk.com/wifiAccessPoints"),
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
246 'wired': URIRef("http://bigasterisk.com/houseOpenNet")}[dev['contype']], ctx))
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
247 if 'clientHostname' in dev and dev['clientHostname']:
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
248 g.add((uri, ROOM['wifiNetworkName'], Literal(dev['clientHostname']), ctx))
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
249 if 'name' in dev and dev['name']:
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
250 g.add((uri, ROOM['deviceName'], Literal(dev['name']), ctx))
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
251 if 'signal' in dev:
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
252 g.add((uri, ROOM['signalStrength'], Literal(dev['signal']), ctx))
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
253 if 'model' in dev:
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
254 g.add((uri, ROOM['networkModel'], Literal(dev['model']), ctx))
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
255 try:
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
256 conn = whenConnected(mongo, dev['mac'])
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
257 except ValueError:
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
258 traceback.print_exc()
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
259 pass
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
260 else:
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
261 g.add((uri, ROOM['connectedAgo'],
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
262 Literal(connectedAgoString(conn)), ctx))
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
263 g.add((uri, ROOM['connected'], Literal(conn), ctx))
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
264 masterGraph.setToGraph(g)
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
265
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
266
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
267 if __name__ == '__main__':
994
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
268 args = docopt.docopt('''
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
269 Usage:
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
270 tomatoWifi [options]
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
271
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
272 Options:
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
273 -v, --verbose more logging
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
274 --port=<n> serve on port [default: 9070]
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
275 --poll=<freq> poll frequency [default: .2]
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
276 ''')
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
277 if args['--verbose']:
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
278 from twisted.python import log as twlog
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
279 twlog.startLogging(sys.stdout)
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
280 log.setLevel(10)
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
281 log.setLevel(logging.DEBUG)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
282
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
283 mongo = Connection('bang', 27017, tz_aware=True)['visitor']['visitor']
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
284 influx = InfluxDBClient('bang', 9060, 'root', 'root', 'main')
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
285
1145
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
286 masterGraph = PatchableGraph()
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
287 wifi = Wifi()
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
288 poller = Poller(wifi, mongo)
994
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
289 task.LoopingCall(poller.poll).start(1/float(args['--poll']))
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
290
994
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
291 reactor.listenTCP(int(args['--port']),
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
292 cyclone.web.Application(
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
293 [
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
294 (r"/", Index),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
295 (r'/json', Json),
1145
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
296 (r'/graph', CycloneGraphHandler, {'masterGraph': masterGraph}),
0ba7a29a1886 wifi now uses PatchableGraph
drewp <drewp@bigasterisk.com>
parents: 1114
diff changeset
297 (r'/graph/events', CycloneGraphEventsHandler, {'masterGraph': masterGraph}),
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
298 (r'/table', Table),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
299 #(r'/activity', Activity),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
300 ],
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
301 wifi=wifi,
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
302 poller=poller,
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
303 mongo=mongo))
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
304 reactor.run()