Mercurial > code > home > repos > homeauto

annotate service/tomatoWifi/tomatoWifi.py @ 1114:e69599f47cf0

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
wifi write to influxdb Ignore-this: 3d5b0194b9db14e7e261a0852d24e42 darcs-hash:b0774c8cb9b49e99cab8f551a0c6e830d968432f
author drewp <drewp@bigasterisk.com>
date Fri, 16 Sep 2016 01:26:54 -0700
parents 3a9f20f881d1
children 70954bd426be
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
1 #!/usr/bin/python
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
2 """
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
3 scrape the tomato router status pages to see who's connected to the
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
4 wifi access points. Includes leases that aren't currently connected.
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
5
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
6 Returns:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
7 json listing (for magma page)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
8 rdf graph (for reasoning)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
9 activity stream, when we start saving history
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
10
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
11 Todo: this should be the one polling and writing to mongo, not entrancemusic
967
2489d111f4f1 whitespace
drewp <drewp@bigasterisk.com>
parents: 966
diff changeset
12
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
13 """
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
14 from __future__ import division
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
15 import sys, cyclone.web, json, traceback, time, pystache, datetime, logging
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
16 import web.utils
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
17 from cyclone.httpclient import fetch
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
18 sys.path.append("/home/drewp/projects/photo/lib/python2.7/site-packages")
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
19 from dateutil import tz
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
20 from twisted.internet import reactor, task
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
21 from twisted.internet.defer import inlineCallbacks
994
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
22 import docopt
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
23 from influxdb import InfluxDBClient
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
24 from pymongo import Connection, DESCENDING
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
25 from rdflib import Namespace, Literal, URIRef
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
26 sys.path.append("/my/site/magma")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
27 from stategraph import StateGraph
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
28 from wifi import Wifi
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
29
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
30 sys.path.append("/my/proj/homeauto/lib")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
31 from cycloneerr import PrettyErrorHandler
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
32 from logsetup import log
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
33
841
ddc2cdbfde50 rdflib and jsonlib api updates
drewp <drewp@bigasterisk.com>
parents: 806
diff changeset
34
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
35 DEV = Namespace("http://projects.bigasterisk.com/device/")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
36 ROOM = Namespace("http://projects.bigasterisk.com/room/")
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
37 reasoning = "http://bang:9071/"
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
38
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
39 class Index(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
40 def get(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
41
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
42 age = time.time() - self.settings.poller.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
43 if age > 10:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
44 raise ValueError("poll data is stale. age=%s" % age)
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
45
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
46 self.set_header("Content-Type", "text/html")
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
47 self.write(open("index.html").read())
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
48
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
49 def whenConnected(mongo, macThatIsNowConnected):
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
50 lastArrive = None
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
51 for ev in mongo.find({'address': macThatIsNowConnected.upper()},
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
52 sort=[('created', -1)],
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
53 max_scan=100000):
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
54 if ev['action'] == 'arrive':
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
55 lastArrive = ev
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
56 if ev['action'] == 'leave':
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
57 break
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
58 if lastArrive is None:
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
59 raise ValueError("no past arrivals")
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
60
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
61 return lastArrive['created']
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
62
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
63 def connectedAgoString(conn):
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
64 return web.utils.datestr(
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
65 conn.astimezone(tz.tzutc()).replace(tzinfo=None))
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
66
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
67 class Table(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
68 def get(self):
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
69 def rowDict(row):
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
70 row['cls'] = "signal" if row.get('connected') else "nosignal"
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
71 if 'name' not in row:
879
691aa1333073 handle wifi users with no clientHostname
drewp <drewp@bigasterisk.com>
parents: 867
diff changeset
72 row['name'] = row.get('clientHostname', '-')
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
73 if 'signal' not in row:
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
74 row['signal'] = 'yes' if row.get('connected') else 'no'
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
75
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
76 try:
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
77 conn = whenConnected(self.settings.mongo, row.get('mac', '??'))
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
78 row['connectedAgo'] = connectedAgoString(conn)
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
79 except ValueError:
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
80 row['connectedAgo'] = 'yes' if row.get('connected') else ''
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
81 row['router'] = row.get('ssid', '')
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
82 return row
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
83
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
84 self.set_header("Content-Type", "application/xhtml+xml")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
85 self.write(pystache.render(
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
86 open("table.mustache").read(),
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
87 dict(
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
88 rows=sorted(map(rowDict, self.settings.poller.lastAddrs),
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
89 key=lambda a: (not a.get('connected'),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
90 a.get('name'))))))
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
91
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
92
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
93 class Json(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
94 def get(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
95 self.set_header("Content-Type", "application/json")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
96 age = time.time() - self.settings.poller.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
97 if age > 10:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
98 raise ValueError("poll data is stale. age=%s" % age)
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
99 self.write(json.dumps({"wifi" : self.settings.poller.lastAddrs,
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
100 "dataAge" : age}))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
101
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
102 class GraphHandler(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
103 def get(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
104 g = StateGraph(ctx=DEV['wifi'])
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
105
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
106 # someday i may also record specific AP and their strength,
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
107 # for positioning. But many users just want to know that the
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
108 # device is connected to some bigasterisk AP.
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
109 aps = URIRef("http://bigasterisk.com/wifiAccessPoints")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
110 age = time.time() - self.settings.poller.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
111 if age > 10:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
112 raise ValueError("poll data is stale. age=%s" % age)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
113
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
114 for dev in self.settings.poller.lastAddrs:
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
115 if not dev.get('connected'):
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
116 continue
966
cce0107a78b4 scan dhcpd.leases to get more info about networked devices
drewp <drewp@bigasterisk.com>
parents: 879
diff changeset
117 uri = URIRef("http://bigasterisk.com/mac/%s" % dev['mac'].lower())
cce0107a78b4 scan dhcpd.leases to get more info about networked devices
drewp <drewp@bigasterisk.com>
parents: 879
diff changeset
118 g.add((uri, ROOM['macAddress'], Literal(dev['mac'].lower())))
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
119
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
120 g.add((uri, ROOM['connected'], aps))
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
121 if 'clientHostname' in dev:
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
122 g.add((uri, ROOM['wifiNetworkName'], Literal(dev['clientHostname'])))
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
123 if 'name' in dev:
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
124 g.add((uri, ROOM['deviceName'], Literal(dev['name'])))
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
125 if 'signal' in dev:
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
126 g.add((uri, ROOM['signalStrength'], Literal(dev['signal'])))
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
127 try:
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
128 conn = whenConnected(self.settings.mongo, dev['mac'])
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
129 except ValueError:
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
130 pass
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
131 else:
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
132 g.add((uri, ROOM['connectedAgo'],
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
133 Literal(connectedAgoString(conn))))
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
134 g.add((uri, ROOM['connected'], Literal(conn)))
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
135
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
136 self.set_header('Content-type', 'application/x-trig')
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
137 self.write(g.asTrig())
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
138
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
139 class Poller(object):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
140 def __init__(self, wifi, mongo):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
141 self.wifi = wifi
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
142 self.mongo = mongo
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
143 self.lastAddrs = []
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
144 self.lastWithSignal = []
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
145 self.lastPollTime = 0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
146
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
147 def assertCurrent(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
148 dt = time.time() - self.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
149 assert dt < 10, "last poll was %s sec ago" % dt
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
150
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
151 @inlineCallbacks
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
152 def poll(self):
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
153
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
154 connectedField = 'connected'
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
155 now = int(time.time())
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
156
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
157 # UVA mode:
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
158 addDhcpData = lambda *args: None
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
159
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
160 try:
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
161 newAddrs = yield self.wifi.getPresentMacAddrs()
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
162 addDhcpData(newAddrs)
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
163
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
164 newWithSignal = [a for a in newAddrs if a.get('connected')]
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
165
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
166 actions = self.computeActions(newWithSignal)
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
167 points = []
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
168 for action in actions:
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
169 log.info("action: %s", action)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
170 action['created'] = datetime.datetime.now(tz.gettz('UTC'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
171 mongo.save(action)
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
172 points.append(
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
173 self.influxPoint(now, action['address'].lower(),
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
174 1 if action['action'] == 'arrive' else 0))
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
175 try:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
176 self.doEntranceMusic(action)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
177 except Exception, e:
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
178 log.error("entrancemusic error: %r", e)
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
179
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
180 if now // 3600 > self.lastPollTime // 3600:
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
181 log.info('hourly writes')
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
182 for addr in newWithSignal:
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
183 points.append(self.influxPoint(now, addr['mac'].lower(), 1))
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
184
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
185 influx.write_points(points, time_precision='s')
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
186 self.lastWithSignal = newWithSignal
855
87111270eacf ping reasoning
drewp <drewp@bigasterisk.com>
parents: 841
diff changeset
187 if actions: # this doesn't currently include signal strength changes
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
188 fetch(reasoning + "immediateUpdate",
1084
3a9f20f881d1 immediate mode needs a PUT request
drewp <drewp@bigasterisk.com>
parents: 994
diff changeset
189 method='PUT',
857
1168b0cd90af fix fetch calls in tomatowifi
drewp <drewp@bigasterisk.com>
parents: 856
diff changeset
190 timeout=2,
1168b0cd90af fix fetch calls in tomatowifi
drewp <drewp@bigasterisk.com>
parents: 856
diff changeset
191 headers={'user-agent': ['tomatoWifi']}).addErrback(log.warn)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
192 self.lastAddrs = newAddrs
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
193 self.lastPollTime = now
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
194 except Exception, e:
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
195 log.error("poll error: %r\n%s", e, traceback.format_exc())
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
196
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
197 def influxPoint(self, now, address, value):
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
198 return {
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
199 'measurement': 'presence',
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
200 'tags': {'sensor': 'wifi', 'address': address,},
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
201 'fields': {'value': value},
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
202 'time': now,
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
203 }
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
204
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
205 def computeActions(self, newWithSignal):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
206 actions = []
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
207
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
208 def makeAction(addr, act):
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
209 d = dict(sensor="wifi",
980
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
210 address=addr.get('mac').upper(), # mongo data is legacy uppercase
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
211 name=addr.get('name'),
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
212 networkName=addr.get('clientHostname'),
2f1cb8b5950a rewrites for better graph export, removal of dhcp reader
drewp <drewp@bigasterisk.com>
parents: 967
diff changeset
213 action=act)
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
214 if act == 'arrive' and 'ip' in addr:
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
215 # this won't cover the possible case that you get on
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
216 # wifi but don't have an ip yet. We'll record an
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
217 # action with no ip and then never record your ip.
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
218 d['ip'] = addr['ip']
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
219 return d
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
220
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
221 for addr in newWithSignal:
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
222 if addr['mac'] not in [r['mac'] for r in self.lastWithSignal]:
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
223 actions.append(makeAction(addr, 'arrive'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
224
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
225 for addr in self.lastWithSignal:
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
226 if addr['mac'] not in [r['mac'] for r in newWithSignal]:
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
227 actions.append(makeAction(addr, 'leave'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
228
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
229 return actions
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
230
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
231
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
232 # these need to move out to their own service
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
233 def doEntranceMusic(self, action):
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
234 import restkit, jsonlib
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
235 dt = self.deltaSinceLastArrive(action['name'])
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
236 log.debug("dt=%s", dt)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
237 if dt > datetime.timedelta(hours=1):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
238 hub = restkit.Resource(
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
239 # PSHB not working yet; "http://bang:9030/"
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
240 "http://slash:9049/"
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
241 )
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
242 action = action.copy()
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
243 del action['created']
806
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
244 del action['_id']
85e50a597244 entrancemusic improve logging
drewp <drewp@bigasterisk.com>
parents: 805
diff changeset
245 log.info("post to %s", hub)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
246 hub.post("visitorNet", payload=jsonlib.dumps(action))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
247
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
248 def deltaSinceLastArrive(self, name):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
249 results = list(self.mongo.find({'name' : name}).sort('created',
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
250 DESCENDING).limit(1))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
251 if not results:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
252 return datetime.timedelta.max
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
253 now = datetime.datetime.now(tz.gettz('UTC'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
254 last = results[0]['created'].replace(tzinfo=tz.gettz('UTC'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
255 return now - last
856
4b386fc51325 rewrite tomatowifi from restkit to cyclone httpclient
drewp <drewp@bigasterisk.com>
parents: 855
diff changeset
256
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
257
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
258 if __name__ == '__main__':
994
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
259 args = docopt.docopt('''
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
260 Usage:
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
261 tomatoWifi [options]
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
262
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
263 Options:
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
264 -v, --verbose more logging
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
265 --port=<n> serve on port [default: 9070]
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
266 --poll=<freq> poll frequency [default: .2]
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
267 ''')
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
268 if args['--verbose']:
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
269 from twisted.python import log as twlog
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
270 twlog.startLogging(sys.stdout)
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
271 log.setLevel(10)
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
272 log.setLevel(logging.DEBUG)
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
273
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
274 mongo = Connection('bang', 27017, tz_aware=True)['visitor']['visitor']
1114
e69599f47cf0 wifi write to influxdb
drewp <drewp@bigasterisk.com>
parents: 1084
diff changeset
275 influx = InfluxDBClient('bang', 9060, 'root', 'root', 'main')
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
276
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
277 wifi = Wifi()
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
278 poller = Poller(wifi, mongo)
994
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
279 task.LoopingCall(poller.poll).start(1/float(args['--poll']))
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
280
994
e7af84a10adc switch to docopt
drewp <drewp@bigasterisk.com>
parents: 980
diff changeset
281 reactor.listenTCP(int(args['--port']),
867
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
282 cyclone.web.Application(
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
283 [
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
284 (r"/", Index),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
285 (r'/json', Json),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
286 (r'/graph', GraphHandler),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
287 (r'/table', Table),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
288 #(r'/activity', Activity),
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
289 ],
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
290 wifi=wifi,
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
291 poller=poller,
d9bc9a82dcca redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp <drewp@bigasterisk.com>
parents: 857
diff changeset
292 mongo=mongo))
805
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
293 reactor.run()