Mercurial > code > home > repos > homeauto

annotate service/wifi/wifi.py @ 1657:274bb6c04627

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
update currently-working tests
author drewp@bigasterisk.com
date Sun, 19 Sep 2021 13:23:50 -0700
parents a93fbf0d0daa
children f88ff1021ee0
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
1 """
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
2 scrape the tomato router status pages to see who's connected to the
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
3 wifi access points. Includes leases that aren't currently connected.
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
4
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
5 Returns:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
6 json listing (for magma page)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
7 rdf graph (for reasoning)
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
8 activity stream, when we start saving history
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
9
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
10 Todo: this should be the one polling and writing to mongo, not entrancemusic
162
bb70eaa45666 whitespace
drewp@bigasterisk.com
parents: 161
diff changeset
11
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
12 """
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
13 import sys, json, traceback, time, datetime, logging
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
14 from typing import List
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
15
51
d2842eedd56d rewrite tomatowifi from restkit to cyclone httpclient
drewp@bigasterisk.com
parents: 50
diff changeset
16 from cyclone.httpclient import fetch
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
17 from dateutil import tz
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
18 from influxdb import InfluxDBClient
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
19 from pymongo import MongoClient as Connection, DESCENDING
427
db031d9ec28e don't use 'connected' for time and for network. add rdf:type.
drewp@bigasterisk.com
parents: 423
diff changeset
20 from rdflib import Namespace, Literal, ConjunctiveGraph, RDF
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
21 from twisted.internet import reactor, task
62
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
22 from twisted.internet.defer import inlineCallbacks
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
23 import ago
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
24 import cyclone.web
189
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
25 import docopt
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
26 import pystache
383
8f5a16a55f64 various docker setups and build fixes
drewp@bigasterisk.com
parents: 340
diff changeset
27
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
28 from cycloneerr import PrettyErrorHandler
564
d0830394bf7b update build on service/wifi
drewp@bigasterisk.com
parents: 427
diff changeset
29 from standardservice.logsetup import log
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
30 from patchablegraph import PatchableGraph, CycloneGraphEventsHandler, CycloneGraphHandler
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
31 from scrape import Wifi, SeenNode
36
0ab069867c64 rdflib and jsonlib api updates
drewp@bigasterisk.com
parents: 1
diff changeset
32
422
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
33 AST = Namespace("http://bigasterisk.com/")
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
34 DEV = Namespace("http://projects.bigasterisk.com/device/")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
35 ROOM = Namespace("http://projects.bigasterisk.com/room/")
1
26a6cf58743d entrancemusic improve logging
drewp@bigasterisk.com
parents: 0
diff changeset
36
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
37 class Index(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
38 def get(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
39 age = time.time() - self.settings.poller.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
40 if age > 10:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
41 raise ValueError("poll data is stale. age=%s" % age)
659
ea9ed11598b0 remote suspend, wifi group arg
drewp@bigasterisk.com
parents: 566
diff changeset
42
175
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
43 self.set_header("Content-Type", "text/html")
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
44 self.write(open("index.html").read())
51
d2842eedd56d rewrite tomatowifi from restkit to cyclone httpclient
drewp@bigasterisk.com
parents: 50
diff changeset
45
175
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
46 def whenConnected(mongo, macThatIsNowConnected):
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
47 lastArrive = None
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
48 for ev in mongo.find({'address': macThatIsNowConnected.upper()},
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
49 sort=[('created', -1)],
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
50 max_scan=100000):
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
51 if ev['action'] == 'arrive':
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
52 lastArrive = ev
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
53 if ev['action'] == 'leave':
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
54 break
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
55 if lastArrive is None:
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
56 raise ValueError("no past arrivals")
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
57
175
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
58 return lastArrive['created']
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
59
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
60 def connectedAgoString(conn):
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
61 return ago.human(conn.astimezone(tz.tzutc()).replace(tzinfo=None))
659
ea9ed11598b0 remote suspend, wifi group arg
drewp@bigasterisk.com
parents: 566
diff changeset
62
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
63 class Table(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
64 def get(self):
62
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
65 def rowDict(row):
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
66 row['cls'] = "signal" if row.get('connected') else "nosignal"
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
67 if 'name' not in row:
74
bca6d6c63bdc handle wifi users with no clientHostname
drewp@bigasterisk.com
parents: 62
diff changeset
68 row['name'] = row.get('clientHostname', '-')
62
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
69 if 'signal' not in row:
175
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
70 row['signal'] = 'yes' if row.get('connected') else 'no'
62
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
71
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
72 try:
175
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
73 conn = whenConnected(self.settings.mongo, row.get('mac', '??'))
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
74 row['connectedAgo'] = connectedAgoString(conn)
62
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
75 except ValueError:
175
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
76 row['connectedAgo'] = 'yes' if row.get('connected') else ''
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
77 row['router'] = row.get('ssid', '')
62
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
78 return row
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
79
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
80 self.set_header("Content-Type", "application/xhtml+xml")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
81 self.write(pystache.render(
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
82 open("table.mustache").read(),
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
83 dict(
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
84 rows=sorted(map(rowDict, self.settings.poller.lastAddrs),
62
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
85 key=lambda a: (not a.get('connected'),
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
86 a.get('name'))))))
51
d2842eedd56d rewrite tomatowifi from restkit to cyclone httpclient
drewp@bigasterisk.com
parents: 50
diff changeset
87
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
88 class Json(PrettyErrorHandler, cyclone.web.RequestHandler):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
89 def get(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
90 self.set_header("Content-Type", "application/json")
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
91 age = time.time() - self.settings.poller.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
92 if age > 10:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
93 raise ValueError("poll data is stale. age=%s" % age)
51
d2842eedd56d rewrite tomatowifi from restkit to cyclone httpclient
drewp@bigasterisk.com
parents: 50
diff changeset
94 self.write(json.dumps({"wifi" : self.settings.poller.lastAddrs,
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
95 "dataAge" : age}))
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
96
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
97 class Poller(object):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
98 def __init__(self, wifi, mongo):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
99 self.wifi = wifi
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
100 self.mongo = mongo
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
101 self.lastAddrs = [] # List[SeenNode]
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
102 self.lastWithSignal = []
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
103 self.lastPollTime = 0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
104
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
105 def assertCurrent(self):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
106 dt = time.time() - self.lastPollTime
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
107 assert dt < 10, "last poll was %s sec ago" % dt
51
d2842eedd56d rewrite tomatowifi from restkit to cyclone httpclient
drewp@bigasterisk.com
parents: 50
diff changeset
108
d2842eedd56d rewrite tomatowifi from restkit to cyclone httpclient
drewp@bigasterisk.com
parents: 50
diff changeset
109 @inlineCallbacks
659
ea9ed11598b0 remote suspend, wifi group arg
drewp@bigasterisk.com
parents: 566
diff changeset
110 def poll(self):
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
111 try:
51
d2842eedd56d rewrite tomatowifi from restkit to cyclone httpclient
drewp@bigasterisk.com
parents: 50
diff changeset
112 newAddrs = yield self.wifi.getPresentMacAddrs()
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
113 self.onNodes(newAddrs)
421
47d7dd31bb2c port to py3
drewp@bigasterisk.com
parents: 420
diff changeset
114 except Exception as e:
175
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
115 log.error("poll error: %r\n%s", e, traceback.format_exc())
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
116
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
117 def onNodes(self, newAddrs: List[SeenNode]):
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
118 now = int(time.time())
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
119 newWithSignal = [a for a in newAddrs if a.connected]
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
120
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
121 actions = self.computeActions(newWithSignal)
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
122 points = []
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
123 for action in actions:
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
124 log.info("action: %s", action)
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
125 action['created'] = datetime.datetime.now(tz.gettz('UTC'))
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
126 mongo.save(action)
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
127 points.append(
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
128 self.influxPoint(now, action['address'].lower(),
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
129 1 if action['action'] == 'arrive' else 0))
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
130 if now // 3600 > self.lastPollTime // 3600:
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
131 log.info('hourly writes')
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
132 for addr in newWithSignal:
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
133 points.append(self.influxPoint(now, addr.mac.lower(), 1))
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
134
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
135 influx.write_points(points, time_precision='s')
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
136 self.lastWithSignal = newWithSignal
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
137 self.lastAddrs = newAddrs
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
138 self.lastPollTime = now
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
139
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
140 self.updateGraph(masterGraph)
566
c3d06c350e24 no more immediateUpdate since we push patch events now. and the code was broken for py3 anyway
drewp@bigasterisk.com
parents: 564
diff changeset
141
309
68c2a5f1d563 wifi write to influxdb
drewp@bigasterisk.com
parents: 279
diff changeset
142 def influxPoint(self, now, address, value):
68c2a5f1d563 wifi write to influxdb
drewp@bigasterisk.com
parents: 279
diff changeset
143 return {
68c2a5f1d563 wifi write to influxdb
drewp@bigasterisk.com
parents: 279
diff changeset
144 'measurement': 'presence',
68c2a5f1d563 wifi write to influxdb
drewp@bigasterisk.com
parents: 279
diff changeset
145 'tags': {'sensor': 'wifi', 'address': address,},
68c2a5f1d563 wifi write to influxdb
drewp@bigasterisk.com
parents: 279
diff changeset
146 'fields': {'value': value},
68c2a5f1d563 wifi write to influxdb
drewp@bigasterisk.com
parents: 279
diff changeset
147 'time': now,
68c2a5f1d563 wifi write to influxdb
drewp@bigasterisk.com
parents: 279
diff changeset
148 }
659
ea9ed11598b0 remote suspend, wifi group arg
drewp@bigasterisk.com
parents: 566
diff changeset
149
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
150 def computeActions(self, newWithSignal):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
151 actions = []
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
152
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
153 def makeAction(addr: SeenNode, act: str):
62
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
154 d = dict(sensor="wifi",
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
155 address=addr.mac.upper(), # mongo data is legacy uppercase
175
c81a451f9b26 rewrites for better graph export, removal of dhcp reader
drewp@bigasterisk.com
parents: 162
diff changeset
156 action=act)
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
157 if act == 'arrive':
62
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
158 # this won't cover the possible case that you get on
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
159 # wifi but don't have an ip yet. We'll record an
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
160 # action with no ip and then never record your ip.
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
161 d['ip'] = addr.ip
659
ea9ed11598b0 remote suspend, wifi group arg
drewp@bigasterisk.com
parents: 566
diff changeset
162 return d
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
163
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
164 for addr in newWithSignal:
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
165 if addr.mac not in [r.mac for r in self.lastWithSignal]:
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
166 actions.append(makeAction(addr, 'arrive'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
167
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
168 for addr in self.lastWithSignal:
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
169 if addr.mac not in [r.mac for r in newWithSignal]:
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
170 actions.append(makeAction(addr, 'leave'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
171
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
172 return actions
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
173
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
174 def deltaSinceLastArrive(self, name):
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
175 results = list(self.mongo.find({'name' : name}).sort('created',
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
176 DESCENDING).limit(1))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
177 if not results:
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
178 return datetime.timedelta.max
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
179 now = datetime.datetime.now(tz.gettz('UTC'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
180 last = results[0]['created'].replace(tzinfo=tz.gettz('UTC'))
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
181 return now - last
51
d2842eedd56d rewrite tomatowifi from restkit to cyclone httpclient
drewp@bigasterisk.com
parents: 50
diff changeset
182
340
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
183 def updateGraph(self, masterGraph):
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
184 g = ConjunctiveGraph()
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
185 ctx = DEV['wifi']
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
186
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
187 # someday i may also record specific AP and their strength,
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
188 # for positioning. But many users just want to know that the
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
189 # device is connected to some bigasterisk AP.
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
190 age = time.time() - self.lastPollTime
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
191 if age > 10:
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
192 raise ValueError("poll data is stale. age=%s" % age)
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
193
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
194 for dev in self.lastAddrs:
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
195 if not dev.connected:
340
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
196 continue
427
db031d9ec28e don't use 'connected' for time and for network. add rdf:type.
drewp@bigasterisk.com
parents: 423
diff changeset
197 g.add((dev.uri, RDF.type, ROOM['NetworkedDevice'], ctx))
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
198 g.add((dev.uri, ROOM['macAddress'], Literal(dev.mac), ctx))
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
199 g.add((dev.uri, ROOM['ipAddress'], Literal(dev.ip), ctx))
340
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
200
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
201 for s,p,o in dev.stmts:
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
202 g.add((s, p, o, ctx))
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
203
340
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
204 try:
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
205 conn = whenConnected(mongo, dev.mac)
340
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
206 except ValueError:
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
207 traceback.print_exc()
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
208 pass
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
209 else:
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
210 g.add((dev.uri, ROOM['connectedAgo'],
340
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
211 Literal(connectedAgoString(conn)), ctx))
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
212 g.add((dev.uri, ROOM['connected'], Literal(conn), ctx))
340
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
213 masterGraph.setToGraph(g)
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
214
659
ea9ed11598b0 remote suspend, wifi group arg
drewp@bigasterisk.com
parents: 566
diff changeset
215 class RemoteSuspend(PrettyErrorHandler, cyclone.web.RequestHandler):
ea9ed11598b0 remote suspend, wifi group arg
drewp@bigasterisk.com
parents: 566
diff changeset
216 def post(self):
ea9ed11598b0 remote suspend, wifi group arg
drewp@bigasterisk.com
parents: 566
diff changeset
217 # windows is running shutter (https://www.den4b.com/products/shutter)
ea9ed11598b0 remote suspend, wifi group arg
drewp@bigasterisk.com
parents: 566
diff changeset
218 fetch('http://DESKTOP-GOU4AC4:8011/action', postdata={'id': 'Sleep'})
ea9ed11598b0 remote suspend, wifi group arg
drewp@bigasterisk.com
parents: 566
diff changeset
219
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
220
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
221 if __name__ == '__main__':
189
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
222 args = docopt.docopt('''
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
223 Usage:
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
224 wifi.py [options]
189
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
225
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
226 Options:
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
227 -v, --verbose more logging
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
228 --port=<n> serve on port [default: 9070]
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
229 --poll=<freq> poll frequency [default: .2]
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
230 ''')
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
231 if args['--verbose']:
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
232 from twisted.python import log as twlog
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
233 twlog.startLogging(sys.stdout)
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
234 log.setLevel(10)
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
235 log.setLevel(logging.DEBUG)
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
236
62
f8cc3d1baa85 redo wifi scraper to work with zyxel router report page too. add last connected time (from mongo) to web table
drewp@bigasterisk.com
parents: 52
diff changeset
237 mongo = Connection('bang', 27017, tz_aware=True)['visitor']['visitor']
309
68c2a5f1d563 wifi write to influxdb
drewp@bigasterisk.com
parents: 279
diff changeset
238 influx = InfluxDBClient('bang', 9060, 'root', 'root', 'main')
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
239
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
240 config = ConjunctiveGraph()
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
241 config.parse(open('private_config.n3'), format='n3')
659
ea9ed11598b0 remote suspend, wifi group arg
drewp@bigasterisk.com
parents: 566
diff changeset
242
340
70954bd426be wifi now uses PatchableGraph
drewp@bigasterisk.com
parents: 309
diff changeset
243 masterGraph = PatchableGraph()
423
e0703c7824e9 very big rewrite. py3; orbi-only for now; n3 config file; delete or move out dead code
drewp@bigasterisk.com
parents: 422
diff changeset
244 wifi = Wifi(config)
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
245 poller = Poller(wifi, mongo)
189
fc184ea1416d switch to docopt
drewp@bigasterisk.com
parents: 175
diff changeset
246 task.LoopingCall(poller.poll).start(1/float(args['--poll']))
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
247
422
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
248 reactor.listenTCP(
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
249 int(args['--port']),
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
250 cyclone.web.Application(
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
251 [
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
252 (r"/", Index),
674
2b9865bf1737 streamed-graph finally imports and builds. polymer bindings don't work in a <dom-bind> though
drewp@bigasterisk.com
parents: 662
diff changeset
253 (r"/build/(bundle\.js)", cyclone.web.StaticFileHandler, {"path": 'build'}),
422
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
254 (r'/json', Json),
722
a93fbf0d0daa dep updates; graph url renames; and other build updates
drewp@bigasterisk.com
parents: 674
diff changeset
255 (r'/graph/wifi', CycloneGraphHandler, {'masterGraph': masterGraph}),
a93fbf0d0daa dep updates; graph url renames; and other build updates
drewp@bigasterisk.com
parents: 674
diff changeset
256 (r'/graph/wifi/events', CycloneGraphEventsHandler, {'masterGraph': masterGraph}),
422
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
257 (r'/table', Table),
659
ea9ed11598b0 remote suspend, wifi group arg
drewp@bigasterisk.com
parents: 566
diff changeset
258 (r'/remoteSuspend', RemoteSuspend),
422
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
259 #(r'/activity', Activity),
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
260 ],
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
261 wifi=wifi,
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
262 poller=poller,
19460b3f7baf factor out some URI generation
drewp@bigasterisk.com
parents: 421
diff changeset
263 mongo=mongo))
0
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
264 reactor.run()