Mercurial > code > home > repos > homeauto
annotate service/piNode/piNode.py @ 591:1398f4ec01a5
pinode use prettyerrorhandler
Ignore-this: 20e96013aa365fd180774ffb5b558245
author | drewp@bigasterisk.com |
---|---|
date | Sat, 06 Jul 2019 00:49:12 -0700 |
parents | f397ec8bd13d |
children | 1249dcc657b8 |
rev | line source |
---|---|
182 | 1 from __future__ import division |
552
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
2 import sys, logging, socket, json, time, pkg_resources |
182 | 3 import cyclone.web |
251
254df9f881a6
start sending oneshot events from some devices
drewp@bigasterisk.com
parents:
243
diff
changeset
|
4 from cyclone.httpclient import fetch |
226 | 5 from rdflib import Namespace, URIRef, Literal, Graph, RDF, ConjunctiveGraph |
184
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
6 from rdflib.parser import StringInputSource |
182 | 7 from twisted.internet import reactor, task |
589
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
8 from twisted.internet.defer import inlineCallbacks, maybeDeferred, gatherResults |
347 | 9 from twisted.internet.threads import deferToThread |
182 | 10 from docopt import docopt |
347 | 11 import etcd3 |
466
1122016d16eb
stats handler, events output fix, build updaets
drewp@bigasterisk.com
parents:
408
diff
changeset
|
12 from greplin import scales |
1122016d16eb
stats handler, events output fix, build updaets
drewp@bigasterisk.com
parents:
408
diff
changeset
|
13 from greplin.scales.cyclonehandler import StatsHandler |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
14 |
182 | 15 logging.basicConfig(level=logging.DEBUG) |
331
a94f2a522d41
build and import updates for rdfdb, etc
drewp@bigasterisk.com
parents:
325
diff
changeset
|
16 |
226 | 17 from patchablegraph import PatchableGraph, CycloneGraphHandler, CycloneGraphEventsHandler |
589
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
18 from cycloneerr import PrettyErrorHandler |
338
f64e20d3407e
RgbPixelsAnimation and docker build updates
drewp@bigasterisk.com
parents:
331
diff
changeset
|
19 |
f64e20d3407e
RgbPixelsAnimation and docker build updates
drewp@bigasterisk.com
parents:
331
diff
changeset
|
20 from rdfdb.rdflibpatch import inContext |
f64e20d3407e
RgbPixelsAnimation and docker build updates
drewp@bigasterisk.com
parents:
331
diff
changeset
|
21 from rdfdb.patch import Patch |
f64e20d3407e
RgbPixelsAnimation and docker build updates
drewp@bigasterisk.com
parents:
331
diff
changeset
|
22 |
183
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
23 try: |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
24 import pigpio |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
25 except ImportError: |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
26 class pigpio(object): |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
27 @staticmethod |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
28 def pi(): |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
29 return None |
182 | 30 |
31 import devices | |
293 | 32 from export_to_influxdb import InfluxExporter |
221
666e0e756ce6
piNode support for temp sensors. proper hostname lookup
drewp@bigasterisk.com
parents:
220
diff
changeset
|
33 |
182 | 34 log = logging.getLogger() |
35 logging.getLogger('serial').setLevel(logging.WARN) | |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
36 |
182 | 37 ROOM = Namespace('http://projects.bigasterisk.com/room/') |
38 HOST = Namespace('http://bigasterisk.com/ruler/host/') | |
39 | |
40 hostname = socket.gethostname() | |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
41 CTX = ROOM['pi/%s' % hostname] |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
42 |
466
1122016d16eb
stats handler, events output fix, build updaets
drewp@bigasterisk.com
parents:
408
diff
changeset
|
43 STATS = scales.collection('/root', |
552
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
44 scales.PmfStat('configReread'), |
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
45 scales.IntStat('pollException'), |
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
46 scales.PmfStat('boardPoll'), |
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
47 scales.PmfStat('sendOneshot'), |
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
48 scales.PmfStat('outputStatements'), |
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
49 |
466
1122016d16eb
stats handler, events output fix, build updaets
drewp@bigasterisk.com
parents:
408
diff
changeset
|
50 ) |
258 | 51 def patchRandid(): |
52 """ | |
53 I'm concerned urandom is slow on raspberry pi, and I'm adding to | |
54 graphs a lot. Unclear what the ordered return values might do to | |
55 the balancing of the graph. | |
56 """ | |
57 _id_serial = [1000] | |
58 def randid(): | |
59 _id_serial[0] += 1 | |
60 return _id_serial[0] | |
61 import rdflib.plugins.memory | |
62 rdflib.plugins.memory.randid = randid | |
63 patchRandid() | |
64 | |
183
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
65 class Config(object): |
408
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
66 def __init__(self, masterGraph, hubHost): |
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
67 self.etcd = etcd3.client(host=hubHost, port=9022) |
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
68 |
347 | 69 self.masterGraph = masterGraph |
408
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
70 self.hubHost = hubHost |
347 | 71 self.configGraph = ConjunctiveGraph() |
72 self.boards = [] | |
73 self.etcPrefix = 'pi/' | |
74 | |
75 self.reread() | |
76 | |
77 deferToThread(self.watchEtcd) | |
78 | |
79 def watchEtcd(self): | |
408
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
80 events, cancel = self.etcd.watch_prefix(self.etcPrefix) |
347 | 81 reactor.addSystemEventTrigger('before', 'shutdown', cancel) |
82 for ev in events: | |
83 log.info('%s changed', ev.key) | |
84 reactor.callFromThread(self.configChanged) | |
85 | |
86 def configChanged(self): | |
87 self.cancelRead() | |
88 self.rereadLater = reactor.callLater(.1, self.reread) | |
89 | |
90 def cancelRead(self): | |
91 if getattr(self, 'rereadLater', None): | |
92 self.rereadLater.cancel() | |
93 self.rereadLater = None | |
552
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
94 |
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
95 @STATS.configReread.time() |
347 | 96 def reread(self): |
97 self.rereadLater = None | |
183
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
98 log.info('read config') |
347 | 99 self.configGraph = ConjunctiveGraph() |
408
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
100 for v, md in self.etcd.get_prefix(self.etcPrefix): |
347 | 101 log.info(' read file %r', md.key) |
102 self.configGraph.parse(StringInputSource(v), format='n3') | |
103 self.configGraph.bind('', ROOM) | |
104 self.configGraph.bind('rdf', RDF) | |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
105 # config graph is too noisy; maybe make it a separate resource |
347 | 106 #masterGraph.patch(Patch(addGraph=self.configGraph)) |
107 self.setupBoards() | |
108 | |
109 def setupBoards(self): | |
110 thisHost = Literal(hostname) | |
111 for row in self.configGraph.query( | |
112 'SELECT ?board WHERE { ?board a :PiBoard; :hostname ?h }', | |
113 initBindings=dict(h=thisHost)): | |
114 thisBoard = row.board | |
115 break | |
116 else: | |
117 log.warn("config had no board for :hostname %s. Waiting for config update." % | |
118 thisHost) | |
119 self.boards = [] | |
120 return | |
121 | |
122 log.info("found config for board %r" % thisBoard) | |
408
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
123 self.boards = [Board(self.configGraph, self.masterGraph, thisBoard, self.hubHost)] |
347 | 124 self.boards[0].startPolling() |
125 | |
182 | 126 |
127 class Board(object): | |
128 """similar to arduinoNode.Board but without the communications stuff""" | |
408
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
129 def __init__(self, graph, masterGraph, uri, hubHost): |
182 | 130 self.graph, self.uri = graph, uri |
408
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
131 self.hubHost = hubHost |
226 | 132 self.masterGraph = masterGraph |
349
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
133 self.masterGraph.setToGraph(self.staticStmts()) |
182 | 134 self.pi = pigpio.pi() |
135 self._devs = devices.makeDevices(graph, self.uri, self.pi) | |
183
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
136 log.debug('found %s devices', len(self._devs)) |
182 | 137 self._statementsFromInputs = {} # input device uri: latest statements |
243
141079644c45
piNode poll switches much faster. mirror the logic in arduinoNode though vari-rate poll is not supported yet
drewp@bigasterisk.com
parents:
233
diff
changeset
|
138 self._lastPollTime = {} # input device uri: time() |
293 | 139 self._influx = InfluxExporter(self.graph) |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
140 for d in self._devs: |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
141 self.syncMasterGraphToHostStatements(d) |
243
141079644c45
piNode poll switches much faster. mirror the logic in arduinoNode though vari-rate poll is not supported yet
drewp@bigasterisk.com
parents:
233
diff
changeset
|
142 |
182 | 143 def startPolling(self): |
243
141079644c45
piNode poll switches much faster. mirror the logic in arduinoNode though vari-rate poll is not supported yet
drewp@bigasterisk.com
parents:
233
diff
changeset
|
144 task.LoopingCall(self._poll).start(.05) |
182 | 145 |
552
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
146 @STATS.boardPoll.time() # not differentiating multiple boards here |
182 | 147 def _poll(self): |
271
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
148 try: |
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
149 self._pollMaybeError() |
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
150 except Exception: |
552
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
151 STATS.pollException += 1 |
271
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
152 log.exception("During poll:") |
589
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
153 |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
154 |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
155 @inlineCallbacks |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
156 def _pollOneDev(self, i, pollTime): |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
157 now = time.time() |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
158 if (hasattr(i, 'pollPeriod') and |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
159 self._lastPollTime.get(i.uri, 0) + i.pollPeriod > now): |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
160 return |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
161 #need something like: |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
162 # with i.pollTiming.time(): |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
163 new = yield maybeDeferred(i.poll) |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
164 pollTime[i.uri] = time.time() - now |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
165 if isinstance(new, dict): # new style |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
166 oneshot = new['oneshot'] |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
167 new = new['latest'] |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
168 else: |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
169 oneshot = None |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
170 |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
171 self._updateMasterWithNewPollStatements(i.uri, new) |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
172 |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
173 if oneshot: |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
174 self._sendOneshot(oneshot) |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
175 self._lastPollTime[i.uri] = now |
271
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
176 |
589
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
177 @inlineCallbacks |
271
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
178 def _pollMaybeError(self): |
304
e7cbf250188a
influx output, fade support, switch to Adafruit_DHT, start of Lcd8544
drewp@bigasterisk.com
parents:
293
diff
changeset
|
179 pollTime = {} # uri: sec |
589
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
180 start = time.time() |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
181 yield gatherResults([self._pollOneDev(i, pollTime) |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
182 for i in self._devs], consumeErrors=True) |
251
254df9f881a6
start sending oneshot events from some devices
drewp@bigasterisk.com
parents:
243
diff
changeset
|
183 |
304
e7cbf250188a
influx output, fade support, switch to Adafruit_DHT, start of Lcd8544
drewp@bigasterisk.com
parents:
293
diff
changeset
|
184 if log.isEnabledFor(logging.DEBUG): |
e7cbf250188a
influx output, fade support, switch to Adafruit_DHT, start of Lcd8544
drewp@bigasterisk.com
parents:
293
diff
changeset
|
185 log.debug('poll times:') |
e7cbf250188a
influx output, fade support, switch to Adafruit_DHT, start of Lcd8544
drewp@bigasterisk.com
parents:
293
diff
changeset
|
186 for u, s in sorted(pollTime.items()): |
e7cbf250188a
influx output, fade support, switch to Adafruit_DHT, start of Lcd8544
drewp@bigasterisk.com
parents:
293
diff
changeset
|
187 log.debug(" %.4f ms %s", s * 1000, u) |
589
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
188 log.debug('total poll time: %f ms done in %f ms elapsed', |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
189 sum(pollTime.values()) * 1000, |
f397ec8bd13d
pinode devs can now poll in parallel (within one poll step). doesn't help much.
drewp@bigasterisk.com
parents:
553
diff
changeset
|
190 (time.time() - start) * 1000) |
338
f64e20d3407e
RgbPixelsAnimation and docker build updates
drewp@bigasterisk.com
parents:
331
diff
changeset
|
191 |
f64e20d3407e
RgbPixelsAnimation and docker build updates
drewp@bigasterisk.com
parents:
331
diff
changeset
|
192 pollResults = map(set, self._statementsFromInputs.values()) |
f64e20d3407e
RgbPixelsAnimation and docker build updates
drewp@bigasterisk.com
parents:
331
diff
changeset
|
193 if pollResults: |
f64e20d3407e
RgbPixelsAnimation and docker build updates
drewp@bigasterisk.com
parents:
331
diff
changeset
|
194 self._influx.exportToInflux(set.union(*pollResults)) |
221
666e0e756ce6
piNode support for temp sensors. proper hostname lookup
drewp@bigasterisk.com
parents:
220
diff
changeset
|
195 |
349
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
196 def _updateMasterWithNewPollStatements(self, dev, new): |
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
197 prev = self._statementsFromInputs.get(dev, set()) |
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
198 |
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
199 # it's important that quads from different devices |
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
200 # don't clash, since that can lead to inconsistent |
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
201 # patches (e.g. |
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
202 # dev1 changes value from 1 to 2; |
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
203 # dev2 changes value from 2 to 3; |
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
204 # dev1 changes from 2 to 4 but this patch will |
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
205 # fail since the '2' statement is gone) |
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
206 self.masterGraph.patch(Patch.fromDiff(inContext(prev, dev), |
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
207 inContext(new, dev))) |
88bd46f4e28c
more robust about not accumulating old stmts
drewp@bigasterisk.com
parents:
347
diff
changeset
|
208 self._statementsFromInputs[dev] = new |
552
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
209 |
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
210 @STATS.sendOneshot.time() |
251
254df9f881a6
start sending oneshot events from some devices
drewp@bigasterisk.com
parents:
243
diff
changeset
|
211 def _sendOneshot(self, oneshot): |
254df9f881a6
start sending oneshot events from some devices
drewp@bigasterisk.com
parents:
243
diff
changeset
|
212 body = (' '.join('%s %s %s .' % (s.n3(), p.n3(), o.n3()) |
254df9f881a6
start sending oneshot events from some devices
drewp@bigasterisk.com
parents:
243
diff
changeset
|
213 for s,p,o in oneshot)).encode('utf8') |
408
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
214 url = 'http://%s:9071/oneShot' % self.hubHost |
271
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
215 d = fetch(method='POST', |
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
216 url=url, |
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
217 headers={'Content-Type': ['text/n3']}, |
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
218 postdata=body, |
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
219 timeout=5) |
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
220 def err(e): |
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
221 log.info('oneshot post to %r failed: %s', |
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
222 url, e.getErrorMessage()) |
e7a30f72536a
rewrite oneShotPost to ease debugging. add try-block around polling
drewp@bigasterisk.com
parents:
269
diff
changeset
|
223 d.addErrback(err) |
251
254df9f881a6
start sending oneshot events from some devices
drewp@bigasterisk.com
parents:
243
diff
changeset
|
224 |
552
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
225 @STATS.outputStatements.time() |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
226 def outputStatements(self, stmts): |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
227 unused = set(stmts) |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
228 for dev in self._devs: |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
229 stmtsForDev = [] |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
230 for pat in dev.outputPatterns(): |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
231 if [term is None for term in pat] != [False, False, True]: |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
232 raise NotImplementedError |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
233 for stmt in stmts: |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
234 if stmt[:2] == pat[:2]: |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
235 stmtsForDev.append(stmt) |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
236 unused.discard(stmt) |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
237 if stmtsForDev: |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
238 log.info("output goes to action handler for %s" % dev.uri) |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
239 dev.sendOutput(stmtsForDev) |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
240 |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
241 # Dev *could* change hostStatements at any time, and |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
242 # we're not currently tracking that, but the usual is |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
243 # to change them in response to sendOutput so this |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
244 # should be good enough. The right answer is to give |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
245 # each dev the masterGraph for it to write to. |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
246 self.syncMasterGraphToHostStatements(dev) |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
247 log.info("output and masterGraph sync complete") |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
248 if unused: |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
249 log.info("Board %s doesn't care about these statements:", self.uri) |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
250 for s in unused: |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
251 log.warn("%r", s) |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
252 |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
253 def syncMasterGraphToHostStatements(self, dev): |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
254 hostStmtCtx = URIRef(dev.uri + '/host') |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
255 newQuads = inContext(dev.hostStatements(), hostStmtCtx) |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
256 p = self.masterGraph.patchSubgraph(hostStmtCtx, newQuads) |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
257 log.debug("patch master with these host stmts %s", p) |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
258 |
226 | 259 def staticStmts(self): |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
260 return [(HOST[hostname], ROOM['connectedTo'], self.uri, CTX)] |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
261 |
183
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
262 def description(self): |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
263 """for web page""" |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
264 return { |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
265 'uri': self.uri, |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
266 'devices': [d.description() for d in self._devs], |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
267 'graph': 'http://sticker:9059/graph', #todo |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
268 } |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
269 |
591 | 270 class Dot(PrettyErrorHandler, cyclone.web.RequestHandler): |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
271 def get(self): |
347 | 272 configGraph = self.settings.config.configGraph |
273 dot = dotrender.render(configGraph, self.settings.config.boards) | |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
274 self.write(dot) |
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
275 |
184
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
276 def rdfGraphBody(body, headers): |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
277 g = Graph() |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
278 g.parse(StringInputSource(body), format='nt') |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
279 return g |
183
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
280 |
591 | 281 class OutputPage(PrettyErrorHandler, cyclone.web.RequestHandler): |
182 | 282 def put(self): |
184
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
283 arg = self.request.arguments |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
284 if arg.get('s') and arg.get('p'): |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
285 subj = URIRef(arg['s'][-1]) |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
286 pred = URIRef(arg['p'][-1]) |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
287 turtleLiteral = self.request.body |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
288 try: |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
289 obj = Literal(float(turtleLiteral)) |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
290 except ValueError: |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
291 obj = Literal(turtleLiteral) |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
292 stmt = (subj, pred, obj) |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
293 else: |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
294 g = rdfGraphBody(self.request.body, self.request.headers) |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
295 assert len(g) == 1, len(g) |
e052416a2290
piNode allow nt graphs as the body of a PUT /output
drewp@bigasterisk.com
parents:
183
diff
changeset
|
296 stmt = g.triples((None, None, None)).next() |
182 | 297 |
347 | 298 for b in self.settings.config.boards: |
299 b.outputStatements([stmt]) | |
182 | 300 |
591 | 301 class Boards(PrettyErrorHandler, cyclone.web.RequestHandler): |
183
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
302 def get(self): |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
303 self.set_header('Content-type', 'application/json') |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
304 self.write(json.dumps({ |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
305 'host': hostname, |
347 | 306 'boards': [b.description() for b in self.settings.config.boards] |
183
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
307 }, indent=2)) |
634d6e477953
get piNode working, for motionsensor at least
drewp@bigasterisk.com
parents:
182
diff
changeset
|
308 |
182 | 309 def main(): |
310 arg = docopt(""" | |
311 Usage: piNode.py [options] | |
312 | |
408
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
313 -v Verbose |
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
314 --ow Just report onewire device URIs and readings, then exit. |
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
315 --hub=HOST Hostname for etc3 and oneshot posts. [default: bang.vpn-home.bigasterisk.com] |
182 | 316 """) |
317 log.setLevel(logging.WARN) | |
318 if arg['-v']: | |
319 from twisted.python import log as twlog | |
320 twlog.startLogging(sys.stdout) | |
321 | |
322 log.setLevel(logging.DEBUG) | |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
323 |
408
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
324 if arg['--ow']: |
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
325 log.setLevel(logging.INFO) |
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
326 for stmt in devices.OneWire().poll(): |
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
327 print stmt |
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
328 return |
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
329 |
226 | 330 masterGraph = PatchableGraph() |
408
0787cd64ecf8
cmdline flag on piNode to pick hub host
drewp@bigasterisk.com
parents:
382
diff
changeset
|
331 config = Config(masterGraph, arg['--hub']) |
182 | 332 |
552
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
333 static = pkg_resources.resource_filename('homeauto_anynode', 'static/') |
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
334 |
182 | 335 reactor.listenTCP(9059, cyclone.web.Application([ |
336 (r"/()", cyclone.web.StaticFileHandler, { | |
552
98384c3ccd33
update build and deps to use invoke and to use new lib layout, plus more stats collection
drewp@bigasterisk.com
parents:
466
diff
changeset
|
337 "path": static, "default_filename": "index.html"}), |
553 | 338 (r'/static/(.*)', cyclone.web.StaticFileHandler, {"path": static}), |
466
1122016d16eb
stats handler, events output fix, build updaets
drewp@bigasterisk.com
parents:
408
diff
changeset
|
339 (r'/stats/(.*)', StatsHandler, {'serverName': 'piNode'}), |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
340 (r'/boards', Boards), |
226 | 341 (r"/graph", CycloneGraphHandler, {'masterGraph': masterGraph}), |
342 (r"/graph/events", CycloneGraphEventsHandler, {'masterGraph': masterGraph}), | |
182 | 343 (r'/output', OutputPage), |
233
4ebb5cc30002
server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents:
230
diff
changeset
|
344 (r'/dot', Dot), |
466
1122016d16eb
stats handler, events output fix, build updaets
drewp@bigasterisk.com
parents:
408
diff
changeset
|
345 ], config=config, debug=arg['-v']), interface='::') |
258 | 346 log.warn('serving on 9059') |
182 | 347 reactor.run() |
348 | |
349 main() |