Mercurial > code > home > repos > homeauto

annotate service/reasoning/reasoning.py @ 1555:606bb60a5e5c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot Ignore-this: 263923a8d12db2173017bc9dbfc638ba darcs-hash:cedac5a49c9cbd0b8c34c80efa9282853fda2cc5
author drewp <drewp@bigasterisk.com>
date Thu, 13 Feb 2020 23:00:06 -0800
parents 0da337780f22
children f3f667769aef
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
1 #!bin/python
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
2 """
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
3 Graph consists of:
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
4 input/* (read at startup)
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
5 webinput/* (new files are noticed in here)
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
6 any number of remote graphs, specified in the other graph as objects of (:reasoning, :source, *), reread constantly
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
7
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
8 gather subgraphs from various services, run them through a rules
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
9 engine, and make http requests with the conclusions.
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
10
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
11 E.g. 'when drew's phone is near the house, and someone is awake,
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
12 unlock the door when the door's motion sensor is activated'
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
13
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
14 When do we gather? The services should be able to trigger us, perhaps
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
15 with PSHB, that their graph has changed.
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
16 """
1108
8caf62030955 reasoning uses sse_collector
drewp <drewp@bigasterisk.com>
parents: 1093
diff changeset
17 from crochet import no_setup
8caf62030955 reasoning uses sse_collector
drewp <drewp@bigasterisk.com>
parents: 1093
diff changeset
18 no_setup()
8caf62030955 reasoning uses sse_collector
drewp <drewp@bigasterisk.com>
parents: 1093
diff changeset
19
1085
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
20 import json, time, traceback, sys
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
21 from logging import getLogger, DEBUG, WARN
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
22
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
23 from colorlog import ColoredFormatter
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
24 from docopt import docopt
1555
606bb60a5e5c something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot
drewp <drewp@bigasterisk.com>
parents: 1523
diff changeset
25 from rdflib import Namespace, Literal, RDF, Graph, URIRef
606bb60a5e5c something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot
drewp <drewp@bigasterisk.com>
parents: 1523
diff changeset
26 from twisted.internet import reactor, defer
1085
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
27 import cyclone.web, cyclone.websocket
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
28
1092
54de5144900d switch from evtiming to greplin.scales. Optimize rules reader to reuse previous data (400ms -> 0.6ms)
drewp <drewp@bigasterisk.com>
parents: 1091
diff changeset
29 from greplin import scales
54de5144900d switch from evtiming to greplin.scales. Optimize rules reader to reuse previous data (400ms -> 0.6ms)
drewp <drewp@bigasterisk.com>
parents: 1091
diff changeset
30 from greplin.scales.cyclonehandler import StatsHandler
54de5144900d switch from evtiming to greplin.scales. Optimize rules reader to reuse previous data (400ms -> 0.6ms)
drewp <drewp@bigasterisk.com>
parents: 1091
diff changeset
31
1089
cb7fa2f30df9 rules become simple-looking again; fix the ambiguity in memory after loading them.
drewp <drewp@bigasterisk.com>
parents: 1088
diff changeset
32 from inference import infer, readRules
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
33 from actions import Actions, PutOutputsTable
1080
4d16fa39d54a refactor inputgraph
drewp <drewp@bigasterisk.com>
parents: 1070
diff changeset
34 from inputgraph import InputGraph
1089
cb7fa2f30df9 rules become simple-looking again; fix the ambiguity in memory after loading them.
drewp <drewp@bigasterisk.com>
parents: 1088
diff changeset
35 from escapeoutputstatements import unquoteOutputStatements
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
36
1403
69b3f5b3aeda comments and logging
drewp <drewp@bigasterisk.com>
parents: 1374
diff changeset
37 from standardservice.logsetup import log, verboseLogging
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
38 from patchablegraph import PatchableGraph, CycloneGraphHandler, CycloneGraphEventsHandler
1085
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
39
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
40
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
41 ROOM = Namespace("http://projects.bigasterisk.com/room/")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
42 DEV = Namespace("http://projects.bigasterisk.com/device/")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
43
1086
6ab5238fc049 refactor
drewp <drewp@bigasterisk.com>
parents: 1085
diff changeset
44 NS = {'': ROOM, 'dev': DEV}
1088
0f6128740000 fix input/output statement ambiguity problem by allowing quoted output statements
drewp <drewp@bigasterisk.com>
parents: 1087
diff changeset
45
1092
54de5144900d switch from evtiming to greplin.scales. Optimize rules reader to reuse previous data (400ms -> 0.6ms)
drewp <drewp@bigasterisk.com>
parents: 1091
diff changeset
46 STATS = scales.collection('/web',
1197
d8acab2b01f5 mqtt has two devices now. various older cleanups.
drewp <drewp@bigasterisk.com>
parents: 1156
diff changeset
47 scales.PmfStat('graphChanged'),
d8acab2b01f5 mqtt has two devices now. various older cleanups.
drewp <drewp@bigasterisk.com>
parents: 1156
diff changeset
48 scales.PmfStat('updateRules'),
d8acab2b01f5 mqtt has two devices now. various older cleanups.
drewp <drewp@bigasterisk.com>
parents: 1156
diff changeset
49 )
1092
54de5144900d switch from evtiming to greplin.scales. Optimize rules reader to reuse previous data (400ms -> 0.6ms)
drewp <drewp@bigasterisk.com>
parents: 1091
diff changeset
50
1555
606bb60a5e5c something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot
drewp <drewp@bigasterisk.com>
parents: 1523
diff changeset
51 def ntStatement(stmt):
606bb60a5e5c something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot
drewp <drewp@bigasterisk.com>
parents: 1523
diff changeset
52 def compact(u):
606bb60a5e5c something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot
drewp <drewp@bigasterisk.com>
parents: 1523
diff changeset
53 if isinstance(u, URIRef) and u.startswith(ROOM):
606bb60a5e5c something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot
drewp <drewp@bigasterisk.com>
parents: 1523
diff changeset
54 return 'room:' + u[len(ROOM):]
606bb60a5e5c something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot
drewp <drewp@bigasterisk.com>
parents: 1523
diff changeset
55 return u.n3()
606bb60a5e5c something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot
drewp <drewp@bigasterisk.com>
parents: 1523
diff changeset
56 return '%s %s %s .' % (compact(stmt[0]), compact(stmt[1]), compact(stmt[2]))
606bb60a5e5c something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot
drewp <drewp@bigasterisk.com>
parents: 1523
diff changeset
57
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
58 class Reasoning(object):
1411
21d0cd98ef7a mock output mode
drewp <drewp@bigasterisk.com>
parents: 1403
diff changeset
59 def __init__(self, mockOutput=False):
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
60 self.prevGraph = None
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
61
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
62 self.rulesN3 = "(not read yet)"
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
63 self.inferred = Graph() # gets replaced in each graphChanged call
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
64 self.outputGraph = PatchableGraph() # copy of inferred, for now
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
65
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
66 self.inputGraph = InputGraph([], self.graphChanged)
1555
606bb60a5e5c something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot
drewp <drewp@bigasterisk.com>
parents: 1523
diff changeset
67 self.actions = Actions(self.inputGraph, sendToLiveClients, mockOutput=mockOutput)
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
68 self.inputGraph.updateFileData()
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
69
1197
d8acab2b01f5 mqtt has two devices now. various older cleanups.
drewp <drewp@bigasterisk.com>
parents: 1156
diff changeset
70 @STATS.updateRules.time()
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
71 def updateRules(self):
1089
cb7fa2f30df9 rules become simple-looking again; fix the ambiguity in memory after loading them.
drewp <drewp@bigasterisk.com>
parents: 1088
diff changeset
72 rulesPath = 'rules.n3'
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
73 try:
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
74 t1 = time.time()
1092
54de5144900d switch from evtiming to greplin.scales. Optimize rules reader to reuse previous data (400ms -> 0.6ms)
drewp <drewp@bigasterisk.com>
parents: 1091
diff changeset
75 self.rulesN3, self.ruleStore = readRules(
1089
cb7fa2f30df9 rules become simple-looking again; fix the ambiguity in memory after loading them.
drewp <drewp@bigasterisk.com>
parents: 1088
diff changeset
76 rulesPath, outputPatterns=[
1091
352ecf3c9aea more output statement patterns
drewp <drewp@bigasterisk.com>
parents: 1090
diff changeset
77 # Incomplete. See escapeoutputstatements.py for
352ecf3c9aea more output statement patterns
drewp <drewp@bigasterisk.com>
parents: 1090
diff changeset
78 # explanation.
352ecf3c9aea more output statement patterns
drewp <drewp@bigasterisk.com>
parents: 1090
diff changeset
79 (None, ROOM['brightness'], None),
352ecf3c9aea more output statement patterns
drewp <drewp@bigasterisk.com>
parents: 1090
diff changeset
80 (None, ROOM['playState'], None),
352ecf3c9aea more output statement patterns
drewp <drewp@bigasterisk.com>
parents: 1090
diff changeset
81 (None, ROOM['powerState'], None),
352ecf3c9aea more output statement patterns
drewp <drewp@bigasterisk.com>
parents: 1090
diff changeset
82 (None, ROOM['state'], None),
352ecf3c9aea more output statement patterns
drewp <drewp@bigasterisk.com>
parents: 1090
diff changeset
83 ])
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
84 ruleParseTime = time.time() - t1
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
85 except ValueError:
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
86 # this is so if you're just watching the inferred output,
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
87 # you'll see the error too
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
88 self.inferred = Graph()
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
89 self.inferred.add((ROOM['reasoner'], ROOM['ruleParseError'],
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
90 Literal(traceback.format_exc())))
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
91 self.copyOutput()
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
92 raise
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
93 return [(ROOM['reasoner'], ROOM['ruleParseTime'],
1086
6ab5238fc049 refactor
drewp <drewp@bigasterisk.com>
parents: 1085
diff changeset
94 Literal(ruleParseTime))], ruleParseTime
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
95
1092
54de5144900d switch from evtiming to greplin.scales. Optimize rules reader to reuse previous data (400ms -> 0.6ms)
drewp <drewp@bigasterisk.com>
parents: 1091
diff changeset
96 @STATS.graphChanged.time()
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
97 def graphChanged(self, inputGraph, oneShot=False, oneShotGraph=None):
1090
373704a3ba0f minor cleanups
drewp <drewp@bigasterisk.com>
parents: 1089
diff changeset
98 """
373704a3ba0f minor cleanups
drewp <drewp@bigasterisk.com>
parents: 1089
diff changeset
99 If we're getting called for a oneShot event, the oneShotGraph
373704a3ba0f minor cleanups
drewp <drewp@bigasterisk.com>
parents: 1089
diff changeset
100 statements are already in inputGraph.getGraph().
373704a3ba0f minor cleanups
drewp <drewp@bigasterisk.com>
parents: 1089
diff changeset
101 """
1085
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
102 log.info("----------------------")
1197
d8acab2b01f5 mqtt has two devices now. various older cleanups.
drewp <drewp@bigasterisk.com>
parents: 1156
diff changeset
103 log.info("graphChanged (oneShot=%s %s stmts):",
d8acab2b01f5 mqtt has two devices now. various older cleanups.
drewp <drewp@bigasterisk.com>
parents: 1156
diff changeset
104 oneShot, len(oneShotGraph) if oneShotGraph is not None else 0)
d8acab2b01f5 mqtt has two devices now. various older cleanups.
drewp <drewp@bigasterisk.com>
parents: 1156
diff changeset
105 if oneShotGraph:
d8acab2b01f5 mqtt has two devices now. various older cleanups.
drewp <drewp@bigasterisk.com>
parents: 1156
diff changeset
106 for stmt in oneShotGraph:
1555
606bb60a5e5c something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot
drewp <drewp@bigasterisk.com>
parents: 1523
diff changeset
107 log.info(" oneshot -> %s", ntStatement(stmt))
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
108 t1 = time.time()
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
109 oldInferred = self.inferred
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
110 try:
1086
6ab5238fc049 refactor
drewp <drewp@bigasterisk.com>
parents: 1085
diff changeset
111 ruleStatStmts, ruleParseSec = self.updateRules()
1092
54de5144900d switch from evtiming to greplin.scales. Optimize rules reader to reuse previous data (400ms -> 0.6ms)
drewp <drewp@bigasterisk.com>
parents: 1091
diff changeset
112
1093
e3ae71fbd3bd logging and cleanups. Upgrade fuxi to have the cPickle optimization.
drewp <drewp@bigasterisk.com>
parents: 1092
diff changeset
113 self.inferred, inferSec = self._makeInferred(inputGraph.getGraph())
1088
0f6128740000 fix input/output statement ambiguity problem by allowing quoted output statements
drewp <drewp@bigasterisk.com>
parents: 1087
diff changeset
114
1089
cb7fa2f30df9 rules become simple-looking again; fix the ambiguity in memory after loading them.
drewp <drewp@bigasterisk.com>
parents: 1088
diff changeset
115 self.inferred += unquoteOutputStatements(self.inferred)
1092
54de5144900d switch from evtiming to greplin.scales. Optimize rules reader to reuse previous data (400ms -> 0.6ms)
drewp <drewp@bigasterisk.com>
parents: 1091
diff changeset
116
1090
373704a3ba0f minor cleanups
drewp <drewp@bigasterisk.com>
parents: 1089
diff changeset
117 self.inferred += ruleStatStmts
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
118
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
119 if oneShot:
1090
373704a3ba0f minor cleanups
drewp <drewp@bigasterisk.com>
parents: 1089
diff changeset
120 # It's possible a oneShotGraph statement didn't
373704a3ba0f minor cleanups
drewp <drewp@bigasterisk.com>
parents: 1089
diff changeset
121 # trigger a rule to do something, but was itself the
373704a3ba0f minor cleanups
drewp <drewp@bigasterisk.com>
parents: 1089
diff changeset
122 # output statement. Probably we could just mix in the
373704a3ba0f minor cleanups
drewp <drewp@bigasterisk.com>
parents: 1089
diff changeset
123 # whole inputGraph here and not special-case the
373704a3ba0f minor cleanups
drewp <drewp@bigasterisk.com>
parents: 1089
diff changeset
124 # oneShotGraph.
373704a3ba0f minor cleanups
drewp <drewp@bigasterisk.com>
parents: 1089
diff changeset
125 self.inferred += oneShotGraph
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
126
1093
e3ae71fbd3bd logging and cleanups. Upgrade fuxi to have the cPickle optimization.
drewp <drewp@bigasterisk.com>
parents: 1092
diff changeset
127 t3 = time.time()
1555
606bb60a5e5c something with rx on inputgraph, i forget. also cleanup imports and logging of oneshot
drewp <drewp@bigasterisk.com>
parents: 1523
diff changeset
128 self.actions.putResults(self.inferred)
1093
e3ae71fbd3bd logging and cleanups. Upgrade fuxi to have the cPickle optimization.
drewp <drewp@bigasterisk.com>
parents: 1092
diff changeset
129 putResultsTime = time.time() - t3
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
130 finally:
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
131 if oneShot:
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
132 self.inferred = oldInferred
1093
e3ae71fbd3bd logging and cleanups. Upgrade fuxi to have the cPickle optimization.
drewp <drewp@bigasterisk.com>
parents: 1092
diff changeset
133 log.info("graphChanged took %.1f ms (rule parse %.1f ms, infer %.1f ms, putResults %.1f ms)" %
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
134 ((time.time() - t1) * 1000,
1086
6ab5238fc049 refactor
drewp <drewp@bigasterisk.com>
parents: 1085
diff changeset
135 ruleParseSec * 1000,
1093
e3ae71fbd3bd logging and cleanups. Upgrade fuxi to have the cPickle optimization.
drewp <drewp@bigasterisk.com>
parents: 1092
diff changeset
136 inferSec * 1000,
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
137 putResultsTime * 1000))
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
138 if not oneShot:
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
139 self.copyOutput()
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
140
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
141 def copyOutput(self):
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
142 self.outputGraph.setToGraph((s,p,o,ROOM['inferred']) for s,p,o in self.inferred)
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
143
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
144 def _makeInferred(self, inputGraph):
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
145 t1 = time.time()
1093
e3ae71fbd3bd logging and cleanups. Upgrade fuxi to have the cPickle optimization.
drewp <drewp@bigasterisk.com>
parents: 1092
diff changeset
146
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
147 out = infer(inputGraph, self.ruleStore)
1086
6ab5238fc049 refactor
drewp <drewp@bigasterisk.com>
parents: 1085
diff changeset
148 for p, n in NS.iteritems():
6ab5238fc049 refactor
drewp <drewp@bigasterisk.com>
parents: 1085
diff changeset
149 out.bind(p, n, override=True)
1093
e3ae71fbd3bd logging and cleanups. Upgrade fuxi to have the cPickle optimization.
drewp <drewp@bigasterisk.com>
parents: 1092
diff changeset
150
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
151 inferenceTime = time.time() - t1
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
152 out.add((ROOM['reasoner'], ROOM['inferenceTime'],
916
e8cce78a79d6 switch reasoning from restkit to async cyclone http client
drewp <drewp@bigasterisk.com>
parents: 882
diff changeset
153 Literal(inferenceTime)))
1093
e3ae71fbd3bd logging and cleanups. Upgrade fuxi to have the cPickle optimization.
drewp <drewp@bigasterisk.com>
parents: 1092
diff changeset
154 return out, inferenceTime
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
155
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
156
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
157
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
158 class Index(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
159 def get(self):
985
7e99f05c21cd reasoning index page to html
drewp <drewp@bigasterisk.com>
parents: 984
diff changeset
160 self.set_header("Content-Type", "text/html")
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
161 self.write(open('index.html').read())
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
162
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
163 class ImmediateUpdate(cyclone.web.RequestHandler):
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
164 def put(self):
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
165 """
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
166 request an immediate load of the remote graphs; the thing we
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
167 do in the background anyway. No payload.
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
168
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
169 Using PUT because this is idempotent and retryable and
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
170 everything.
859
379cefec542e screen out more values that change on every update
drewp <drewp@bigasterisk.com>
parents: 854
diff changeset
171
379cefec542e screen out more values that change on every update
drewp <drewp@bigasterisk.com>
parents: 854
diff changeset
172 todo: this should do the right thing when many requests come
379cefec542e screen out more values that change on every update
drewp <drewp@bigasterisk.com>
parents: 854
diff changeset
173 in very quickly
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
174 """
1108
8caf62030955 reasoning uses sse_collector
drewp <drewp@bigasterisk.com>
parents: 1093
diff changeset
175 log.warn("immediateUpdate from %s %s - ignored",
1085
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
176 self.request.headers.get('User-Agent', '?'),
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
177 self.request.headers['Host'])
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
178 self.set_status(202)
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
179
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
180 class OneShot(cyclone.web.RequestHandler):
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
181 def post(self):
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
182 """
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
183 payload is an rdf graph. The statements are momentarily added
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
184 to the input graph for exactly one update.
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
185
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
186 todo: how do we go from a transition like doorclosed-to-open
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
187 to a oneshot event? the upstream shouldn't have to do it. Do
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
188 we make those oneshot events here? for every object change?
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
189 there are probably special cases regarding startup time when
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
190 everything appears to be a 'change'.
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
191 """
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
192 try:
1197
d8acab2b01f5 mqtt has two devices now. various older cleanups.
drewp <drewp@bigasterisk.com>
parents: 1156
diff changeset
193 log.info('POST to oneShot, headers=%s', self.request.headers)
1086
6ab5238fc049 refactor
drewp <drewp@bigasterisk.com>
parents: 1085
diff changeset
194 dt = self.settings.reasoning.inputGraph.addOneShotFromString(
6ab5238fc049 refactor
drewp <drewp@bigasterisk.com>
parents: 1085
diff changeset
195 self.request.body, self.request.headers['content-type'])
6ab5238fc049 refactor
drewp <drewp@bigasterisk.com>
parents: 1085
diff changeset
196 self.set_header('x-graph-ms', str(1000 * dt))
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
197 except Exception as e:
1120
ba0cec7e1ac9 log oneshot tracebacks
drewp <drewp@bigasterisk.com>
parents: 1119
diff changeset
198 traceback.print_exc()
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
199 log.error(e)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
200 raise
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
201
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
202 # for reuse
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
203 class GraphResource(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
204 def get(self, which):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
205 self.set_header("Content-Type", "application/json")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
206 r = self.settings.reasoning
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
207 g = {'lastInput': r.inputGraph.getGraph(),
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
208 'lastOutput': r.inferred,
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
209 }[which]
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
210 self.write(self.jsonRdf(g))
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
211
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
212 def jsonRdf(self, g):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
213 return json.dumps(sorted(list(g)))
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
214
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
215 class NtGraphs(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
216 """same as what gets posted above"""
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
217 def get(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
218 r = self.settings.reasoning
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
219 inputGraphNt = r.inputGraph.getGraph().serialize(format="nt")
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
220 inferredNt = r.inferred.serialize(format="nt")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
221 self.set_header("Content-Type", "application/json")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
222 self.write(json.dumps({"input": inputGraphNt,
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
223 "inferred": inferredNt}))
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
224
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
225 class Rules(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
226 def get(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
227 self.set_header("Content-Type", "text/plain")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
228 self.write(self.settings.reasoning.rulesN3)
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
229
850
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
230 class Status(cyclone.web.RequestHandler):
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
231 def get(self):
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
232 self.set_header("Content-Type", "text/plain")
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
233 g = self.settings.reasoning.inputGraph.getGraph()
850
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
234 msg = ""
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
235 for badSource in g.subjects(RDF.type, ROOM['FailedGraphLoad']):
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
236 msg += "GET %s failed (%s). " % (
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
237 badSource, g.value(badSource, ROOM['graphLoadError']))
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
238 if not msg:
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
239 self.finish("all inputs ok")
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
240 return
850
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
241 self.set_status(500)
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
242 self.finish(msg)
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
243
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
244 class Static(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
245 def get(self, p):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
246 self.write(open(p).read())
850
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
247
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
248 liveClients = set()
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
249 def sendToLiveClients(d=None, asJson=None):
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
250 j = asJson or json.dumps(d)
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
251 for c in liveClients:
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
252 c.sendMessage(j)
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
253
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
254 class Events(cyclone.websocket.WebSocketHandler):
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
255
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
256 def connectionMade(self, *args, **kwargs):
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
257 log.info("websocket opened")
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
258 liveClients.add(self)
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
259
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
260 def connectionLost(self, reason):
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
261 log.info("websocket closed")
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
262 liveClients.remove(self)
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
263
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
264 def messageReceived(self, message):
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
265 log.info("got message %s" % message)
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
266
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
267 class Application(cyclone.web.Application):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
268 def __init__(self, reasoning):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
269 handlers = [
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
270 (r"/", Index),
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
271 (r"/immediateUpdate", ImmediateUpdate),
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
272 (r"/oneShot", OneShot),
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
273 (r'/putOutputs', PutOutputsTable),
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
274 (r'/(jquery.min.js)', Static),
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
275 (r'/(lastInput|lastOutput)Graph', GraphResource),
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
276
1523
0da337780f22 dep updates; graph url renames; and other build updates
drewp <drewp@bigasterisk.com>
parents: 1412
diff changeset
277 (r"/graph/reasoning", CycloneGraphHandler, {'masterGraph': reasoning.outputGraph}),
0da337780f22 dep updates; graph url renames; and other build updates
drewp <drewp@bigasterisk.com>
parents: 1412
diff changeset
278 (r"/graph/reasoning/events", CycloneGraphEventsHandler, {'masterGraph': reasoning.outputGraph}),
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
279
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
280 (r'/ntGraphs', NtGraphs),
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
281 (r'/rules', Rules),
850
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
282 (r'/status', Status),
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
283 (r'/events', Events),
1092
54de5144900d switch from evtiming to greplin.scales. Optimize rules reader to reuse previous data (400ms -> 0.6ms)
drewp <drewp@bigasterisk.com>
parents: 1091
diff changeset
284 (r'/stats/(.*)', StatsHandler, {'serverName': 'reasoning'}),
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
285 ]
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
286 cyclone.web.Application.__init__(self, handlers, reasoning=reasoning)
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
287
1085
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
288 def configLogging(arg):
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
289 log.setLevel(WARN)
1412
302063bfb8ff reasoning web page uses rdf/browse/graphView for inputs and outputs now
drewp <drewp@bigasterisk.com>
parents: 1411
diff changeset
290
1403
69b3f5b3aeda comments and logging
drewp <drewp@bigasterisk.com>
parents: 1374
diff changeset
291 if arg['-i'] or arg['-r'] or arg['-o'] or arg['-v']:
1085
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
292 log.handlers[0].setFormatter(ColoredFormatter("%(log_color)s%(levelname)-8s %(name)-6s %(filename)-12s:%(lineno)-3s %(funcName)-20s%(reset)s %(white)s%(message)s",
1040
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
293 datefmt=None,
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
294 reset=True,
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
295 log_colors={
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
296 'DEBUG': 'cyan',
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
297 'INFO': 'green',
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
298 'WARNING': 'yellow',
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
299 'ERROR': 'red',
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
300 'CRITICAL': 'red,bg_white',
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
301 },
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
302 secondary_log_colors={},
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
303 style='%'
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
304 ))
1266
ee5698ef64dd add separate :matchPredicate support. some build and log edits.
drewp <drewp@bigasterisk.com>
parents: 1197
diff changeset
305 defer.setDebugging(True)
ee5698ef64dd add separate :matchPredicate support. some build and log edits.
drewp <drewp@bigasterisk.com>
parents: 1197
diff changeset
306
1040
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
307
1085
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
308 if arg['-i']:
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
309 import twisted.python.log
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
310 twisted.python.log.startLogging(sys.stdout)
1085
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
311
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
312 getLogger('fetch').setLevel(DEBUG if arg['-i'] else WARN)
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
313 log.setLevel(DEBUG if arg['-r'] else WARN)
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
314 getLogger('output').setLevel(DEBUG if arg['-o'] else WARN)
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
315
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
316
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
317 if __name__ == '__main__':
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
318 arg = docopt("""
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
319 Usage: reasoning.py [options]
850
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
320
1108
8caf62030955 reasoning uses sse_collector
drewp <drewp@bigasterisk.com>
parents: 1093
diff changeset
321 -i Verbose log on the input phase
1085
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
322 -r Verbose log on the reasoning phase and web stuff
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
323 -o Verbose log on the actions/output phase
1403
69b3f5b3aeda comments and logging
drewp <drewp@bigasterisk.com>
parents: 1374
diff changeset
324 -v Everywhere verbose
1411
21d0cd98ef7a mock output mode
drewp <drewp@bigasterisk.com>
parents: 1403
diff changeset
325 --mockoutput Don't make outgoing requests
1085
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
326 """)
1411
21d0cd98ef7a mock output mode
drewp <drewp@bigasterisk.com>
parents: 1403
diff changeset
327
21d0cd98ef7a mock output mode
drewp <drewp@bigasterisk.com>
parents: 1403
diff changeset
328 r = Reasoning(arg['--mockoutput'])
1085
dda3f1524a52 lots of logging updates (patch may be imprecise)
drewp <drewp@bigasterisk.com>
parents: 1080
diff changeset
329 configLogging(arg)
1040
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
330 reactor.listenTCP(9071, Application(r), interface='::')
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
331 reactor.run()