Mercurial > code > home > repos > homeauto

annotate service/reasoning/reasoning.py @ 1052:2434e88d8bb0

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
add bed buttons Ignore-this: 912a92f7a8e173fe64aa4c1df9ef7dba darcs-hash:2e4031a9cacedb28f57c36708d69d6ec3610ef9d
author drewp <drewp@bigasterisk.com>
date Mon, 08 Feb 2016 23:48:29 -0800
parents a328cc370b22
children e5c27d2f11ab
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
1 #!bin/python
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
2 """
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
3 Graph consists of:
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
4 input/* (read at startup)
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
5 webinput/* (new files are noticed in here)
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
6 any number of remote graphs, specified in the other graph as objects of (:reasoning, :source, *), reread constantly
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
7
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
8 gather subgraphs from various services, run them through a rules
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
9 engine, and make http requests with the conclusions.
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
10
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
11 E.g. 'when drew's phone is near the house, and someone is awake,
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
12 unlock the door when the door's motion sensor is activated'
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
13
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
14 When do we gather? The services should be able to trigger us, perhaps
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
15 with PSHB, that their graph has changed.
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
16 """
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
17
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
18
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
19 from twisted.internet import reactor, task
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
20 from twisted.python.filepath import FilePath
934
3bb18b7d21df reasoning actions: generalize them a bit but then add a bunch of special cases for mpd for now
drewp <drewp@bigasterisk.com>
parents: 923
diff changeset
21 import time, traceback, sys, json, logging, urllib
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
22 from rdflib import Graph, ConjunctiveGraph
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
23 from rdflib import Namespace, URIRef, Literal, RDF
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
24 from rdflib.parser import StringInputSource
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
25
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
26 import cyclone.web, cyclone.websocket
1040
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
27 from inference import infer
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
28 from rdflibtrig import addTrig
853
39857f82922e refactor graph ops
drewp <drewp@bigasterisk.com>
parents: 852
diff changeset
29 from graphop import graphEqual
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
30 from docopt import docopt
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
31 from actions import Actions
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
32 from FuXi.Rete.RuleStore import N3RuleStore
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
33
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
34 sys.path.append("../../lib")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
35 from logsetup import log
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
36 log.setLevel(logging.WARN)
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
37 outlog = logging.getLogger('output')
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
38 outlog.setLevel(logging.WARN)
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
39
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
40 ROOM = Namespace("http://projects.bigasterisk.com/room/")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
41 DEV = Namespace("http://projects.bigasterisk.com/device/")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
42
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
43
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
44 class InputGraph(object):
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
45 def __init__(self, inputDirs, onChange):
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
46 """
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
47 this has one Graph that's made of:
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
48 - all .n3 files from inputDirs (read at startup)
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
49 - all the remote graphs, specified in the file graphs
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
50
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
51 call updateFileData or updateRemoteData to reread those
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
52 graphs. getGraph to access the combined graph.
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
53
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
54 onChange(self) is called if the contents of the full graph
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
55 change (in an interesting way) during updateFileData or
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
56 updateRemoteData. Interesting means statements other than the
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
57 ones with the predicates on the boring list. onChange(self,
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
58 oneShot=True) means: don't store the result of this change
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
59 anywhere; it needs to be processed only once
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
60 """
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
61 self.inputDirs = inputDirs
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
62 self.onChange = onChange
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
63 self._fileGraph = Graph()
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
64 self._remoteGraph = None
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
65 self._combinedGraph = None
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
66 self._oneShotAdditionGraph = None
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
67
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
68 def updateFileData(self):
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
69 """
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
70 make sure we contain the correct data from the files in inputDirs
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
71 """
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
72 # this sample one is actually only needed for the output, but I don't
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
73 # think I want to have a separate graph for the output
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
74 # handling
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
75 log.debug("read file graphs")
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
76 for fp in FilePath("input").walk():
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
77 if fp.isdir():
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
78 continue
916
e8cce78a79d6 switch reasoning from restkit to async cyclone http client
drewp <drewp@bigasterisk.com>
parents: 882
diff changeset
79 if fp.splitext()[1] != '.n3':
e8cce78a79d6 switch reasoning from restkit to async cyclone http client
drewp <drewp@bigasterisk.com>
parents: 882
diff changeset
80 continue
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
81 log.debug("read %s", fp)
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
82 # todo: if this fails, leave the report in the graph
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
83 self._fileGraph.parse(fp.open(), format="n3")
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
84 self._combinedGraph = None
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
85
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
86 self.onChange(self)
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
87
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
88 def updateRemoteData(self):
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
89 """
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
90 read all remote graphs (which are themselves enumerated within
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
91 the file data)
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
92 """
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
93 log.debug("read remote graphs")
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
94 g = ConjunctiveGraph()
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
95 for source in self._fileGraph.objects(ROOM['reasoning'],
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
96 ROOM['source']):
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
97 try:
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
98 # this part could be parallelized
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
99 fetchTime = addTrig(g, source)
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
100 except Exception, e:
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
101 log.error(" adding source %s: %s", source, e)
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
102 g.add((URIRef(source), ROOM['graphLoadError'], Literal(str(e))))
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
103 g.add((URIRef(source), RDF.type, ROOM['FailedGraphLoad']))
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
104 else:
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
105 g.add((URIRef(source), ROOM['graphLoadMs'],
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
106 Literal(round(fetchTime * 1000, 1))))
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
107 prevGraph = self._remoteGraph
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
108 self._remoteGraph = g
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
109 self._combinedGraph = None
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
110 if (prevGraph is None or
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
111 not graphEqual(g, prevGraph, ignorePredicates=[
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
112 ROOM.signalStrength,
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
113 ROOM.graphLoadSecs,
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
114 # perhaps anything with a number-datatype for its
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
115 # object should be filtered out, and you have to make
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
116 # an upstream quantization (e.g. 'temp high'/'temp
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
117 # low') if you want to do reasoning on the difference
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
118 URIRef("http://bigasterisk.com/map#lastSeenAgoSec"),
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
119 URIRef("http://bigasterisk.com/map#lastSeenAgo"),
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
120 URIRef("http://projects.bigasterisk.com/room/usingPower"),
859
379cefec542e screen out more values that change on every update
drewp <drewp@bigasterisk.com>
parents: 854
diff changeset
121 URIRef("http://projects.bigasterisk.com/room/idleTimeMinutes"),
379cefec542e screen out more values that change on every update
drewp <drewp@bigasterisk.com>
parents: 854
diff changeset
122 URIRef("http://projects.bigasterisk.com/room/idleTimeMs"),
923
4ae49c6adecb errors and logging in reasoning
drewp <drewp@bigasterisk.com>
parents: 919
diff changeset
123 ROOM.history,
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
124 ])):
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
125 log.debug(" remote graph changed")
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
126 self.onChange(self)
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
127 else:
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
128 log.debug(" remote graph is unchanged")
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
129
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
130 def addOneShot(self, g):
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
131 """
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
132 add this graph to the total, call onChange, and then revert
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
133 the addition of this graph
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
134 """
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
135 self._oneShotAdditionGraph = g
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
136 self._combinedGraph = None
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
137 try:
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
138 self.onChange(self, oneShot=True, oneShotGraph=g)
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
139 finally:
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
140 self._oneShotAdditionGraph = None
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
141 self._combinedGraph = None
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
142
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
143 def getGraph(self):
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
144 """rdflib Graph with the file+remote contents of the input graph"""
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
145 # this could be much faster with the combined readonly graph
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
146 # view from rdflib
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
147 if self._combinedGraph is None:
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
148 self._combinedGraph = Graph()
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
149 if self._fileGraph:
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
150 for s in self._fileGraph:
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
151 self._combinedGraph.add(s)
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
152 if self._remoteGraph:
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
153 for s in self._remoteGraph:
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
154 self._combinedGraph.add(s)
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
155 if self._oneShotAdditionGraph:
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
156 for s in self._oneShotAdditionGraph:
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
157 self._combinedGraph.add(s)
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
158
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
159 return self._combinedGraph
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
160
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
161
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
162 class Reasoning(object):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
163 def __init__(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
164 self.prevGraph = None
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
165 self.lastPollTime = 0
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
166 self.lastError = ""
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
167
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
168 self.actions = Actions(sendToLiveClients)
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
169
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
170 self.rulesN3 = "(not read yet)"
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
171 self.inferred = Graph() # gets replaced in each graphChanged call
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
172
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
173 self.inputGraph = InputGraph([], self.graphChanged)
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
174 self.inputGraph.updateFileData()
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
175
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
176
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
177 def readRules(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
178 self.rulesN3 = open('rules.n3').read() # for web display
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
179 self.ruleStore = N3RuleStore()
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
180 self.ruleGraph = Graph(self.ruleStore)
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
181 self.ruleGraph.parse('rules.n3', format='n3') # for inference
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
182
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
183 def poll(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
184 try:
851
0d86b3955bcd rewriting reasoning to use graphs for config
drewp <drewp@bigasterisk.com>
parents: 850
diff changeset
185 self.inputGraph.updateRemoteData()
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
186 self.lastPollTime = time.time()
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
187 except Exception, e:
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
188 log.error(traceback.format_exc())
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
189 self.lastError = str(e)
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
190
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
191 def updateRules(self):
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
192 try:
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
193 t1 = time.time()
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
194 self.readRules()
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
195 ruleParseTime = time.time() - t1
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
196 except ValueError:
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
197 # this is so if you're just watching the inferred output,
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
198 # you'll see the error too
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
199 self.inferred = Graph()
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
200 self.inferred.add((ROOM['reasoner'], ROOM['ruleParseError'],
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
201 Literal(traceback.format_exc())))
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
202 raise
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
203 return [(ROOM['reasoner'], ROOM['ruleParseTime'],
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
204 Literal(ruleParseTime))]
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
205
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
206 def graphChanged(self, inputGraph, oneShot=False, oneShotGraph=None):
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
207 t1 = time.time()
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
208 oldInferred = self.inferred
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
209 try:
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
210 ruleStmts = self.updateRules()
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
211
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
212 g = inputGraph.getGraph()
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
213 self.inferred = self._makeInferred(g)
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
214 [self.inferred.add(s) for s in ruleStmts]
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
215
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
216 if oneShot:
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
217 # unclear where this should go, but the oneshot'd
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
218 # statements should be just as usable as inferred
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
219 # ones.
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
220 for s in oneShotGraph:
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
221 self.inferred.add(s)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
222
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
223 t2 = time.time()
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
224 self.actions.putResults(self.inputGraph.getGraph(), self.inferred)
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
225 putResultsTime = time.time() - t2
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
226 finally:
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
227 if oneShot:
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
228 self.inferred = oldInferred
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
229 log.info("graphChanged %.1f ms (putResults %.1f ms)" %
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
230 ((time.time() - t1) * 1000,
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
231 putResultsTime * 1000))
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
232
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
233 def _makeInferred(self, inputGraph):
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
234 t1 = time.time()
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
235 out = infer(inputGraph, self.ruleStore)
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
236 inferenceTime = time.time() - t1
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
237
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
238 out.add((ROOM['reasoner'], ROOM['inferenceTime'],
916
e8cce78a79d6 switch reasoning from restkit to async cyclone http client
drewp <drewp@bigasterisk.com>
parents: 882
diff changeset
239 Literal(inferenceTime)))
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
240 return out
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
241
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
242
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
243
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
244 class Index(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
245 def get(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
246 # make sure GET / fails if our poll loop died
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
247 ago = time.time() - self.settings.reasoning.lastPollTime
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
248 if ago > 2:
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
249 self.set_status(500)
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
250 self.finish("last poll was %s sec ago. last error: %s" %
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
251 (ago, self.settings.reasoning.lastError))
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
252 return
985
7e99f05c21cd reasoning index page to html
drewp <drewp@bigasterisk.com>
parents: 984
diff changeset
253 self.set_header("Content-Type", "text/html")
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
254 self.write(open('index.html').read())
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
255
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
256 class ImmediateUpdate(cyclone.web.RequestHandler):
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
257 def put(self):
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
258 """
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
259 request an immediate load of the remote graphs; the thing we
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
260 do in the background anyway. No payload.
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
261
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
262 Using PUT because this is idempotent and retryable and
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
263 everything.
859
379cefec542e screen out more values that change on every update
drewp <drewp@bigasterisk.com>
parents: 854
diff changeset
264
379cefec542e screen out more values that change on every update
drewp <drewp@bigasterisk.com>
parents: 854
diff changeset
265 todo: this should do the right thing when many requests come
379cefec542e screen out more values that change on every update
drewp <drewp@bigasterisk.com>
parents: 854
diff changeset
266 in very quickly
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
267 """
854
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
268 print self.request.headers
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
269 log.info("immediateUpdate from %s",
1815d2343a3f cleanup
drewp <drewp@bigasterisk.com>
parents: 853
diff changeset
270 self.request.headers.get('User-Agent', '?'))
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
271 r.poll()
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
272 self.set_status(202)
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
273
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
274 def parseRdf(text, contentType):
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
275 g = Graph()
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
276 g.parse(StringInputSource(text), format={
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
277 'text/n3': 'n3',
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
278 }[contentType])
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
279 return g
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
280
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
281 class OneShot(cyclone.web.RequestHandler):
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
282 def post(self):
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
283 """
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
284 payload is an rdf graph. The statements are momentarily added
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
285 to the input graph for exactly one update.
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
286
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
287 todo: how do we go from a transition like doorclosed-to-open
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
288 to a oneshot event? the upstream shouldn't have to do it. Do
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
289 we make those oneshot events here? for every object change?
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
290 there are probably special cases regarding startup time when
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
291 everything appears to be a 'change'.
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
292 """
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
293 try:
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
294 g = parseRdf(self.request.body, self.request.headers['content-type'])
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
295 for s in g:
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
296 print "stmt", s
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
297 if not len(g):
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
298 log.warn("incoming oneshot graph had no statements: %r", self.request.body)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
299 return
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
300 self.settings.reasoning.inputGraph.addOneShot(g)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
301 except Exception as e:
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
302 log.error(e)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
303 raise
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
304
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
305 # for reuse
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
306 class GraphResource(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
307 def get(self, which):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
308 self.set_header("Content-Type", "application/json")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
309 r = self.settings.reasoning
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
310 g = {'lastInput': r.inputGraph.getGraph(),
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
311 'lastOutput': r.inferred,
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
312 }[which]
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
313 self.write(self.jsonRdf(g))
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
314
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
315 def jsonRdf(self, g):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
316 return json.dumps(sorted(list(g)))
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
317
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
318 class NtGraphs(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
319 """same as what gets posted above"""
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
320 def get(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
321 r = self.settings.reasoning
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
322 inputGraphNt = r.inputGraph.getGraph().serialize(format="nt")
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
323 inferredNt = r.inferred.serialize(format="nt")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
324 self.set_header("Content-Type", "application/json")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
325 self.write(json.dumps({"input": inputGraphNt,
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
326 "inferred": inferredNt}))
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
327
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
328 class Rules(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
329 def get(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
330 self.set_header("Content-Type", "text/plain")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
331 self.write(self.settings.reasoning.rulesN3)
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
332
850
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
333 class Status(cyclone.web.RequestHandler):
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
334 def get(self):
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
335 self.set_header("Content-Type", "text/plain")
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
336 g = self.settings.reasoning.inputGraph.getGraph()
850
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
337 msg = ""
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
338 for badSource in g.subjects(RDF.type, ROOM['FailedGraphLoad']):
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
339 msg += "GET %s failed (%s). " % (
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
340 badSource, g.value(badSource, ROOM['graphLoadError']))
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
341 if not msg:
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
342 self.finish("all inputs ok")
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
343 return
850
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
344 self.set_status(500)
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
345 self.finish(msg)
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
346
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
347 class Static(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
348 def get(self, p):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
349 self.write(open(p).read())
850
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
350
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
351 liveClients = set()
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
352 def sendToLiveClients(d=None, asJson=None):
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
353 j = asJson or json.dumps(d)
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
354 for c in liveClients:
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
355 c.sendMessage(j)
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
356
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
357 class Events(cyclone.websocket.WebSocketHandler):
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
358
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
359 def connectionMade(self, *args, **kwargs):
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
360 log.info("websocket opened")
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
361 liveClients.add(self)
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
362
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
363 def connectionLost(self, reason):
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
364 log.info("websocket closed")
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
365 liveClients.remove(self)
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
366
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
367 def messageReceived(self, message):
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
368 log.info("got message %s" % message)
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
369
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
370 class Application(cyclone.web.Application):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
371 def __init__(self, reasoning):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
372 handlers = [
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
373 (r"/", Index),
852
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
374 (r"/immediateUpdate", ImmediateUpdate),
6ccd930834d1 scan more input files. oneshot and immediate update features.
drewp <drewp@bigasterisk.com>
parents: 851
diff changeset
375 (r"/oneShot", OneShot),
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
376 (r'/(jquery.min.js)', Static),
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
377 (r'/(lastInput|lastOutput)Graph', GraphResource),
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
378 (r'/ntGraphs', NtGraphs),
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
379 (r'/rules', Rules),
850
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
380 (r'/status', Status),
869
15af9d8c7186 reasoning has a websocket server that broadcasts some events
drewp <drewp@bigasterisk.com>
parents: 859
diff changeset
381 (r'/events', Events),
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
382 ]
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
383 cyclone.web.Application.__init__(self, handlers, reasoning=reasoning)
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
384
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
385 if __name__ == '__main__':
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
386
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
387 arg = docopt("""
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
388 Usage: reasoning.py [options]
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
389
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
390 -v Verbose
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
391 """)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
392
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
393 r = Reasoning()
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
394 if arg['-v']:
1040
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
395 from colorlog import ColoredFormatter
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
396 log.handlers[0].setFormatter(ColoredFormatter("%(log_color)s%(levelname)-8s%(reset)s %(white)s%(message)s",
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
397 datefmt=None,
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
398 reset=True,
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
399 log_colors={
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
400 'DEBUG': 'cyan',
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
401 'INFO': 'green',
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
402 'WARNING': 'yellow',
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
403 'ERROR': 'red',
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
404 'CRITICAL': 'red,bg_white',
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
405 },
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
406 secondary_log_colors={},
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
407 style='%'
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
408 ))
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
409
984
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
410 import twisted.python.log
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
411 twisted.python.log.startLogging(sys.stdout)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
412 log.setLevel(logging.DEBUG)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
413 outlog.setLevel(logging.DEBUG)
850
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches
drewp <drewp@bigasterisk.com>
parents: 838
diff changeset
414
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
415 task.LoopingCall(r.poll).start(1.0)
1040
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 985
diff changeset
416 reactor.listenTCP(9071, Application(r), interface='::')
825
fc753b24f69a move reasoning from /my/proj/room, new integration with magma
drewp <drewp@bigasterisk.com>
parents:
diff changeset
417 reactor.run()