homeauto: service/reasoning/reasoning.py annotate

annotate service/reasoning/reasoning.py @ 1054:bbaf0576f653

fetch all source graphs in parallel Ignore-this: 666d61fa9c69f78846987e0ccea750d4 darcs-hash:fee0fc60a6fd66102d3a2b73bf980b393e1349c6

author	drewp <drewp@bigasterisk.com>
date	Tue, 09 Feb 2016 22:01:19 -0800
parents	a328cc370b22
children	c1287ab87add

rev	line source
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	1 #!bin/python
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	2 """
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	3 Graph consists of:
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	4 input/* (read at startup)
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	5 webinput/* (new files are noticed in here)
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	6 any number of remote graphs, specified in the other graph as objects of (:reasoning, :source, *), reread constantly
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	7
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	8 gather subgraphs from various services, run them through a rules
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	9 engine, and make http requests with the conclusions.
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	10
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	11 E.g. 'when drew's phone is near the house, and someone is awake,
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	12 unlock the door when the door's motion sensor is activated'
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	13
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	14 When do we gather? The services should be able to trigger us, perhaps
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	15 with PSHB, that their graph has changed.
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	16 """
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	17
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	18
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	19 from twisted.internet import reactor, task
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	20 from twisted.internet.defer import inlineCallbacks, gatherResults
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	21 from twisted.python.filepath import FilePath
934 3bb18b7d21df reasoning actions: generalize them a bit but then add a bunch of special cases for mpd for now drewp <drewp@bigasterisk.com> parents: 923 diff changeset	22 import time, traceback, sys, json, logging, urllib
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	23 from rdflib import Graph, ConjunctiveGraph
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	24 from rdflib import Namespace, URIRef, Literal, RDF
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	25 from rdflib.parser import StringInputSource
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	26
869 15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	27 import cyclone.web, cyclone.websocket
1040 2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	28 from inference import infer
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	29 from rdflibtrig import addTrig
853 39857f82922e refactor graph ops drewp <drewp@bigasterisk.com> parents: 852 diff changeset	30 from graphop import graphEqual
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	31 from docopt import docopt
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	32 from actions import Actions
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	33 from FuXi.Rete.RuleStore import N3RuleStore
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	34
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	35 sys.path.append("../../lib")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	36 from logsetup import log
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	37 log.setLevel(logging.WARN)
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	38 outlog = logging.getLogger('output')
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	39 outlog.setLevel(logging.WARN)
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	40
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	41 ROOM = Namespace("http://projects.bigasterisk.com/room/")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	42 DEV = Namespace("http://projects.bigasterisk.com/device/")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	43
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	44
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	45 class InputGraph(object):
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	46 def __init__(self, inputDirs, onChange):
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	47 """
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	48 this has one Graph that's made of:
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	49 - all .n3 files from inputDirs (read at startup)
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	50 - all the remote graphs, specified in the file graphs
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	51
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	52 call updateFileData or updateRemoteData to reread those
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	53 graphs. getGraph to access the combined graph.
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	54
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	55 onChange(self) is called if the contents of the full graph
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	56 change (in an interesting way) during updateFileData or
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	57 updateRemoteData. Interesting means statements other than the
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	58 ones with the predicates on the boring list. onChange(self,
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	59 oneShot=True) means: don't store the result of this change
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	60 anywhere; it needs to be processed only once
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	61 """
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	62 self.inputDirs = inputDirs
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	63 self.onChange = onChange
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	64 self._fileGraph = Graph()
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	65 self._remoteGraph = None
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	66 self._combinedGraph = None
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	67 self._oneShotAdditionGraph = None
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	68
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	69 def updateFileData(self):
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	70 """
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	71 make sure we contain the correct data from the files in inputDirs
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	72 """
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	73 # this sample one is actually only needed for the output, but I don't
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	74 # think I want to have a separate graph for the output
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	75 # handling
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	76 log.debug("read file graphs")
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	77 for fp in FilePath("input").walk():
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	78 if fp.isdir():
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	79 continue
916 e8cce78a79d6 switch reasoning from restkit to async cyclone http client drewp <drewp@bigasterisk.com> parents: 882 diff changeset	80 if fp.splitext()[1] != '.n3':
e8cce78a79d6 switch reasoning from restkit to async cyclone http client drewp <drewp@bigasterisk.com> parents: 882 diff changeset	81 continue
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	82 log.debug("read %s", fp)
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	83 # todo: if this fails, leave the report in the graph
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	84 self._fileGraph.parse(fp.open(), format="n3")
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	85 self._combinedGraph = None
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	86
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	87 self.onChange(self)
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	88
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	89 @inlineCallbacks
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	90 def updateRemoteData(self):
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	91 """
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	92 read all remote graphs (which are themselves enumerated within
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	93 the file data)
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	94 """
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	95 t1 = time.time()
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	96 log.debug("read remote graphs")
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	97 g = ConjunctiveGraph()
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	98
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	99 @inlineCallbacks
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	100 def fetchOne(source):
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	101 try:
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	102 # this part could be parallelized
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	103 fetchTime = yield addTrig(g, source)
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	104 except Exception, e:
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	105 log.error(" can't add source %s: %s", source, e)
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	106 g.add((URIRef(source), ROOM['graphLoadError'], Literal(str(e))))
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	107 g.add((URIRef(source), RDF.type, ROOM['FailedGraphLoad']))
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	108 else:
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	109 g.add((URIRef(source), ROOM['graphLoadMs'],
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	110 Literal(round(fetchTime * 1000, 1))))
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	111
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	112 fetchDone = []
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	113 for source in self._fileGraph.objects(ROOM['reasoning'],
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	114 ROOM['source']):
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	115 fetchDone.append(fetchOne(source))
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	116 yield gatherResults(fetchDone, consumeErrors=True)
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	117 log.debug("loaded all in %.1f ms", 1000 * (time.time() - t1))
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	118
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	119 prevGraph = self._remoteGraph
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	120 self._remoteGraph = g
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	121 self._combinedGraph = None
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	122 if (prevGraph is None or
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	123 not graphEqual(g, prevGraph, ignorePredicates=[
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	124 ROOM['signalStrength'],
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	125 # perhaps anything with a number-datatype for its
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	126 # object should be filtered out, and you have to make
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	127 # an upstream quantization (e.g. 'temp high'/'temp
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	128 # low') if you want to do reasoning on the difference
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	129 URIRef("http://bigasterisk.com/map#lastSeenAgoSec"),
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	130 URIRef("http://bigasterisk.com/map#lastSeenAgo"),
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	131 ROOM['usingPower'],
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	132 ROOM['idleTimeMinutes'],
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	133 ROOM['idleTimeMs'],
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	134 ROOM['graphLoadMs'],
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	135 ROOM['localTimeToSecond'],
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	136 ROOM['history'],
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	137 ROOM['temperatureF'],
bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	138 ROOM['connectedAgo'],
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	139 ])):
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	140 log.debug(" remote graph changed")
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	141 self.onChange(self)
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	142 else:
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	143 log.debug(" remote graph has no changes to trigger rules")
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	144
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	145 def addOneShot(self, g):
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	146 """
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	147 add this graph to the total, call onChange, and then revert
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	148 the addition of this graph
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	149 """
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	150 self._oneShotAdditionGraph = g
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	151 self._combinedGraph = None
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	152 try:
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	153 self.onChange(self, oneShot=True, oneShotGraph=g)
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	154 finally:
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	155 self._oneShotAdditionGraph = None
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	156 self._combinedGraph = None
851 0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	157
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	158 def getGraph(self):
0d86b3955bcd rewriting reasoning to use graphs for config drewp <drewp@bigasterisk.com> parents: 850 diff changeset	159 """rdflib Graph with the file+remote contents of the input graph"""
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	160 # this could be much faster with the combined readonly graph
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	161 # view from rdflib
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	162 if self._combinedGraph is None:
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	163 self._combinedGraph = Graph()
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	164 if self._fileGraph:
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	165 for s in self._fileGraph:
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	166 self._combinedGraph.add(s)
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	167 if self._remoteGraph:
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	168 for s in self._remoteGraph:
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	169 self._combinedGraph.add(s)
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	170 if self._oneShotAdditionGraph:
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	171 for s in self._oneShotAdditionGraph:
869 15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	172 self._combinedGraph.add(s)
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	173
869 15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	174 return self._combinedGraph
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	175
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	176
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	177 class Reasoning(object):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	178 def __init__(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	179 self.prevGraph = None
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	180 self.lastPollTime = 0
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	181 self.lastError = ""
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	182
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	183 self.actions = Actions(sendToLiveClients)
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	184
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	185 self.rulesN3 = "(not read yet)"
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	186 self.inferred = Graph() # gets replaced in each graphChanged call
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	187
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	188 self.inputGraph = InputGraph([], self.graphChanged)
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	189 self.inputGraph.updateFileData()
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	190
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	191
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	192 def readRules(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	193 self.rulesN3 = open('rules.n3').read() # for web display
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	194 self.ruleStore = N3RuleStore()
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	195 self.ruleGraph = Graph(self.ruleStore)
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	196 self.ruleGraph.parse('rules.n3', format='n3') # for inference
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	197
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	198 @inlineCallbacks
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	199 def poll(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	200 try:
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	201 yield self.inputGraph.updateRemoteData()
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	202 self.lastPollTime = time.time()
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	203 except Exception, e:
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	204 log.error(traceback.format_exc())
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	205 self.lastError = str(e)
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	206
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	207 def updateRules(self):
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	208 try:
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	209 t1 = time.time()
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	210 self.readRules()
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	211 ruleParseTime = time.time() - t1
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	212 except ValueError:
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	213 # this is so if you're just watching the inferred output,
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	214 # you'll see the error too
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	215 self.inferred = Graph()
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	216 self.inferred.add((ROOM['reasoner'], ROOM['ruleParseError'],
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	217 Literal(traceback.format_exc())))
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	218 raise
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	219 return [(ROOM['reasoner'], ROOM['ruleParseTime'],
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	220 Literal(ruleParseTime))]
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	221
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	222 def graphChanged(self, inputGraph, oneShot=False, oneShotGraph=None):
869 15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	223 t1 = time.time()
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	224 oldInferred = self.inferred
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	225 try:
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	226 ruleStmts = self.updateRules()
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	227
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	228 g = inputGraph.getGraph()
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	229 self.inferred = self._makeInferred(g)
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	230 [self.inferred.add(s) for s in ruleStmts]
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	231
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	232 if oneShot:
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	233 # unclear where this should go, but the oneshot'd
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	234 # statements should be just as usable as inferred
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	235 # ones.
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	236 for s in oneShotGraph:
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	237 self.inferred.add(s)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	238
869 15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	239 t2 = time.time()
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	240 self.actions.putResults(self.inputGraph.getGraph(), self.inferred)
869 15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	241 putResultsTime = time.time() - t2
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	242 finally:
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	243 if oneShot:
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	244 self.inferred = oldInferred
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	245 log.info("graphChanged %.1f ms (putResults %.1f ms)" %
869 15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	246 ((time.time() - t1) * 1000,
1045 a328cc370b22 ipv6 fetch support. refactor Actions to new class and file drewp <drewp@bigasterisk.com> parents: 1040 diff changeset	247 putResultsTime * 1000))
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	248
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	249 def _makeInferred(self, inputGraph):
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	250 t1 = time.time()
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	251 out = infer(inputGraph, self.ruleStore)
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	252 inferenceTime = time.time() - t1
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	253
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	254 out.add((ROOM['reasoner'], ROOM['inferenceTime'],
916 e8cce78a79d6 switch reasoning from restkit to async cyclone http client drewp <drewp@bigasterisk.com> parents: 882 diff changeset	255 Literal(inferenceTime)))
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	256 return out
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	257
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	258
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	259
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	260 class Index(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	261 def get(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	262 # make sure GET / fails if our poll loop died
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	263 ago = time.time() - self.settings.reasoning.lastPollTime
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	264 if ago > 2:
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	265 self.set_status(500)
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	266 self.finish("last poll was %s sec ago. last error: %s" %
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	267 (ago, self.settings.reasoning.lastError))
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	268 return
985 7e99f05c21cd reasoning index page to html drewp <drewp@bigasterisk.com> parents: 984 diff changeset	269 self.set_header("Content-Type", "text/html")
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	270 self.write(open('index.html').read())
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	271
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	272 class ImmediateUpdate(cyclone.web.RequestHandler):
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	273 @inlineCallbacks
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	274 def put(self):
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	275 """
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	276 request an immediate load of the remote graphs; the thing we
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	277 do in the background anyway. No payload.
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	278
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	279 Using PUT because this is idempotent and retryable and
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	280 everything.
859 379cefec542e screen out more values that change on every update drewp <drewp@bigasterisk.com> parents: 854 diff changeset	281
379cefec542e screen out more values that change on every update drewp <drewp@bigasterisk.com> parents: 854 diff changeset	282 todo: this should do the right thing when many requests come
379cefec542e screen out more values that change on every update drewp <drewp@bigasterisk.com> parents: 854 diff changeset	283 in very quickly
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	284 """
854 1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	285 print self.request.headers
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	286 log.info("immediateUpdate from %s",
1815d2343a3f cleanup drewp <drewp@bigasterisk.com> parents: 853 diff changeset	287 self.request.headers.get('User-Agent', '?'))
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	288 yield r.poll()
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	289 self.set_status(202)
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	290
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	291 def parseRdf(text, contentType):
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	292 g = Graph()
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	293 g.parse(StringInputSource(text), format={
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	294 'text/n3': ['n3'],
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	295 }[contentType])
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	296 return g
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	297
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	298 class OneShot(cyclone.web.RequestHandler):
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	299 def post(self):
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	300 """
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	301 payload is an rdf graph. The statements are momentarily added
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	302 to the input graph for exactly one update.
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	303
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	304 todo: how do we go from a transition like doorclosed-to-open
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	305 to a oneshot event? the upstream shouldn't have to do it. Do
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	306 we make those oneshot events here? for every object change?
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	307 there are probably special cases regarding startup time when
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	308 everything appears to be a 'change'.
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	309 """
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	310 try:
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	311 g = parseRdf(self.request.body, self.request.headers['content-type'])
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	312 for s in g:
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	313 print "stmt", s
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	314 if not len(g):
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	315 log.warn("incoming oneshot graph had no statements: %r", self.request.body)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	316 return
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	317 self.settings.reasoning.inputGraph.addOneShot(g)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	318 except Exception as e:
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	319 log.error(e)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	320 raise
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	321
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	322 # for reuse
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	323 class GraphResource(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	324 def get(self, which):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	325 self.set_header("Content-Type", "application/json")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	326 r = self.settings.reasoning
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	327 g = {'lastInput': r.inputGraph.getGraph(),
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	328 'lastOutput': r.inferred,
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	329 }[which]
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	330 self.write(self.jsonRdf(g))
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	331
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	332 def jsonRdf(self, g):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	333 return json.dumps(sorted(list(g)))
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	334
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	335 class NtGraphs(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	336 """same as what gets posted above"""
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	337 def get(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	338 r = self.settings.reasoning
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	339 inputGraphNt = r.inputGraph.getGraph().serialize(format="nt")
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	340 inferredNt = r.inferred.serialize(format="nt")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	341 self.set_header("Content-Type", "application/json")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	342 self.write(json.dumps({"input": inputGraphNt,
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	343 "inferred": inferredNt}))
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	344
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	345 class Rules(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	346 def get(self):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	347 self.set_header("Content-Type", "text/plain")
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	348 self.write(self.settings.reasoning.rulesN3)
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	349
850 887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	350 class Status(cyclone.web.RequestHandler):
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	351 def get(self):
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	352 self.set_header("Content-Type", "text/plain")
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	353 g = self.settings.reasoning.inputGraph.getGraph()
850 887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	354 msg = ""
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	355 for badSource in g.subjects(RDF.type, ROOM['FailedGraphLoad']):
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	356 msg += "GET %s failed (%s). " % (
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	357 badSource, g.value(badSource, ROOM['graphLoadError']))
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	358 if not msg:
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	359 self.finish("all inputs ok")
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	360 return
850 887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	361 self.set_status(500)
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	362 self.finish(msg)
887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	363
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	364 class Static(cyclone.web.RequestHandler):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	365 def get(self, p):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	366 self.write(open(p).read())
850 887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	367
869 15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	368 liveClients = set()
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	369 def sendToLiveClients(d=None, asJson=None):
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	370 j = asJson or json.dumps(d)
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	371 for c in liveClients:
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	372 c.sendMessage(j)
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	373
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	374 class Events(cyclone.websocket.WebSocketHandler):
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	375
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	376 def connectionMade(self, args, *kwargs):
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	377 log.info("websocket opened")
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	378 liveClients.add(self)
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	379
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	380 def connectionLost(self, reason):
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	381 log.info("websocket closed")
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	382 liveClients.remove(self)
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	383
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	384 def messageReceived(self, message):
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	385 log.info("got message %s" % message)
15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	386
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	387 class Application(cyclone.web.Application):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	388 def __init__(self, reasoning):
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	389 handlers = [
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	390 (r"/", Index),
852 6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	391 (r"/immediateUpdate", ImmediateUpdate),
6ccd930834d1 scan more input files. oneshot and immediate update features. drewp <drewp@bigasterisk.com> parents: 851 diff changeset	392 (r"/oneShot", OneShot),
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	393 (r'/(jquery.min.js)', Static),
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	394 (r'/(lastInput\|lastOutput)Graph', GraphResource),
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	395 (r'/ntGraphs', NtGraphs),
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	396 (r'/rules', Rules),
850 887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	397 (r'/status', Status),
869 15af9d8c7186 reasoning has a websocket server that broadcasts some events drewp <drewp@bigasterisk.com> parents: 859 diff changeset	398 (r'/events', Events),
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	399 ]
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	400 cyclone.web.Application.__init__(self, handlers, reasoning=reasoning)
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	401
fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	402 if __name__ == '__main__':
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	403
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	404 arg = docopt("""
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	405 Usage: reasoning.py [options]
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	406
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	407 -v Verbose (and slow updates)
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	408 """)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	409
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	410 r = Reasoning()
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	411 if arg['-v']:
1040 2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	412 from colorlog import ColoredFormatter
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	413 log.handlers[0].setFormatter(ColoredFormatter("%(log_color)s%(levelname)-8s%(reset)s %(white)s%(message)s",
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	414 datefmt=None,
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	415 reset=True,
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	416 log_colors={
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	417 'DEBUG': 'cyan',
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	418 'INFO': 'green',
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	419 'WARNING': 'yellow',
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	420 'ERROR': 'red',
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	421 'CRITICAL': 'red,bg_white',
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	422 },
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	423 secondary_log_colors={},
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	424 style='%'
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	425 ))
2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	426
984 5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	427 import twisted.python.log
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	428 twisted.python.log.startLogging(sys.stdout)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	429 log.setLevel(logging.DEBUG)
5da9200418db reasoning: -v verbose setting, more oneshot support, some custom stuff for mpd commands drewp <drewp@bigasterisk.com> parents: 934 diff changeset	430 outlog.setLevel(logging.DEBUG)
850 887d47682d94 /status page, errors on failed graphs, time reports of successful fetches drewp <drewp@bigasterisk.com> parents: 838 diff changeset	431
1054 bbaf0576f653 fetch all source graphs in parallel drewp <drewp@bigasterisk.com> parents: 1045 diff changeset	432 task.LoopingCall(r.poll).start(1.0 if not arg['-v'] else 10)
1040 2fa7daf06f34 rdflib can parse trig now drewp <drewp@bigasterisk.com> parents: 985 diff changeset	433 reactor.listenTCP(9071, Application(r), interface='::')
825 fc753b24f69a move reasoning from /my/proj/room, new integration with magma drewp <drewp@bigasterisk.com> parents: diff changeset	434 reactor.run()

Mercurial > code > home > repos > homeauto

annotate service/reasoning/reasoning.py @ 1054:bbaf0576f653