Mercurial > code > home > repos > homeauto

annotate lib/patchablegraph.py @ 1276:d40214eaca99

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
stats support and maybe a no-op filtering logic change snuck in there Ignore-this: d54125308243159b28ef11e2d09014f4 darcs-hash:dd3f8b8fdb305df34477e6d77129c5032ad0b0ef
author drewp <drewp@bigasterisk.com>
date Sat, 20 Apr 2019 23:51:02 -0700
parents b50a13ef20ba
children 1d2817cb9a6f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
1 """
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
2 Design:
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
3
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
4 1. Services each have (named) graphs, which they patch as things
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
5 change. PatchableGraph is an object for holding this graph.
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
6 2. You can http GET that graph, or ...
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
7 3. You can http GET/SSE that graph and hear about modifications to it
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
8 4. The client that got the graph holds and maintains a copy. The
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
9 client may merge together multiple graphs.
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
10 5. Client queries its graph with low-level APIs or client-side sparql.
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
11 6. When the graph changes, the client knows and can update itself at
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
12 low or high granularity.
1038
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
13
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
14
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
15 See also:
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
16 * http://iswc2007.semanticweb.org/papers/533.pdf RDFSync: efficient remote synchronization of RDF
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
17 models
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
18 * https://www.w3.org/2009/12/rdf-ws/papers/ws07 Supporting Change Propagation in RDF
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
19 * https://www.w3.org/DesignIssues/lncs04/Diff.pdf Delta: an ontology for the distribution of
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
20 differences between RDF graphs
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
21
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
22 """
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
23 import sys, json, logging, itertools
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
24 import cyclone.sse
1136
1e43ec4a5f23 build and import updates for rdfdb, etc
drewp <drewp@bigasterisk.com>
parents: 1103
diff changeset
25 sys.path.append("/my/proj/rdfdb")
1e43ec4a5f23 build and import updates for rdfdb, etc
drewp <drewp@bigasterisk.com>
parents: 1103
diff changeset
26 from rdfdb.grapheditapi import GraphEditApi
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
27 from rdflib import ConjunctiveGraph
1136
1e43ec4a5f23 build and import updates for rdfdb, etc
drewp <drewp@bigasterisk.com>
parents: 1103
diff changeset
28 from rdfdb.rdflibpatch import patchQuads
1e43ec4a5f23 build and import updates for rdfdb, etc
drewp <drewp@bigasterisk.com>
parents: 1103
diff changeset
29 from rdfdb.patch import Patch
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
30 from rdflib_jsonld.serializer import from_rdf
1103
b84e956771fc sse_collector now kind of gets concurrent requests right
drewp <drewp@bigasterisk.com>
parents: 1038
diff changeset
31 from rdflib.parser import StringInputSource
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
32 from cycloneerr import PrettyErrorHandler
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
33 from greplin import scales
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
34
1038
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
35 log = logging.getLogger('patchablegraph')
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
36
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
37 def writeGraphResponse(req, graph, acceptHeader):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
38 if acceptHeader == 'application/nquads':
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
39 req.set_header('Content-type', 'application/nquads')
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
40 graph.serialize(req, format='nquads')
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
41 elif acceptHeader == 'application/ld+json':
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
42 req.set_header('Content-type', 'application/ld+json')
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
43 graph.serialize(req, format='json-ld', indent=2)
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
44 else:
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
45 req.set_header('Content-type', 'application/x-trig')
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
46 graph.serialize(req, format='trig')
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
47
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
48 # forked from /my/proj/light9/light9/rdfdb/rdflibpatch.py
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
49 def _graphFromQuads2(q):
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
50 g = ConjunctiveGraph()
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
51 #g.addN(q) # no effect on nquad output
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
52 for s,p,o,c in q:
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
53 g.get_context(c).add((s,p,o)) # kind of works with broken rdflib nquad serializer code
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
54 #g.store.add((s,p,o), c) # no effect on nquad output
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
55 return g
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
56
1103
b84e956771fc sse_collector now kind of gets concurrent requests right
drewp <drewp@bigasterisk.com>
parents: 1038
diff changeset
57 def jsonFromPatch(p):
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
58 return json.dumps({'patch': {
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
59 'adds': from_rdf(_graphFromQuads2(p.addQuads)),
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
60 'deletes': from_rdf(_graphFromQuads2(p.delQuads)),
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
61 }})
1103
b84e956771fc sse_collector now kind of gets concurrent requests right
drewp <drewp@bigasterisk.com>
parents: 1038
diff changeset
62 patchAsJson = jsonFromPatch # deprecated name
b84e956771fc sse_collector now kind of gets concurrent requests right
drewp <drewp@bigasterisk.com>
parents: 1038
diff changeset
63
b84e956771fc sse_collector now kind of gets concurrent requests right
drewp <drewp@bigasterisk.com>
parents: 1038
diff changeset
64
b84e956771fc sse_collector now kind of gets concurrent requests right
drewp <drewp@bigasterisk.com>
parents: 1038
diff changeset
65 def patchFromJson(j):
b84e956771fc sse_collector now kind of gets concurrent requests right
drewp <drewp@bigasterisk.com>
parents: 1038
diff changeset
66 body = json.loads(j)['patch']
b84e956771fc sse_collector now kind of gets concurrent requests right
drewp <drewp@bigasterisk.com>
parents: 1038
diff changeset
67 a = ConjunctiveGraph()
b84e956771fc sse_collector now kind of gets concurrent requests right
drewp <drewp@bigasterisk.com>
parents: 1038
diff changeset
68 a.parse(StringInputSource(json.dumps(body['adds'])), format='json-ld')
b84e956771fc sse_collector now kind of gets concurrent requests right
drewp <drewp@bigasterisk.com>
parents: 1038
diff changeset
69 d = ConjunctiveGraph()
b84e956771fc sse_collector now kind of gets concurrent requests right
drewp <drewp@bigasterisk.com>
parents: 1038
diff changeset
70 d.parse(StringInputSource(json.dumps(body['deletes'])), format='json-ld')
b84e956771fc sse_collector now kind of gets concurrent requests right
drewp <drewp@bigasterisk.com>
parents: 1038
diff changeset
71 return Patch(addGraph=a, delGraph=d)
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
72
1038
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
73 def graphAsJson(g):
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
74 # This is not the same as g.serialize(format='json-ld')! That
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
75 # version omits literal datatypes.
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
76 return json.dumps(from_rdf(g))
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
77
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
78 _graphsInProcess = itertools.count()
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
79 class PatchableGraph(GraphEditApi):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
80 """
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
81 Master graph that you modify with self.patch, and we get the
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
82 updates to all current listeners.
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
83 """
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
84 def __init__(self):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
85 self._graph = ConjunctiveGraph()
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
86 self._observers = []
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
87 scales.init(self, '/patchableGraph%s' % next(_graphsInProcess))
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
88
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
89 _serialize = scales.PmfStat('serialize')
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
90 def serialize(self, to, **kw):
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
91 with self._serialize.time():
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
92 return self._graph.serialize(to, **kw)
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
93
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
94 _patch = scales.PmfStat('patch')
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
95 _len = scales.IntStat('statementCount')
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
96 def patch(self, p):
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
97 with self._patch.time():
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
98 # assuming no stmt is both in p.addQuads and p.delQuads.
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
99 dels = set([q for q in p.delQuads if inGraph(q, self._graph)])
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
100 adds = set([q for q in p.addQuads if not inGraph(q, self._graph)])
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
101 minimizedP = Patch(addQuads=adds, delQuads=dels)
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
102 if minimizedP.isNoop():
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
103 return
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
104 patchQuads(self._graph,
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
105 deleteQuads=dels,
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
106 addQuads=adds,
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
107 perfect=False) # true?
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
108 for ob in self._observers:
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
109 ob(patchAsJson(p))
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
110 self._len = len(self._graph)
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
111
1038
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
112 def asJsonLd(self):
ffe6a00c6cef server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp <drewp@bigasterisk.com>
parents: 1032
diff changeset
113 return graphAsJson(self._graph)
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
114
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
115 _currentObservers = scales.IntStat('observers/current')
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
116 _observersAdded = scales.IntStat('observers/added')
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
117 def addObserver(self, onPatch):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
118 self._observers.append(onPatch)
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
119 self._currentObservers = len(self._observers)
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
120 self._observersAdded += 1
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
121
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
122 def removeObserver(self, onPatch):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
123 try:
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
124 self._observers.remove(onPatch)
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
125 except ValueError:
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
126 pass
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
127 self._currentObservers = len(self._observers)
1155
3d478b05f9b1 add setToGraph
drewp <drewp@bigasterisk.com>
parents: 1136
diff changeset
128
3d478b05f9b1 add setToGraph
drewp <drewp@bigasterisk.com>
parents: 1136
diff changeset
129 def setToGraph(self, newGraph):
3d478b05f9b1 add setToGraph
drewp <drewp@bigasterisk.com>
parents: 1136
diff changeset
130 self.patch(Patch.fromDiff(self._graph, newGraph))
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
131
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
132 _sendSimpleGraph = scales.PmfStat('serve/simpleGraph')
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
133 _sendFullGraph = scales.PmfStat('serve/events/sendFull')
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
134 _sendPatch = scales.PmfStat('serve/events/sendPatch')
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
135
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
136 class CycloneGraphHandler(PrettyErrorHandler, cyclone.web.RequestHandler):
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
137 def initialize(self, masterGraph):
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
138 self.masterGraph = masterGraph
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
139
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
140 def get(self):
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
141 with self.masterGraph._sendSimpleGraph.time():
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
142 writeGraphResponse(self, self.masterGraph,
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
143 self.request.headers.get('accept'))
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
144
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
145
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
146 class CycloneGraphEventsHandler(cyclone.sse.SSEHandler):
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
147 """
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
148 One session with one client.
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
149
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
150 returns current graph plus future patches to keep remote version
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
151 in sync with ours.
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
152
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
153 intsead of turning off buffering all over, it may work for this
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
154 response to send 'x-accel-buffering: no', per
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
155 http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
156 """
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
157 def __init__(self, application, request, masterGraph):
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
158 cyclone.sse.SSEHandler.__init__(self, application, request)
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
159 self.masterGraph = masterGraph
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
160
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
161 def bind(self):
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
162 with self.masterGraph._sendFullGraph.time():
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
163 graphJson = self.masterGraph.asJsonLd()
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
164 log.debug("send fullGraph event: %s", graphJson)
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
165 self.sendEvent(message=graphJson, event=b'fullGraph')
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
166 self.masterGraph.addObserver(self.onPatch)
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
167
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
168 def onPatch(self, patchJson):
1276
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
169 with self.masterGraph._sendPatch.time():
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
170 # throttle and combine patches here- ideally we could see how
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
171 # long the latency to the client is to make a better rate choice
d40214eaca99 stats support and maybe a no-op filtering logic change snuck in there
drewp <drewp@bigasterisk.com>
parents: 1232
diff changeset
172 self.sendEvent(message=patchJson, event=b'patch')
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
173
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
174 def unbind(self):
1032
69aad813a94e fix patchablegraph unbind event
drewp <drewp@bigasterisk.com>
parents: 1029
diff changeset
175 self.masterGraph.removeObserver(self.onPatch)
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
176