Mercurial > code > home > repos > homeauto

annotate lib/patchablegraph/patchablegraph.py @ 712:d98c3ffe7144

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
new graph output for browsers, with autorefresh Ignore-this: e4ff4dbed311b238d90988a1891ef640
author drewp@bigasterisk.com
date Mon, 03 Feb 2020 23:47:23 -0800
parents e404420a2343
children 58f1877780b9
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
224
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
1 """
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
2 Design:
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
3
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
4 1. Services each have (named) graphs, which they patch as things
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
5 change. PatchableGraph is an object for holding this graph.
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
6 2. You can http GET that graph, or ...
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
7 3. You can http GET/SSE that graph and hear about modifications to it
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
8 4. The client that got the graph holds and maintains a copy. The
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
9 client may merge together multiple graphs.
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
10 5. Client queries its graph with low-level APIs or client-side sparql.
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
11 6. When the graph changes, the client knows and can update itself at
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
12 low or high granularity.
233
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
13
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
14
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
15 See also:
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
16 * http://iswc2007.semanticweb.org/papers/533.pdf RDFSync: efficient remote synchronization of RDF
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
17 models
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
18 * https://www.w3.org/2009/12/rdf-ws/papers/ws07 Supporting Change Propagation in RDF
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
19 * https://www.w3.org/DesignIssues/lncs04/Diff.pdf Delta: an ontology for the distribution of
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
20 differences between RDF graphs
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
21
224
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
22 """
712
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
23 import json, logging, itertools, html
514
495f573af4f4 make patchablegraph release
drewp@bigasterisk.com
parents: 474
diff changeset
24
495f573af4f4 make patchablegraph release
drewp@bigasterisk.com
parents: 474
diff changeset
25 from greplin import scales
331
a94f2a522d41 build and import updates for rdfdb, etc
drewp@bigasterisk.com
parents: 298
diff changeset
26 from rdfdb.grapheditapi import GraphEditApi
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
27 from rdflib import ConjunctiveGraph
712
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
28 from rdflib.namespace import NamespaceManager
514
495f573af4f4 make patchablegraph release
drewp@bigasterisk.com
parents: 474
diff changeset
29 from rdflib.parser import StringInputSource
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
30 from rdflib_jsonld.serializer import from_rdf
514
495f573af4f4 make patchablegraph release
drewp@bigasterisk.com
parents: 474
diff changeset
31 import cyclone.sse
224
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
32 from cycloneerr import PrettyErrorHandler
514
495f573af4f4 make patchablegraph release
drewp@bigasterisk.com
parents: 474
diff changeset
33 from rdfdb.patch import Patch
495f573af4f4 make patchablegraph release
drewp@bigasterisk.com
parents: 474
diff changeset
34 from rdfdb.rdflibpatch import patchQuads, inGraph
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
35
233
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
36 log = logging.getLogger('patchablegraph')
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
37
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
38 # forked from /my/proj/light9/light9/rdfdb/rdflibpatch.py
224
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
39 def _graphFromQuads2(q):
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
40 g = ConjunctiveGraph()
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
41 #g.addN(q) # no effect on nquad output
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
42 for s,p,o,c in q:
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
43 g.get_context(c).add((s,p,o)) # kind of works with broken rdflib nquad serializer code
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
44 #g.store.add((s,p,o), c) # no effect on nquad output
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
45 return g
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
46
298
8d89da1915df sse_collector now kind of gets concurrent requests right
drewp@bigasterisk.com
parents: 233
diff changeset
47 def jsonFromPatch(p):
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
48 return json.dumps({'patch': {
224
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
49 'adds': from_rdf(_graphFromQuads2(p.addQuads)),
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
50 'deletes': from_rdf(_graphFromQuads2(p.delQuads)),
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
51 }})
298
8d89da1915df sse_collector now kind of gets concurrent requests right
drewp@bigasterisk.com
parents: 233
diff changeset
52 patchAsJson = jsonFromPatch # deprecated name
8d89da1915df sse_collector now kind of gets concurrent requests right
drewp@bigasterisk.com
parents: 233
diff changeset
53
704
74ad667f8c5b reformat
drewp@bigasterisk.com
parents: 703
diff changeset
54
298
8d89da1915df sse_collector now kind of gets concurrent requests right
drewp@bigasterisk.com
parents: 233
diff changeset
55 def patchFromJson(j):
8d89da1915df sse_collector now kind of gets concurrent requests right
drewp@bigasterisk.com
parents: 233
diff changeset
56 body = json.loads(j)['patch']
8d89da1915df sse_collector now kind of gets concurrent requests right
drewp@bigasterisk.com
parents: 233
diff changeset
57 a = ConjunctiveGraph()
474
drewp@bigasterisk.com
parents: 473
diff changeset
58 a.parse(StringInputSource(json.dumps(body['adds']).encode('utf8')), format='json-ld')
298
8d89da1915df sse_collector now kind of gets concurrent requests right
drewp@bigasterisk.com
parents: 233
diff changeset
59 d = ConjunctiveGraph()
474
drewp@bigasterisk.com
parents: 473
diff changeset
60 d.parse(StringInputSource(json.dumps(body['deletes']).encode('utf8')), format='json-ld')
298
8d89da1915df sse_collector now kind of gets concurrent requests right
drewp@bigasterisk.com
parents: 233
diff changeset
61 return Patch(addGraph=a, delGraph=d)
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
62
233
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
63 def graphAsJson(g):
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
64 # This is not the same as g.serialize(format='json-ld')! That
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
65 # version omits literal datatypes.
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
66 return json.dumps(from_rdf(g))
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
67
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
68 _graphsInProcess = itertools.count()
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
69 class PatchableGraph(GraphEditApi):
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
70 """
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
71 Master graph that you modify with self.patch, and we get the
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
72 updates to all current listeners.
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
73 """
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
74 def __init__(self):
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
75 self._graph = ConjunctiveGraph()
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
76 self._observers = []
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
77 scales.init(self, '/patchableGraph%s' % next(_graphsInProcess))
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
78
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
79 _serialize = scales.PmfStat('serialize')
711
e404420a2343 don't require first arg on PatchedGraph.serialize
drewp@bigasterisk.com
parents: 708
diff changeset
80 def serialize(self, *arg, **kw):
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
81 with self._serialize.time():
711
e404420a2343 don't require first arg on PatchedGraph.serialize
drewp@bigasterisk.com
parents: 708
diff changeset
82 return self._graph.serialize(*arg, **kw)
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
83
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
84 _patch = scales.PmfStat('patch')
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
85 _len = scales.IntStat('statementCount')
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
86 def patch(self, p):
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
87 with self._patch.time():
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
88 # assuming no stmt is both in p.addQuads and p.delQuads.
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
89 dels = set([q for q in p.delQuads if inGraph(q, self._graph)])
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
90 adds = set([q for q in p.addQuads if not inGraph(q, self._graph)])
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
91 minimizedP = Patch(addQuads=adds, delQuads=dels)
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
92 if minimizedP.isNoop():
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
93 return
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
94 patchQuads(self._graph,
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
95 deleteQuads=dels,
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
96 addQuads=adds,
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
97 perfect=False) # true?
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
98 for ob in self._observers:
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
99 ob(patchAsJson(p))
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
100 self._len = len(self._graph)
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
101
233
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
102 def asJsonLd(self):
4ebb5cc30002 server/browser graph sync. cut dependency on the WS version. merge some changes between arduino/pi code.
drewp@bigasterisk.com
parents: 227
diff changeset
103 return graphAsJson(self._graph)
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
104
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
105 _currentObservers = scales.IntStat('observers/current')
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
106 _observersAdded = scales.IntStat('observers/added')
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
107 def addObserver(self, onPatch):
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
108 self._observers.append(onPatch)
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
109 self._currentObservers = len(self._observers)
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
110 self._observersAdded += 1
704
74ad667f8c5b reformat
drewp@bigasterisk.com
parents: 703
diff changeset
111
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
112 def removeObserver(self, onPatch):
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
113 try:
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
114 self._observers.remove(onPatch)
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
115 except ValueError:
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
116 pass
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
117 self._currentObservers = len(self._observers)
350
a380561fd8a8 add setToGraph
drewp@bigasterisk.com
parents: 331
diff changeset
118
a380561fd8a8 add setToGraph
drewp@bigasterisk.com
parents: 331
diff changeset
119 def setToGraph(self, newGraph):
a380561fd8a8 add setToGraph
drewp@bigasterisk.com
parents: 331
diff changeset
120 self.patch(Patch.fromDiff(self._graph, newGraph))
704
74ad667f8c5b reformat
drewp@bigasterisk.com
parents: 703
diff changeset
121
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
122 _sendSimpleGraph = scales.PmfStat('serve/simpleGraph')
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
123 _sendFullGraph = scales.PmfStat('serve/events/sendFull')
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
124 _sendPatch = scales.PmfStat('serve/events/sendPatch')
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
125
712
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
126
224
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
127 class CycloneGraphHandler(PrettyErrorHandler, cyclone.web.RequestHandler):
712
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
128 def initialize(self, masterGraph: PatchableGraph):
224
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
129 self.masterGraph = masterGraph
704
74ad667f8c5b reformat
drewp@bigasterisk.com
parents: 703
diff changeset
130
224
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
131 def get(self):
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
132 with self.masterGraph._sendSimpleGraph.time():
712
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
133 self._writeGraphResponse()
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
134
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
135 def _writeGraphResponse(self):
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
136 acceptHeader = self.request.headers.get(
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
137 'Accept',
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
138 # see https://github.com/fiorix/cyclone/issues/20
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
139 self.request.headers.get('accept', ''))
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
140
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
141 if acceptHeader == 'application/nquads':
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
142 self.set_header('Content-type', 'application/nquads')
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
143 self.masterGraph.serialize(self, format='nquads')
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
144 elif acceptHeader == 'application/ld+json':
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
145 self.set_header('Content-type', 'application/ld+json')
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
146 self.masterGraph.serialize(self, format='json-ld', indent=2)
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
147 else:
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
148 if acceptHeader.startswith('text/html'):
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
149 self._writeGraphForBrowser()
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
150 return
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
151 self.set_header('Content-type', 'application/x-trig')
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
152 self.masterGraph.serialize(self, format='trig')
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
153
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
154 def _writeGraphForBrowser(self):
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
155 # We think this is a browser, so respond with a live graph view
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
156 # (todo)
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
157 self.set_header('Content-type', 'text/html')
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
158
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
159 self.write(b'''
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
160 <html><body><pre>''')
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
161
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
162 ns = NamespaceManager(self.masterGraph._graph)
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
163 # maybe these could be on the PatchableGraph instance
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
164 ns.bind('ex', 'http://example.com/')
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
165 ns.bind('', 'http://projects.bigasterisk.com/room/')
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
166 ns.bind("rdf", "http://www.w3.org/1999/02/22-rdf-syntax-ns#")
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
167 ns.bind("xsd", "http://www.w3.org/2001/XMLSchema#")
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
168
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
169 for s, p, o, g in sorted(self.masterGraph._graph.quads()):
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
170 g = g.identifier
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
171 nquadLine = f'{s.n3(ns)} {p.n3(ns)} {o.n3(ns)} {g.n3(ns)} .\n'
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
172 self.write(html.escape(nquadLine).encode('utf8'))
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
173
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
174 self.write(b'''
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
175 </pre>
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
176 <p>
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
177 <a href="#">[refresh]</a>
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
178 <label><input type="checkbox"> Auto-refresh</label>
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
179 </p>
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
180 <script>
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
181
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
182 if (new URL(window.location).searchParams.get('autorefresh') == 'on') {
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
183 document.querySelector("input").checked = true;
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
184 setTimeout(() => {
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
185 requestAnimationFrame(() => {
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
186 window.location.replace(window.location.href);
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
187 });
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
188 }, 2000);
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
189 }
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
190
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
191 document.querySelector("a").addEventListener("click", (ev) => {
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
192 ev.preventDefault();
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
193 window.location.replace(window.location.href);
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
194
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
195 });
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
196 document.querySelector("input").addEventListener("change", (ev) => {
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
197 if (document.querySelector("input").checked) {
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
198 const u = new URL(window.location);
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
199 u.searchParams.set('autorefresh', 'on');
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
200 window.location.replace(u.href);
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
201 } else {
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
202 const u = new URL(window.location);
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
203 u.searchParams.delete('autorefresh');
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
204 window.location.replace(u.href);
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
205 }
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
206 });
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
207
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
208 </script>
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
209 </body></html>
d98c3ffe7144 new graph output for browsers, with autorefresh
drewp@bigasterisk.com
parents: 711
diff changeset
210 ''')
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
211
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
212
224
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
213 class CycloneGraphEventsHandler(cyclone.sse.SSEHandler):
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
214 """
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
215 One session with one client.
704
74ad667f8c5b reformat
drewp@bigasterisk.com
parents: 703
diff changeset
216
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
217 returns current graph plus future patches to keep remote version
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
218 in sync with ours.
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
219
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
220 intsead of turning off buffering all over, it may work for this
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
221 response to send 'x-accel-buffering: no', per
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
222 http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
223 """
224
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
224 def __init__(self, application, request, masterGraph):
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
225 cyclone.sse.SSEHandler.__init__(self, application, request)
596c645a1fc5 refactor /graph and /graph/events handlers to lib/
drewp@bigasterisk.com
parents: 223
diff changeset
226 self.masterGraph = masterGraph
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
227
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
228 def bind(self):
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
229 with self.masterGraph._sendFullGraph.time():
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
230 graphJson = self.masterGraph.asJsonLd()
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
231 log.debug("send fullGraph event: %s", graphJson)
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
232 self.sendEvent(message=graphJson, event=b'fullGraph')
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
233 self.masterGraph.addObserver(self.onPatch)
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
234
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
235 def onPatch(self, patchJson):
473
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
236 with self.masterGraph._sendPatch.time():
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
237 # throttle and combine patches here- ideally we could see how
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
238 # long the latency to the client is to make a better rate choice
388769b5f8ff stats support and maybe a no-op filtering logic change snuck in there
drewp@bigasterisk.com
parents: 429
diff changeset
239 self.sendEvent(message=patchJson, event=b'patch')
704
74ad667f8c5b reformat
drewp@bigasterisk.com
parents: 703
diff changeset
240
223
9236b736bc34 add new jsonld/SSE support to environment service as a test
drewp@bigasterisk.com
parents:
diff changeset
241 def unbind(self):
227
c1b98006f56e fix patchablegraph unbind event
drewp@bigasterisk.com
parents: 224
diff changeset
242 self.masterGraph.removeObserver(self.onPatch)