Mercurial > code > home > repos > homeauto

annotate lib/patchablegraph.py @ 1032:69aad813a94e

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fix patchablegraph unbind event Ignore-this: 96f7d4dd71f902730196e7517319192f darcs-hash:90040491939bac62a967c4eb631a8bfd96d9fdba
author drewp <drewp@bigasterisk.com>
date Tue, 26 Jan 2016 03:36:01 -0800
parents 4d36cae32a4c
children 4ebb5cc30002
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
1 """
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
2 Design:
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
3
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
4 1. Services each have (named) graphs, which they patch as things
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
5 change. PatchableGraph is an object for holding this graph.
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
6 2. You can http GET that graph, or ...
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
7 3. You can http GET/SSE that graph and hear about modifications to it
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
8 4. The client that got the graph holds and maintains a copy. The
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
9 client may merge together multiple graphs.
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
10 5. Client queries its graph with low-level APIs or client-side sparql.
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
11 6. When the graph changes, the client knows and can update itself at
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
12 low or high granularity.
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
13 """
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
14 import sys, json
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
15 import cyclone.sse
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
16 sys.path.append("/my/proj/light9")
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
17 from light9.rdfdb.grapheditapi import GraphEditApi
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
18 from rdflib import ConjunctiveGraph
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
19 from light9.rdfdb.rdflibpatch import patchQuads
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
20 from rdflib_jsonld.serializer import from_rdf
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
21 from cycloneerr import PrettyErrorHandler
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
22
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
23 def writeGraphResponse(req, graph, acceptHeader):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
24 if acceptHeader == 'application/nquads':
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
25 req.set_header('Content-type', 'application/nquads')
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
26 graph.serialize(req, format='nquads')
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
27 elif acceptHeader == 'application/ld+json':
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
28 req.set_header('Content-type', 'application/ld+json')
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
29 graph.serialize(req, format='json-ld', indent=2)
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
30 else:
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
31 req.set_header('Content-type', 'application/x-trig')
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
32 graph.serialize(req, format='trig')
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
33
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
34 # forked from /my/proj/light9/light9/rdfdb/rdflibpatch.py
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
35 def _graphFromQuads2(q):
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
36 g = ConjunctiveGraph()
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
37 #g.addN(q) # no effect on nquad output
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
38 for s,p,o,c in q:
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
39 g.get_context(c).add((s,p,o)) # kind of works with broken rdflib nquad serializer code
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
40 #g.store.add((s,p,o), c) # no effect on nquad output
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
41 return g
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
42
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
43 def patchAsJson(p):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
44 return json.dumps({'patch': {
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
45 'adds': from_rdf(_graphFromQuads2(p.addQuads)),
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
46 'deletes': from_rdf(_graphFromQuads2(p.delQuads)),
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
47 }})
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
48
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
49 class PatchableGraph(GraphEditApi):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
50 """
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
51 Master graph that you modify with self.patch, and we get the
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
52 updates to all current listeners.
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
53 """
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
54 def __init__(self):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
55 self._graph = ConjunctiveGraph()
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
56 self._observers = []
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
57
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
58 def serialize(self, to, **kw):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
59 return self._graph.serialize(to, **kw)
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
60
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
61 def patch(self, p):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
62 if p.isNoop():
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
63 return
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
64 patchQuads(self._graph,
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
65 deleteQuads=p.delQuads,
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
66 addQuads=p.addQuads,
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
67 perfect=False) # true?
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
68 for ob in self._observers:
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
69 ob(patchAsJson(p))
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
70
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
71 def addObserver(self, onPatch):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
72 self._observers.append(onPatch)
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
73
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
74 def removeObserver(self, onPatch):
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
75 try:
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
76 self._observers.remove(onPatch)
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
77 except ValueError:
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
78 pass
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
79
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
80
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
81 class CycloneGraphHandler(PrettyErrorHandler, cyclone.web.RequestHandler):
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
82 def initialize(self, masterGraph):
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
83 self.masterGraph = masterGraph
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
84
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
85 def get(self):
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
86 writeGraphResponse(self, self.masterGraph,
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
87 self.request.headers.get('accept'))
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
88
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
89 class CycloneGraphEventsHandler(cyclone.sse.SSEHandler):
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
90 """
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
91 One session with one client.
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
92
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
93 returns current graph plus future patches to keep remote version
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
94 in sync with ours.
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
95
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
96 intsead of turning off buffering all over, it may work for this
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
97 response to send 'x-accel-buffering: no', per
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
98 http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffering
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
99 """
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
100 def __init__(self, application, request, masterGraph):
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
101 cyclone.sse.SSEHandler.__init__(self, application, request)
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
102 self.masterGraph = masterGraph
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
103
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
104 def bind(self):
1029
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
105 self.sendEvent(
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
106 message=self.masterGraph.serialize(None, format='json-ld',
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
107 indent=None),
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
108 event='fullGraph')
4d36cae32a4c refactor /graph and /graph/events handlers to lib/
drewp <drewp@bigasterisk.com>
parents: 1028
diff changeset
109 self.masterGraph.addObserver(self.onPatch)
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
110
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
111 def onPatch(self, patchJson):
1032
69aad813a94e fix patchablegraph unbind event
drewp <drewp@bigasterisk.com>
parents: 1029
diff changeset
112 # throttle and combine patches here- ideally we could see how
69aad813a94e fix patchablegraph unbind event
drewp <drewp@bigasterisk.com>
parents: 1029
diff changeset
113 # long the latency to the client is to make a better rate choice
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
114 self.sendEvent(message=patchJson, event='patch')
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
115
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
116 def unbind(self):
1032
69aad813a94e fix patchablegraph unbind event
drewp <drewp@bigasterisk.com>
parents: 1029
diff changeset
117 self.masterGraph.removeObserver(self.onPatch)
1028
70d52fa8373a add new jsonld/SSE support to environment service as a test
drewp <drewp@bigasterisk.com>
parents:
diff changeset
118