Mercurial > code > home > repos > rdfdb

annotate rdfdb/syncedgraph.py @ 62:bd0e24f6254a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
doc update on SyncedGraph Ignore-this: cec485f067fbde38bd99766e1c57eb68
author drewp@bigasterisk.com
date Fri, 31 May 2019 02:51:08 +0000
parents 7c3cd440619b
children a3718b297d16
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
1 """
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
2 client code uses a SyncedGraph, which has a few things:
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
3
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
4 AutoDepGraphApi - knockoutjs-inspired API for querying the graph in a
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
5 way that lets me call you again when there were changes to the things
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
6 you queried
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
7
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
8 CurrentStateGraphApi - a way to query the graph that doesn't gather
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
9 your dependencies like AutoDepGraphApi does
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
10
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
11 GraphEditApi - methods to write patches to the graph for common
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
12 operations, e.g. replacing a value, or editing a mapping
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
13
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
14 PatchReceiver - our web server that listens to edits from the master graph
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
15
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
16 PatchSender - collects and transmits your graph edits
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
17 """
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
18
48
7c3cd440619b switch to treq http client
Drew Perttula <drewp@bigasterisk.com>
parents: 45
diff changeset
19 import json, logging, traceback
7c3cd440619b switch to treq http client
Drew Perttula <drewp@bigasterisk.com>
parents: 45
diff changeset
20 from typing import Optional
7c3cd440619b switch to treq http client
Drew Perttula <drewp@bigasterisk.com>
parents: 45
diff changeset
21
43
8d4822ae58bc more types, as needed on light9 keyboardcomposer
Drew Perttula <drewp@bigasterisk.com>
parents: 31
diff changeset
22 from rdflib import ConjunctiveGraph, URIRef
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
23 from twisted.internet import defer
15
35e6e1211c49 don't require receiverhost. we need to open a 2-way connection anyhow
Drew Perttula <drewp@bigasterisk.com>
parents: 13
diff changeset
24 import socket
48
7c3cd440619b switch to treq http client
Drew Perttula <drewp@bigasterisk.com>
parents: 45
diff changeset
25 import treq
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
26
48
7c3cd440619b switch to treq http client
Drew Perttula <drewp@bigasterisk.com>
parents: 45
diff changeset
27 from rdfdb.autodepgraphapi import AutoDepGraphApi
1
674833ada390 no light9 package; rdfdb is now a toplevel name
Drew Perttula <drewp@bigasterisk.com>
parents: 0
diff changeset
28 from rdfdb.currentstategraphapi import CurrentStateGraphApi
674833ada390 no light9 package; rdfdb is now a toplevel name
Drew Perttula <drewp@bigasterisk.com>
parents: 0
diff changeset
29 from rdfdb.grapheditapi import GraphEditApi
43
8d4822ae58bc more types, as needed on light9 keyboardcomposer
Drew Perttula <drewp@bigasterisk.com>
parents: 31
diff changeset
30 from rdfdb.patch import Patch
48
7c3cd440619b switch to treq http client
Drew Perttula <drewp@bigasterisk.com>
parents: 45
diff changeset
31 from rdfdb.patchreceiver import PatchReceiver
7c3cd440619b switch to treq http client
Drew Perttula <drewp@bigasterisk.com>
parents: 45
diff changeset
32 from rdfdb.patchsender import PatchSender
7c3cd440619b switch to treq http client
Drew Perttula <drewp@bigasterisk.com>
parents: 45
diff changeset
33 from rdfdb.rdflibpatch import patchQuads
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
34
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
35 # everybody who writes literals needs to get this
24
edca5eaa34ca some py3
drewp@bigasterisk.com
parents: 15
diff changeset
36 from rdfdb.rdflibpatch_literal import patch
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
37 patch()
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
38
48
7c3cd440619b switch to treq http client
Drew Perttula <drewp@bigasterisk.com>
parents: 45
diff changeset
39 log = logging.getLogger('syncedgraph')
7c3cd440619b switch to treq http client
Drew Perttula <drewp@bigasterisk.com>
parents: 45
diff changeset
40
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
41
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
42 class SyncedGraph(CurrentStateGraphApi, AutoDepGraphApi, GraphEditApi):
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
43 """
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
44 graph for clients to use. Changes are synced with the master graph
62
bd0e24f6254a doc update on SyncedGraph
drewp@bigasterisk.com
parents: 48
diff changeset
45 in the rdfdb process.
bd0e24f6254a doc update on SyncedGraph
drewp@bigasterisk.com
parents: 48
diff changeset
46
bd0e24f6254a doc update on SyncedGraph
drewp@bigasterisk.com
parents: 48
diff changeset
47 self.patch(p: Patch) is the only way to write to the graph.
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
48
62
bd0e24f6254a doc update on SyncedGraph
drewp@bigasterisk.com
parents: 48
diff changeset
49 Reading can be done with the AutoDepGraphApi methods which set up
bd0e24f6254a doc update on SyncedGraph
drewp@bigasterisk.com
parents: 48
diff changeset
50 watchers to call you back when the results of the read have
bd0e24f6254a doc update on SyncedGraph
drewp@bigasterisk.com
parents: 48
diff changeset
51 changed (like knockoutjs). Or you can read with
bd0e24f6254a doc update on SyncedGraph
drewp@bigasterisk.com
parents: 48
diff changeset
52 CurrentStateGraphApi which doesn't have watchers, but you have to
bd0e24f6254a doc update on SyncedGraph
drewp@bigasterisk.com
parents: 48
diff changeset
53 opt into using it so it's clear you aren't in an auto-dep context
bd0e24f6254a doc update on SyncedGraph
drewp@bigasterisk.com
parents: 48
diff changeset
54 and meant to set up watchers.
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
55
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
56 You may want to attach to self.initiallySynced deferred so you
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
57 don't attempt patches before we've heard the initial contents of
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
58 the graph. It would be ok to accumulate some patches of new
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
59 material, but usually you won't correctly remove the existing
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
60 statements unless we have the correct graph.
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
61
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
62 If we get out of sync, we abandon our local graph (even any
62
bd0e24f6254a doc update on SyncedGraph
drewp@bigasterisk.com
parents: 48
diff changeset
63 pending local changes) and get the data again from the server.
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
64 """
45
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
65
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
66 def __init__(self,
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
67 rdfdbRoot: URIRef,
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
68 label: str,
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
69 receiverHost: Optional[str] = None):
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
70 """
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
71 label is a string that the server will display in association
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
72 with your connection
13
c9d1764d64ad add web server. remove more traces of light9
Drew Perttula <drewp@bigasterisk.com>
parents: 1
diff changeset
73
c9d1764d64ad add web server. remove more traces of light9
Drew Perttula <drewp@bigasterisk.com>
parents: 1
diff changeset
74 receiverHost is the hostname other nodes can use to talk to me
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
75 """
15
35e6e1211c49 don't require receiverhost. we need to open a 2-way connection anyhow
Drew Perttula <drewp@bigasterisk.com>
parents: 13
diff changeset
76 if receiverHost is None:
35e6e1211c49 don't require receiverhost. we need to open a 2-way connection anyhow
Drew Perttula <drewp@bigasterisk.com>
parents: 13
diff changeset
77 receiverHost = socket.gethostname()
45
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
78
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
79 self.rdfdbRoot = rdfdbRoot
31
3ca1a8774513 typing fixes. rewrite sequentialUri a bit
Drew Perttula <drewp@bigasterisk.com>
parents: 24
diff changeset
80 self.initiallySynced: defer.Deferred[None] = defer.Deferred()
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
81 self._graph = ConjunctiveGraph()
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
82
45
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
83 self._receiver = PatchReceiver(self.rdfdbRoot, receiverHost, label,
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
84 self._onPatch)
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
85
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
86 self._sender = PatchSender(self.rdfdbRoot + 'patches',
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
87 self._receiver.updateResource)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
88 AutoDepGraphApi.__init__(self)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
89 # this needs more state to track if we're doing a resync (and
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
90 # everything has to error or wait) or if we're live
45
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
91
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
92 def resync(self):
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
93 """
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
94 get the whole graph again from the server (e.g. we had a
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
95 conflict while applying a patch and want to return to the
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
96 truth).
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
97
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
98 To avoid too much churn, we remember our old graph and diff it
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
99 against the replacement. This way, our callers only see the
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
100 corrections.
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
101
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
102 Edits you make during a resync will surely be lost, so I
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
103 should just fail them. There should be a notification back to
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
104 UIs who want to show that we're doing a resync.
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
105 """
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
106 log.info('resync')
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
107 self._sender.cancelAll()
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
108 # this should be locked so only one resync goes on at once
48
7c3cd440619b switch to treq http client
Drew Perttula <drewp@bigasterisk.com>
parents: 45
diff changeset
109 return treq.get(self.rdfdbRoot.toPython() + "graph",
45
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
110 headers={
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
111 b'Accept': [b'x-trig']
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
112 },
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
113 ).addCallback(self._resyncGraph)
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
114
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
115 def _resyncGraph(self, response):
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
116 log.warn("new graph in")
45
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
117
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
118 #diff against old entire graph
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
119 #broadcast that change
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
120
43
8d4822ae58bc more types, as needed on light9 keyboardcomposer
Drew Perttula <drewp@bigasterisk.com>
parents: 31
diff changeset
121 def patch(self, p: Patch) -> None:
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
122 """send this patch to the server and apply it to our local
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
123 graph and run handlers"""
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
124
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
125 if p.isNoop():
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
126 log.info("skipping no-op patch")
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
127 return
45
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
128
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
129 # these could fail if we're out of sync. One approach:
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
130 # Rerequest the full state from the server, try the patch
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
131 # again after that, then give up.
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
132 debugKey = '[id=%s]' % (id(p) % 1000)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
133 log.debug("\napply local patch %s %s", debugKey, p)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
134 try:
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
135 patchQuads(self._graph,
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
136 deleteQuads=p.delQuads,
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
137 addQuads=p.addQuads,
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
138 perfect=True)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
139 except ValueError as e:
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
140 log.error(e)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
141 self.sendFailed(None)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
142 return
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
143 log.debug('runDepsOnNewPatch')
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
144 self.runDepsOnNewPatch(p)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
145 log.debug('sendPatch')
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
146 self._sender.sendPatch(p).addErrback(self.sendFailed)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
147 log.debug('patch is done %s', debugKey)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
148
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
149 def suggestPrefixes(self, ctx, prefixes):
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
150 """
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
151 when writing files for this ctx, try to use these n3
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
152 prefixes. async, not guaranteed to finish before any
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
153 particular file flush
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
154 """
44
2f188734adef fix treq payload type
Drew Perttula <drewp@bigasterisk.com>
parents: 43
diff changeset
155 treq.post(self.rdfdbRoot + 'prefixes',
45
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
156 json.dumps({
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
157 'ctx': ctx,
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
158 'prefixes': prefixes
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
159 }).encode('utf8'))
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
160
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
161 def sendFailed(self, result):
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
162 """
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
163 we asked for a patch to be queued and sent to the master, and
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
164 that ultimately failed because of a conflict
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
165 """
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
166 log.warn("sendFailed")
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
167 self.resync()
45
dc61012eeace python reformat
Drew Perttula <drewp@bigasterisk.com>
parents: 44
diff changeset
168
0
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
169 #i think we should receive back all the pending patches,
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
170 #do a resync here,
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
171 #then requeue all the pending patches (minus the failing one?) after that's done.
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
172
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
173 def _onPatch(self, p):
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
174 """
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
175 central server has sent us a patch
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
176 """
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
177 log.debug('_onPatch server has sent us %s', p)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
178 patchQuads(self._graph, p.delQuads, p.addQuads, perfect=True)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
179 log.debug("graph now has %s statements" % len(self._graph))
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
180 try:
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
181 self.runDepsOnNewPatch(p)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
182 except Exception:
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
183 # don't reflect this error back to the server; we did
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
184 # receive its patch correctly. However, we're in a bad
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
185 # state since some dependencies may not have rerun
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
186 traceback.print_exc()
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
187 log.warn("some graph dependencies may not have completely run")
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
188
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
189 if self.initiallySynced:
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
190 self.initiallySynced.callback(None)
d487d597ad33 initial split from light9/
Drew Perttula <drewp@bigasterisk.com>
parents:
diff changeset
191 self.initiallySynced = None