Mercurial > code > home > repos > homeauto

annotate service/mqtt_to_rdf/stmt_chunk.py @ 1659:15e84c71beee

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
parse lists from graph into the Chunks
author drewp@bigasterisk.com
date Sun, 19 Sep 2021 14:42:39 -0700
parents d47832373b34
children 31f7dab6a60b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
1 import itertools
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
2 import logging
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
3 from dataclasses import dataclass
1659
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
4 from typing import Iterable, Iterator, List, Optional, Set, Tuple, cast
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
5
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
6 from rdflib.graph import Graph
1659
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
7 from rdflib.namespace import RDF
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
8 from rdflib.term import BNode, Literal, Node, URIRef, Variable
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
9
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
10 from candidate_binding import CandidateBinding
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
11 from inference_types import BindingUnknown, Inconsistent, Triple
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
12 from rdf_debug import graphDump
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
13
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
14 log = logging.getLogger('infer')
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
15
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
16 INDENT = ' '
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
17
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
18
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
19 @dataclass
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
20 class Chunk: # rename this
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
21 """a statement, maybe with variables in it, except *the object can be an rdf list*.
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
22 This is done to optimize list comparisons (a lot) at the very minor expense of not
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
23 handling certain exotic cases, such as a branching list.
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
24
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
25 Also the subject could be a list, e.g. for (?x ?y) math:sum ?z .
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
26
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
27 Also a function call in a rule is always contained in exactly one chunk.
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
28 """
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
29 # all immutable
1659
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
30 primary: Tuple[Optional[Node], Node, Optional[Node]]
1653
e7d594c065d4 minor refactoring
drewp@bigasterisk.com
parents: 1652
diff changeset
31 subjList: Optional[List[Node]] = None
e7d594c065d4 minor refactoring
drewp@bigasterisk.com
parents: 1652
diff changeset
32 objList: Optional[List[Node]] = None
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
33
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
34 def __post_init__(self):
1659
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
35 if not (((self.primary[0] is not None) ^ (self.subjList is not None)) and
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
36 ((self.primary[2] is not None) ^ (self.objList is not None))):
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
37 raise TypeError("invalid chunk init")
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
38 self.predicate = self.primary[1]
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
39 self.sortKey = (self.primary, tuple(self.subjList or []), tuple(self.objList or []))
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
40
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
41 def __hash__(self):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
42 return hash(self.sortKey)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
43
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
44 def __gt__(self, other):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
45 return self.sortKey > other.sortKey
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
46
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
47
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
48 def totalBindingIfThisStmtWereTrue(self, prevBindings: CandidateBinding, proposed: 'Chunk') -> CandidateBinding:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
49 outBinding = prevBindings.copy()
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
50 for rt, ct in zip(self.primary, proposed.primary):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
51 if isinstance(rt, (Variable, BNode)):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
52 if outBinding.contains(rt) and outBinding.applyTerm(rt) != ct:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
53 msg = f'{rt=} {ct=} {outBinding=}' if log.isEnabledFor(logging.DEBUG) else ''
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
54 raise Inconsistent(msg)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
55 outBinding.addNewBindings(CandidateBinding({rt: ct}))
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
56 return outBinding
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
57
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
58 def myMatches(self, g: 'ChunkedGraph') -> List['Chunk']:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
59 """Chunks from g where self, which may have BindableTerm wildcards, could match that chunk in g."""
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
60 out: List['Chunk'] = []
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
61 log.debug(f'{INDENT*6} {self}.myMatches({g}')
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
62 for ch in g.allChunks():
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
63 if self.matches(ch):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
64 out.append(ch)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
65 #out.sort() # probably leftover- remove?
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
66 return out
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
67
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
68 # could combine this and totalBindingIf into a single ChunkMatch object
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
69 def matches(self, other: 'Chunk') -> bool:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
70 """does this Chunk with potential BindableTerm wildcards match other?"""
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
71 for selfTerm, otherTerm in zip(self.primary, other.primary):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
72 if not isinstance(selfTerm, (Variable, BNode)) and selfTerm != otherTerm:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
73 return False
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
74 return True
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
75
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
76 def __repr__(self):
1659
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
77 pre = ('+'.join('%s' % elem for elem in self.subjList) + '+' if self.subjList else '')
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
78 post = ('+' + '+'.join('%s' % elem for elem in self.objList) if self.objList else '')
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
79 return pre + repr(self.primary) + post
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
80
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
81 def isFunctionCall(self, functionsFor) -> bool:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
82 return bool(list(functionsFor(cast(URIRef, self.predicate))))
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
83
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
84 def isStatic(self) -> bool:
1653
e7d594c065d4 minor refactoring
drewp@bigasterisk.com
parents: 1652
diff changeset
85 return (_stmtIsStatic(self.primary) and all(_termIsStatic(s) for s in (self.subjList or [])) and
e7d594c065d4 minor refactoring
drewp@bigasterisk.com
parents: 1652
diff changeset
86 all(_termIsStatic(s) for s in (self.objList or [])))
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
87
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
88
1653
e7d594c065d4 minor refactoring
drewp@bigasterisk.com
parents: 1652
diff changeset
89 def _stmtIsStatic(stmt: Triple) -> bool:
e7d594c065d4 minor refactoring
drewp@bigasterisk.com
parents: 1652
diff changeset
90 return all(_termIsStatic(t) for t in stmt)
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
91
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
92
1653
e7d594c065d4 minor refactoring
drewp@bigasterisk.com
parents: 1652
diff changeset
93 def _termIsStatic(term: Node) -> bool:
1659
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
94 return isinstance(term, (URIRef, Literal)) or term is None
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
95
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
96
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
97 def applyChunky(cb: CandidateBinding, g: Iterable[Chunk], returnBoundStatementsOnly=True) -> Iterator[Chunk]:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
98 for stmt in g:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
99 try:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
100 bound = Chunk(
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
101 (
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
102 cb.applyTerm(stmt.primary[0], returnBoundStatementsOnly), #
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
103 cb.applyTerm(stmt.primary[1], returnBoundStatementsOnly), #
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
104 cb.applyTerm(stmt.primary[2], returnBoundStatementsOnly)),
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
105 subjList=None,
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
106 objList=None)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
107 except BindingUnknown:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
108 log.debug(f'{INDENT*7} CB.apply cant bind {stmt} using {cb.binding}')
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
109
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
110 continue
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
111 log.debug(f'{INDENT*7} CB.apply took {stmt} to {bound}')
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
112
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
113 yield bound
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
114
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
115
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
116 class ChunkedGraph:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
117 """a Graph converts 1-to-1 with a ChunkedGraph, where the Chunks have
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
118 combined some statements together. (The only exception is that bnodes for
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
119 rdf lists are lost)"""
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
120
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
121 def __init__(
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
122 self,
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
123 graph: Graph,
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
124 functionsFor # get rid of this- i'm just working around a circular import
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
125 ):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
126 self.chunksUsedByFuncs: Set[Chunk] = set()
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
127 self.staticChunks: Set[Chunk] = set()
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
128 self.patternChunks: Set[Chunk] = set()
1659
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
129
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
130 firstNodes = {}
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
131 restNodes = {}
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
132 graphStmts = set()
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
133 for s, p, o in graph:
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
134 if p == RDF['first']:
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
135 firstNodes[s] = o
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
136 elif p == RDF['rest']:
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
137 restNodes[s] = o
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
138 else:
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
139 graphStmts.add((s, p, o))
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
140
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
141 def gatherList(start):
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
142 lst = []
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
143 cur = start
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
144 while cur != RDF['nil']:
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
145 lst.append(firstNodes[cur])
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
146 cur = restNodes[cur]
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
147 return lst
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
148
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
149 for s, p, o in graphStmts:
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
150 subjList = objList = None
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
151 if s in firstNodes:
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
152 subjList = gatherList(s)
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
153 s = None
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
154 if o in firstNodes:
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
155 objList = gatherList(o)
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
156 o = None
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
157 c = Chunk((s, p, o), subjList=subjList, objList=objList)
15e84c71beee parse lists from graph into the Chunks
drewp@bigasterisk.com
parents: 1654
diff changeset
158
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
159 if c.isFunctionCall(functionsFor):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
160 self.chunksUsedByFuncs.add(c)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
161 elif c.isStatic():
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
162 self.staticChunks.add(c)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
163 else:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
164 self.patternChunks.add(c)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
165
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
166 def allPredicatesExceptFunctions(self) -> Set[Node]:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
167 return set(ch.predicate for ch in itertools.chain(self.staticChunks, self.patternChunks))
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
168
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
169 def noPredicatesAppear(self, preds: Iterable[Node]) -> bool:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
170 return self.allPredicatesExceptFunctions().isdisjoint(preds)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
171
1654
d47832373b34 __nonzero__ is called __bool__ in py3! thanks for nothing, linters
drewp@bigasterisk.com
parents: 1653
diff changeset
172 def __bool__(self):
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
173 return bool(self.chunksUsedByFuncs) or bool(self.staticChunks) or bool(self.patternChunks)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
174
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
175 def __repr__(self):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
176 return f'ChunkedGraph({self.__dict__})'
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
177
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
178 def allChunks(self) -> Iterable[Chunk]:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
179 yield from itertools.chain(self.staticChunks, self.patternChunks, self.chunksUsedByFuncs)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
180
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
181 def value(self, subj, pred) -> Node: # throwaway
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
182 for s in self.allChunks():
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
183 s = s.primary
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
184 if (s[0], s[1]) == (subj, pred):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
185 return s[2]
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
186 raise ValueError("value not found")
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
187
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
188 def __contains__(self, ch: Chunk) -> bool:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents:
diff changeset
189 return ch in self.allChunks()