homeauto: service/mqtt_to_rdf/stmt

annotate service/mqtt_to_rdf/stmt_chunk.py @ 1664:1a7c1261302c

logic fix- some bindings were being returned 2+; some 0 times

author	drewp@bigasterisk.com
date	Mon, 20 Sep 2021 23:19:08 -0700
parents	00a5624d1d14
children	89e53cb8a01c

rev	line source
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	1 import itertools
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	2 import logging
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	3 from dataclasses import dataclass
1659 15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	4 from typing import Iterable, Iterator, List, Optional, Set, Tuple, cast
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	5
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	6 from rdflib.graph import Graph
1659 15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	7 from rdflib.namespace import RDF
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	8 from rdflib.term import BNode, Literal, Node, URIRef, Variable
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	9
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	10 from candidate_binding import CandidateBinding
1661 00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1660 diff changeset	11 from inference_types import BindingUnknown, Inconsistent
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	12
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	13 log = logging.getLogger('infer')
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	14
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	15 INDENT = ' '
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	16
1660 31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	17 ChunkPrimaryTriple = Tuple[Optional[Node], Node, Optional[Node]]
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	18
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	19
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	20 @dataclass
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	21 class AlignedRuleChunk:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	22 """a possible association between a rule chunk and a workingSet chunk. Use
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	23 matches() to see if the rule actually fits (and then we might cache some of
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	24 that work when computing the new bindings"""
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	25 ruleChunk: 'Chunk'
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	26 workingSetChunk: 'Chunk'
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	27
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	28 def totalBindingIfThisStmtWereTrue(self, prevBindings: CandidateBinding) -> CandidateBinding:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	29 outBinding = prevBindings.copy()
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	30 for rt, ct in zip(self.ruleChunk._allTerms(), self.workingSetChunk._allTerms()):
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	31 if isinstance(rt, (Variable, BNode)):
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	32 if outBinding.contains(rt) and outBinding.applyTerm(rt) != ct:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	33 msg = f'{rt=} {ct=} {outBinding=}' if log.isEnabledFor(logging.DEBUG) else ''
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	34 raise Inconsistent(msg)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	35 outBinding.addNewBindings(CandidateBinding({rt: ct}))
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	36 return outBinding
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	37
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	38 # could combine this and totalBindingIf into a single ChunkMatch object
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	39 def matches(self) -> bool:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	40 """could this rule, with its BindableTerm wildcards, match workingSetChunk?"""
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	41 for selfTerm, otherTerm in zip(self.ruleChunk._allTerms(), self.workingSetChunk._allTerms()):
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	42 if not isinstance(selfTerm, (Variable, BNode)) and selfTerm != otherTerm:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	43 return False
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	44 return True
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	45
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	46
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	47 @dataclass
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	48 class Chunk: # rename this
1661 00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1660 diff changeset	49 """A statement, maybe with variables in it, except *the subject or object
00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1660 diff changeset	50 can be rdf lists*. This is done to optimize list comparisons (a lot) at the
00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1660 diff changeset	51 very minor expense of not handling certain exotic cases, such as a branching
00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1660 diff changeset	52 list.
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	53
1661 00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1660 diff changeset	54 Example: (?x ?y) math:sum ?z . <-- this becomes one Chunk.
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	55
1661 00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1660 diff changeset	56 A function call in a rule is always contained in exactly one chunk.
00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1660 diff changeset	57
00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1660 diff changeset	58 https://www.w3.org/TeamSubmission/n3/#:~:text=Implementations%20may%20treat%20list%20as%20a%20data%20type
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	59 """
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	60 # all immutable
1660 31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	61 primary: ChunkPrimaryTriple
1653 e7d594c065d4 minor refactoring drewp@bigasterisk.com parents: 1652 diff changeset	62 subjList: Optional[List[Node]] = None
e7d594c065d4 minor refactoring drewp@bigasterisk.com parents: 1652 diff changeset	63 objList: Optional[List[Node]] = None
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	64
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	65 def __post_init__(self):
1659 15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	66 if not (((self.primary[0] is not None) ^ (self.subjList is not None)) and
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	67 ((self.primary[2] is not None) ^ (self.objList is not None))):
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	68 raise TypeError("invalid chunk init")
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	69 self.predicate = self.primary[1]
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	70 self.sortKey = (self.primary, tuple(self.subjList or []), tuple(self.objList or []))
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	71
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	72 def __hash__(self):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	73 return hash(self.sortKey)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	74
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	75 def __lt__(self, other):
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	76 return self.sortKey < other.sortKey
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	77
1660 31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	78 def _allTerms(self) -> Iterator[Node]:
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	79 """the terms in `primary` plus the lists. Output order is undefined but stable between same-sized Chunks"""
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	80 yield self.primary[1]
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	81 if self.primary[0] is not None:
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	82 yield self.primary[0]
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	83 else:
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	84 yield from cast(List[Node], self.subjList)
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	85 if self.primary[2] is not None:
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	86 yield self.primary[2]
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	87 else:
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	88 yield from cast(List[Node], self.objList)
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	89
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	90 def ruleMatchesFrom(self, workingSet: 'ChunkedGraph') -> Iterator[AlignedRuleChunk]:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	91 """Chunks from workingSet where self, which may have BindableTerm wildcards, could match that workingSet Chunk."""
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	92 # if log.isEnabledFor(logging.DEBUG):
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	93 # log.debug(f'{INDENT*6} computing {self}.ruleMatchesFrom({workingSet}')
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	94 allChunksIter = workingSet.allChunks()
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	95 if "stable failures please":
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	96 allChunksIter = sorted(allChunksIter)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	97 for chunk in allChunksIter:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	98 aligned = AlignedRuleChunk(self, chunk)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	99 if aligned.matches():
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	100 yield aligned
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	101
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	102 def __repr__(self):
1659 15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	103 pre = ('+'.join('%s' % elem for elem in self.subjList) + '+' if self.subjList else '')
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	104 post = ('+' + '+'.join('%s' % elem for elem in self.objList) if self.objList else '')
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	105 return pre + repr(self.primary) + post
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	106
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	107 def isFunctionCall(self, functionsFor) -> bool:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	108 return bool(list(functionsFor(cast(URIRef, self.predicate))))
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	109
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	110 def isStatic(self) -> bool:
1660 31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	111 return all(_termIsStatic(s) for s in self._allTerms())
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	112
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	113 def apply(self, cb: CandidateBinding, returnBoundStatementsOnly=True) -> 'Chunk':
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	114 """Chunk like this one but with cb substitutions applied. If the flag is
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	115 True, we raise BindingUnknown instead of leaving a term unbound"""
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	116 fn = lambda t: cb.applyTerm(t, returnBoundStatementsOnly)
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	117 return Chunk(
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	118 (
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	119 fn(self.primary[0]) if self.primary[0] is not None else None, #
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	120 fn(self.primary[1]), #
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	121 fn(self.primary[2]) if self.primary[2] is not None else None),
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	122 subjList=[fn(t) for t in self.subjList] if self.subjList else None,
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	123 objList=[fn(t) for t in self.objList] if self.objList else None,
31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	124 )
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	125
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	126
1660 31f7dab6a60b function evaluation uses Chunk lists now and runs fast. Only a few edge cases still broken drewp@bigasterisk.com parents: 1659 diff changeset	127 def _termIsStatic(term: Optional[Node]) -> bool:
1659 15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	128 return isinstance(term, (URIRef, Literal)) or term is None
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	129
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	130
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	131 def applyChunky(cb: CandidateBinding,
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	132 g: Iterable[AlignedRuleChunk],
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	133 returnBoundStatementsOnly=True) -> Iterator[AlignedRuleChunk]:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	134 for aligned in g:
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	135 try:
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	136 bound = aligned.ruleChunk.apply(cb, returnBoundStatementsOnly=returnBoundStatementsOnly)
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	137 except BindingUnknown:
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	138 log.debug(f'{INDENT*7} CB.apply cant bind {aligned} using {cb.binding}')
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	139
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	140 continue
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	141 log.debug(f'{INDENT*7} CB.apply took {aligned} to {bound}')
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	142
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	143 yield AlignedRuleChunk(bound, aligned.workingSetChunk)
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	144
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	145
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	146 class ChunkedGraph:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	147 """a Graph converts 1-to-1 with a ChunkedGraph, where the Chunks have
1652 dddfa09ea0b9 debug logging and comments drewp@bigasterisk.com parents: 1651 diff changeset	148 combined some statements together. (The only exception is that bnodes for
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	149 rdf lists are lost)"""
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	150
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	151 def __init__(
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	152 self,
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	153 graph: Graph,
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	154 functionsFor # get rid of this- i'm just working around a circular import
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	155 ):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	156 self.chunksUsedByFuncs: Set[Chunk] = set()
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	157 self.staticChunks: Set[Chunk] = set()
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	158 self.patternChunks: Set[Chunk] = set()
1659 15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	159
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	160 firstNodes = {}
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	161 restNodes = {}
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	162 graphStmts = set()
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	163 for s, p, o in graph:
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	164 if p == RDF['first']:
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	165 firstNodes[s] = o
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	166 elif p == RDF['rest']:
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	167 restNodes[s] = o
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	168 else:
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	169 graphStmts.add((s, p, o))
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	170
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	171 def gatherList(start):
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	172 lst = []
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	173 cur = start
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	174 while cur != RDF['nil']:
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	175 lst.append(firstNodes[cur])
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	176 cur = restNodes[cur]
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	177 return lst
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	178
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	179 for s, p, o in graphStmts:
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	180 subjList = objList = None
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	181 if s in firstNodes:
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	182 subjList = gatherList(s)
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	183 s = None
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	184 if o in firstNodes:
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	185 objList = gatherList(o)
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	186 o = None
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	187 c = Chunk((s, p, o), subjList=subjList, objList=objList)
15e84c71beee parse lists from graph into the Chunks drewp@bigasterisk.com parents: 1654 diff changeset	188
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	189 if c.isFunctionCall(functionsFor):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	190 self.chunksUsedByFuncs.add(c)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	191 elif c.isStatic():
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	192 self.staticChunks.add(c)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	193 else:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	194 self.patternChunks.add(c)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	195
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	196 def allPredicatesExceptFunctions(self) -> Set[Node]:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	197 return set(ch.predicate for ch in itertools.chain(self.staticChunks, self.patternChunks))
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	198
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	199 def noPredicatesAppear(self, preds: Iterable[Node]) -> bool:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	200 return self.allPredicatesExceptFunctions().isdisjoint(preds)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	201
1654 d47832373b34 __nonzero__ is called __bool__ in py3! thanks for nothing, linters drewp@bigasterisk.com parents: 1653 diff changeset	202 def __bool__(self):
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	203 return bool(self.chunksUsedByFuncs) or bool(self.staticChunks) or bool(self.patternChunks)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	204
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	205 def __repr__(self):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	206 return f'ChunkedGraph({self.__dict__})'
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	207
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	208 def allChunks(self) -> Iterable[Chunk]:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	209 yield from itertools.chain(self.staticChunks, self.patternChunks, self.chunksUsedByFuncs)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	210
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	211 def __contains__(self, ch: Chunk) -> bool:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: diff changeset	212 return ch in self.allChunks()

Mercurial > code > home > repos > homeauto

annotate service/mqtt_to_rdf/stmt_chunk.py @ 1664:1a7c1261302c