homeauto: service/mqtt_to_rdf/inference.py annotate

annotate service/mqtt_to_rdf/inference.py @ 1640:4bb6f593ebf3

speedups: abort some rules faster

author	drewp@bigasterisk.com
date	Wed, 15 Sep 2021 23:56:02 -0700
parents	ae5ca4ba8954
children	5403c6343fa4

rev	line source
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	1 """
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	2 copied from reasoning 2021-08-29. probably same api. should
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	3 be able to lib/ this out
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	4 """
1588 0757fafbfdab WIP inferencer - partial var and function support drewp@bigasterisk.com parents: 1587 diff changeset	5 import itertools
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	6 import logging
1601 30463df12d89 infer() dumps stats drewp@bigasterisk.com parents: 1600 diff changeset	7 import time
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	8 from collections import defaultdict
1626 7b3656867185 metrics on input graph sizes drewp@bigasterisk.com parents: 1623 diff changeset	9 from dataclasses import dataclass
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	10 from typing import (Dict, Iterator, List, Optional, Sequence, Set, Tuple, Union, cast)
1588 0757fafbfdab WIP inferencer - partial var and function support drewp@bigasterisk.com parents: 1587 diff changeset	11
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	12 from prometheus_client import Histogram, Summary
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	13 from rdflib import RDF, BNode, Graph, Namespace
1589 5c1055be3c36 WIP more debugging, working towards bnode-matching support drewp@bigasterisk.com parents: 1588 diff changeset	14 from rdflib.graph import ConjunctiveGraph, ReadOnlyGraphAggregate
1640 4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	15 from rdflib.term import Node, URIRef, Variable
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	16
1638 0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet) drewp@bigasterisk.com parents: 1637 diff changeset	17 from candidate_binding import BindingConflict, CandidateBinding
1637 ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	18 from inference_types import BindingUnknown, ReadOnlyWorkingSet, Triple
1640 4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	19 from lhs_evaluation import functionsFor, lhsStmtsUsedByFuncs, rulePredicates
1605 449746d1598f WIP move evaluation to new file drewp@bigasterisk.com parents: 1603 diff changeset	20
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	21 log = logging.getLogger('infer')
1593 b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	22 INDENT = ' '
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	23
1626 7b3656867185 metrics on input graph sizes drewp@bigasterisk.com parents: 1623 diff changeset	24 INFER_CALLS = Summary('inference_infer_calls', 'calls')
7b3656867185 metrics on input graph sizes drewp@bigasterisk.com parents: 1623 diff changeset	25 INFER_GRAPH_SIZE = Histogram('inference_graph_size', 'statements', buckets=[2**x for x in range(2, 20, 2)])
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	26
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	27 ROOM = Namespace("http://projects.bigasterisk.com/room/")
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	28 LOG = Namespace('http://www.w3.org/2000/10/swap/log#')
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	29 MATH = Namespace('http://www.w3.org/2000/10/swap/math#')
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	30
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	31
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	32 def stmtTemplate(stmt: Triple) -> Tuple[Optional[Node], Optional[Node], Optional[Node]]:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	33 return (
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	34 None if isinstance(stmt[0], (Variable, BNode)) else stmt[0],
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	35 None if isinstance(stmt[1], (Variable, BNode)) else stmt[1],
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	36 None if isinstance(stmt[2], (Variable, BNode)) else stmt[2],
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	37 )
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	38
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	39
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	40 class NoOptions(ValueError):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	41 """stmtlooper has no possibilites to add to the binding; the whole rule must therefore not apply"""
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	42
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	43
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	44 class Inconsistent(ValueError):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	45 """adding this stmt would be inconsistent with an existing binding"""
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	46
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	47
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	48 _stmtLooperShortId = itertools.count()
bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	49
bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	50
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	51 @dataclass
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	52 class StmtLooper:
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	53 """given one LHS stmt, iterate through the possible matches for it,
bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	54 returning what bindings they would imply. Only distinct bindings are
bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	55 returned. The bindings build on any `prev` StmtLooper's results.
bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	56
bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	57 This iterator is restartable."""
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	58 lhsStmt: Triple
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	59 prev: Optional['StmtLooper']
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	60 workingSet: ReadOnlyWorkingSet
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	61 parent: 'Lhs' # just for lhs.graph, really
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	62
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	63 def __repr__(self):
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	64 return f'StmtLooper{self._shortId}({graphDump([self.lhsStmt])} {"<pastEnd>" if self.pastEnd() else ""})'
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	65
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	66 def __post_init__(self):
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	67 self._shortId = next(_stmtLooperShortId)
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	68 self._myWorkingSetMatches = self._myMatches(self.workingSet)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	69
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	70 self._current = CandidateBinding({})
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	71 self._pastEnd = False
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	72 self._seenBindings: List[CandidateBinding] = []
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	73 self.restart()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	74
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	75 def _myMatches(self, g: Graph) -> List[Triple]:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	76 template = stmtTemplate(self.lhsStmt)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	77
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	78 stmts = sorted(cast(Iterator[Triple], list(g.triples(template))))
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	79 # plus new lhs possibilties...
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	80 # log.debug(f'{INDENT*6} {self} find {len(stmts)=} in {len(self.workingSet)=}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	81
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	82 return stmts
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	83
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	84 def _prevBindings(self) -> CandidateBinding:
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	85 if not self.prev or self.prev.pastEnd():
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	86 return CandidateBinding({})
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	87
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	88 return self.prev.currentBinding()
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	89
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	90 def advance(self):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	91 """update to a new set of bindings we haven't seen (since last restart), or go into pastEnd mode"""
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	92 if self._pastEnd:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	93 raise NotImplementedError('need restart')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	94 log.debug('')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	95 augmentedWorkingSet: Sequence[Triple] = []
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	96 if self.prev is None:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	97 augmentedWorkingSet = self._myWorkingSetMatches
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	98 else:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	99 augmentedWorkingSet = list(self.prev.currentBinding().apply(self._myWorkingSetMatches,
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	100 returnBoundStatementsOnly=False))
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	101
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	102 log.debug(f'{INDENT*6} {self}.advance has {augmentedWorkingSet=}')
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	103
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	104 if self._advanceWithPlainMatches(augmentedWorkingSet):
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	105 return
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	106
1639 ae5ca4ba8954 rm unused computation drewp@bigasterisk.com parents: 1638 diff changeset	107 if self._advanceWithFunctions():
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	108 return
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	109
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	110 log.debug(f'{INDENT*6} {self} is past end')
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	111 self._pastEnd = True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	112
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	113 def _advanceWithPlainMatches(self, augmentedWorkingSet: Sequence[Triple]) -> bool:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	114 log.debug(f'{INDENT*7} {self} mines {len(augmentedWorkingSet)} matching augmented statements')
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	115 for s in augmentedWorkingSet:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	116 log.debug(f'{INDENT*7} {s}')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	117
1637 ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	118 for stmt in augmentedWorkingSet:
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	119 try:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	120 outBinding = self._totalBindingIfThisStmtWereTrue(stmt)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	121 except Inconsistent:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	122 log.debug(f'{INDENT*7} {self} - {stmt} would be inconsistent with prev bindings')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	123 continue
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	124
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	125 log.debug(f'{INDENT*7} {outBinding=} {self._seenBindings=}')
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	126 if outBinding not in self._seenBindings:
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	127 self._seenBindings.append(outBinding.copy())
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	128 self._current = outBinding
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	129 log.debug(f'{INDENT*7} new binding from {self} -> {outBinding}')
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	130 return True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	131 return False
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	132
1639 ae5ca4ba8954 rm unused computation drewp@bigasterisk.com parents: 1638 diff changeset	133 def _advanceWithFunctions(self) -> bool:
1637 ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	134 pred: Node = self.lhsStmt[1]
1640 4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	135 if not isinstance(pred, URIRef):
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	136 raise NotImplementedError
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	137
1637 ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	138 for functionType in functionsFor(pred):
ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	139 fn = functionType(self.lhsStmt, self.parent.graph)
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	140 try:
1637 ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	141 out = fn.bind(self._prevBindings())
ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	142 except BindingUnknown:
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	143 pass
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	144 else:
1637 ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	145 if out is not None:
ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	146 binding: CandidateBinding = self._prevBindings().copy()
ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	147 binding.addNewBindings(out)
ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	148 if binding not in self._seenBindings:
ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	149 self._seenBindings.append(binding)
ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	150 self._current = binding
ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	151 log.debug(f'{INDENT*7} new binding from {self} -> {binding}')
ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	152 return True
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	153
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	154 return False
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	155
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	156 def _boundOperands(self, operands) -> List[Node]:
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	157 pb: CandidateBinding = self._prevBindings()
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	158
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	159 boundOperands: List[Node] = []
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	160 for op in operands:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	161 if isinstance(op, (Variable, BNode)):
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	162 boundOperands.append(pb.applyTerm(op))
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	163 else:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	164 boundOperands.append(op)
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	165 return boundOperands
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	166
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	167 def _totalBindingIfThisStmtWereTrue(self, newStmt: Triple) -> CandidateBinding:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	168 outBinding = self._prevBindings().copy()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	169 for rt, ct in zip(self.lhsStmt, newStmt):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	170 if isinstance(rt, (Variable, BNode)):
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	171 if outBinding.contains(rt) and outBinding.applyTerm(rt) != ct:
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	172 raise Inconsistent(f'{rt=} {ct=} {outBinding=}')
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	173 outBinding.addNewBindings(CandidateBinding({rt: ct}))
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	174 return outBinding
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	175
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	176 def currentBinding(self) -> CandidateBinding:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	177 if self.pastEnd():
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	178 raise NotImplementedError()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	179 return self._current
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	180
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	181 def pastEnd(self) -> bool:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	182 return self._pastEnd
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	183
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	184 def restart(self):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	185 self._pastEnd = False
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	186 self._seenBindings = []
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	187 self.advance()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	188 if self.pastEnd():
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	189 raise NoOptions()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	190
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	191
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	192 @dataclass
1593 b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	193 class Lhs:
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	194 graph: Graph
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	195
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	196 def __post_init__(self):
1636 3252bdc284bc rm dead code from previous tries drewp@bigasterisk.com parents: 1635 diff changeset	197 pass
1602 e3c44ac6d3c5 do findEvals once at setRules time drewp@bigasterisk.com parents: 1601 diff changeset	198
1609 34f2817320cc new tests for a smaller part of the inner loop drewp@bigasterisk.com parents: 1608 diff changeset	199 def __repr__(self):
34f2817320cc new tests for a smaller part of the inner loop drewp@bigasterisk.com parents: 1608 diff changeset	200 return f"Lhs({graphDump(self.graph)})"
34f2817320cc new tests for a smaller part of the inner loop drewp@bigasterisk.com parents: 1608 diff changeset	201
1621 da235054caa0 rename workingSet, sometimes drewp@bigasterisk.com parents: 1620 diff changeset	202 def findCandidateBindings(self, knownTrue: ReadOnlyWorkingSet, stats) -> Iterator['BoundLhs']:
1593 b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	203 """bindings that fit the LHS of a rule, using statements from workingSet and functions
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	204 from LHS"""
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	205 if self.graph.__len__() == 0:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	206 # special case- no LHS!
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	207 yield BoundLhs(self, CandidateBinding({}))
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	208 return
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	209
1640 4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	210 if self._checkPredicateCounts(knownTrue):
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	211 stats['_checkPredicateCountsCulls'] += 1
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	212 return
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	213
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	214 log.debug(f'{INDENT*4} build new StmtLooper stack')
1593 b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	215
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	216 try:
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	217 stmtStack = self._assembleRings(knownTrue)
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	218 except NoOptions:
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	219 log.debug(f'{INDENT*5} start up with no options; 0 bindings')
1593 b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	220 return
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	221 self._debugStmtStack('initial odometer', stmtStack)
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	222 self._assertAllRingsAreValid(stmtStack)
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	223
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	224 lastRing = stmtStack[-1]
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	225 iterCount = 0
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	226 while True:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	227 iterCount += 1
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	228 if iterCount > 10:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	229 raise ValueError('stuck')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	230
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	231 log.debug(f'{INDENT*4} vv findCandBindings iteration {iterCount}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	232
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	233 yield BoundLhs(self, lastRing.currentBinding())
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	234
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	235 self._debugStmtStack('odometer', stmtStack)
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	236
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	237 done = self._advanceAll(stmtStack)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	238
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	239 self._debugStmtStack('odometer after ({done=})', stmtStack)
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	240
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	241 log.debug(f'{INDENT*4} ^^ findCandBindings iteration done')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	242 if done:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	243 break
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	244
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	245 def _debugStmtStack(self, label, stmtStack):
bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	246 log.debug(f'{INDENT*5} {label}:')
bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	247 for l in stmtStack:
bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	248 log.debug(f'{INDENT*6} {l} curbind={l.currentBinding() if not l.pastEnd() else "<end>"}')
bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	249
1640 4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	250 def _checkPredicateCounts(self, knownTrue):
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	251 """raise NoOptions quickly in some cases"""
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	252 myPreds = set(p for s, p, o in self.graph if isinstance(p, URIRef))
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	253 myPreds -= rulePredicates()
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	254 myPreds -= {RDF.first, RDF.rest}
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	255 if any((None, p, None) not in knownTrue for p in set(myPreds)):
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	256 return True
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	257 return False
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	258
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	259 def _assembleRings(self, knownTrue: ReadOnlyWorkingSet) -> List[StmtLooper]:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	260 """make StmtLooper for each stmt in our LHS graph, but do it in a way that they all
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	261 start out valid (or else raise NoOptions)"""
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	262
1637 ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	263 usedByFuncs: Set[Triple] = lhsStmtsUsedByFuncs(self.graph)
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	264
1637 ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	265 stmtsToAdd = list(self.graph - usedByFuncs)
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	266
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	267 # sort them by variable dependencies; don't just try all perms!
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	268 def lightSortKey(stmt): # Not this. Though it helps performance on the big rdf list cases.
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	269 (s, p, o) = stmt
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	270 return p == MATH['sum'], p, s, o
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	271
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	272 stmtsToAdd.sort(key=lightSortKey)
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	273
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	274 for perm in itertools.permutations(stmtsToAdd):
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	275 stmtStack: List[StmtLooper] = []
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	276 prev: Optional[StmtLooper] = None
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	277 log.debug(f'{INDENT*5} try stmts in this order: {" -> ".join(graphDump([p]) for p in perm)}')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	278
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	279 for s in perm:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	280 try:
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	281 elem = StmtLooper(s, prev, knownTrue, parent=self)
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	282 except NoOptions:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	283 log.debug(f'{INDENT*6} permutation didnt work, try another')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	284 break
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	285 stmtStack.append(elem)
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	286 prev = stmtStack[-1]
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	287 else:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	288 return stmtStack
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	289 log.debug(f'{INDENT*6} no perms worked- rule cannot match anything')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	290
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	291 raise NoOptions()
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	292
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	293 def _advanceAll(self, stmtStack: List[StmtLooper]) -> bool:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	294 carry = True # 1st elem always must advance
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	295 for i, ring in enumerate(stmtStack):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	296 # unlike normal odometer, advancing any earlier ring could invalidate later ones
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	297 if carry:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	298 log.debug(f'{INDENT*5} advanceAll [{i}] {ring} carry/advance')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	299 ring.advance()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	300 carry = False
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	301 if ring.pastEnd():
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	302 if ring is stmtStack[-1]:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	303 log.debug(f'{INDENT*5} advanceAll [{i}] {ring} says we done')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	304 return True
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	305 log.debug(f'{INDENT*5} advanceAll [{i}] {ring} restart')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	306 ring.restart()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	307 carry = True
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	308 return False
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	309
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	310 def _assertAllRingsAreValid(self, stmtStack):
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	311 if any(ring.pastEnd() for ring in stmtStack): # this is an unexpected debug assertion
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	312 log.debug(f'{INDENT*5} some rings started at pastEnd {stmtStack}')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	313 raise NoOptions()
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	314
1592 d7b66234064b pure reordering of funcs to make the next diffs smaller drewp@bigasterisk.com parents: 1591 diff changeset	315
1622 38bd8ef9ef67 add CandidateTermMatches, unused so far drewp@bigasterisk.com parents: 1621 diff changeset	316 @dataclass
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	317 class BoundLhs:
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	318 lhs: Lhs
1610 6fc48ef4c696 mysteriously lost an important line drewp@bigasterisk.com parents: 1609 diff changeset	319 binding: CandidateBinding
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	320
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	321
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	322 @dataclass
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	323 class Rule:
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	324 lhsGraph: Graph
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	325 rhsGraph: Graph
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	326
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	327 def __post_init__(self):
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	328 self.lhs = Lhs(self.lhsGraph)
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	329 #
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	330 self.rhsBnodeMap = {}
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	331
1612 272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	332 def applyRule(self, workingSet: Graph, implied: Graph, stats: Dict):
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	333 for bound in self.lhs.findCandidateBindings(ReadOnlyGraphAggregate([workingSet]), stats):
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	334 log.debug(f'{INDENT*5} +rule has a working binding: {bound}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	335
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	336 # rhs could have more bnodes, and they just need to be distinct per rule-firing that we do
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	337 existingRhsBnodes = set()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	338 for stmt in self.rhsGraph:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	339 for t in stmt:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	340 if isinstance(t, BNode):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	341 existingRhsBnodes.add(t)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	342 # if existingRhsBnodes:
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	343 # log.debug(f'{INDENT*6} mapping rhs bnodes {existingRhsBnodes} to new ones')
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	344
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	345 for b in existingRhsBnodes:
1612 272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	346
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	347 key = tuple(sorted(bound.binding.binding.items())), b
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	348 self.rhsBnodeMap.setdefault(key, BNode())
1638 0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet) drewp@bigasterisk.com parents: 1637 diff changeset	349 try:
0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet) drewp@bigasterisk.com parents: 1637 diff changeset	350 bound.binding.addNewBindings(CandidateBinding({b: self.rhsBnodeMap[key]}))
0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet) drewp@bigasterisk.com parents: 1637 diff changeset	351 except BindingConflict:
0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet) drewp@bigasterisk.com parents: 1637 diff changeset	352 continue
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	353
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	354 # for lhsBoundStmt in bound.binding.apply(bound.lhsStmtsWithoutEvals()):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	355 # log.debug(f'{INDENT*6} adding to workingSet {lhsBoundStmt=}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	356 # workingSet.add(lhsBoundStmt)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	357 # log.debug(f'{INDENT*6} rhsGraph is good: {list(self.rhsGraph)}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	358
1612 272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	359 for newStmt in bound.binding.apply(self.rhsGraph):
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	360 # log.debug(f'{INDENT*6} adding {newStmt=}')
1612 272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	361 workingSet.add(newStmt)
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	362 implied.add(newStmt)
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	363
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	364
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	365 class Inference:
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	366
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	367 def __init__(self) -> None:
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	368 self.rules = []
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	369
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	370 def setRules(self, g: ConjunctiveGraph):
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	371 self.rules: List[Rule] = []
1599 abbf0eb0e640 fix a bug with a slightly moer complicated set of rules drewp@bigasterisk.com parents: 1598 diff changeset	372 for stmt in g:
abbf0eb0e640 fix a bug with a slightly moer complicated set of rules drewp@bigasterisk.com parents: 1598 diff changeset	373 if stmt[1] == LOG['implies']:
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	374 self.rules.append(Rule(stmt[0], stmt[2]))
1612 272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	375 # other stmts should go to a default working set?
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	376
1601 30463df12d89 infer() dumps stats drewp@bigasterisk.com parents: 1600 diff changeset	377 @INFER_CALLS.time()
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	378 def infer(self, graph: Graph):
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	379 """
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	380 returns new graph of inferred statements.
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	381 """
1626 7b3656867185 metrics on input graph sizes drewp@bigasterisk.com parents: 1623 diff changeset	382 n = graph.__len__()
7b3656867185 metrics on input graph sizes drewp@bigasterisk.com parents: 1623 diff changeset	383 INFER_GRAPH_SIZE.observe(n)
7b3656867185 metrics on input graph sizes drewp@bigasterisk.com parents: 1623 diff changeset	384 log.info(f'{INDENT*0} Begin inference of graph len={n} with rules len={len(self.rules)}:')
1601 30463df12d89 infer() dumps stats drewp@bigasterisk.com parents: 1600 diff changeset	385 startTime = time.time()
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	386 stats: Dict[str, Union[int, float]] = defaultdict(lambda: 0)
1589 5c1055be3c36 WIP more debugging, working towards bnode-matching support drewp@bigasterisk.com parents: 1588 diff changeset	387 # everything that is true: the input graph, plus every rule conclusion we can make
1593 b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	388 workingSet = Graph()
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	389 workingSet += graph
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	390
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	391 # just the statements that came from RHS's of rules that fired.
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	392 implied = ConjunctiveGraph()
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	393
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	394 bailout_iterations = 100
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	395 delta = 1
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	396 stats['initWorkingSet'] = cast(int, workingSet.__len__())
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	397 while delta > 0 and bailout_iterations > 0:
1620 92f8deb59735 log layout drewp@bigasterisk.com parents: 1616 diff changeset	398 log.debug('')
1597 387a9cb66517 logging adjustments drewp@bigasterisk.com parents: 1596 diff changeset	399 log.info(f'{INDENT1}iteration ({bailout_iterations} left)')
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	400 bailout_iterations -= 1
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	401 delta = -len(implied)
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	402 self._iterateAllRules(workingSet, implied, stats)
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	403 delta += len(implied)
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	404 stats['iterations'] += 1
1597 387a9cb66517 logging adjustments drewp@bigasterisk.com parents: 1596 diff changeset	405 log.info(f'{INDENT*2} this inference iteration added {delta} more implied stmts')
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	406 stats['timeSpent'] = round(time.time() - startTime, 3)
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	407 stats['impliedStmts'] = len(implied)
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	408 log.info(f'{INDENT*0} Inference done {dict(stats)}. Implied:')
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	409 for st in implied:
1597 387a9cb66517 logging adjustments drewp@bigasterisk.com parents: 1596 diff changeset	410 log.info(f'{INDENT*1} {st}')
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	411 return implied
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	412
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	413 def _iterateAllRules(self, workingSet: Graph, implied: Graph, stats):
1612 272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	414 for i, rule in enumerate(self.rules):
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	415 self._logRuleApplicationHeader(workingSet, i, rule)
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	416 rule.applyRule(workingSet, implied, stats)
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	417
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	418 def _logRuleApplicationHeader(self, workingSet, i, r: Rule):
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	419 if not log.isEnabledFor(logging.DEBUG):
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	420 return
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	421
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	422 log.debug('')
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	423 log.debug(f'{INDENT*2} workingSet:')
1597 387a9cb66517 logging adjustments drewp@bigasterisk.com parents: 1596 diff changeset	424 for j, stmt in enumerate(sorted(workingSet)):
387a9cb66517 logging adjustments drewp@bigasterisk.com parents: 1596 diff changeset	425 log.debug(f'{INDENT*3} ({j}) {stmt}')
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	426
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	427 log.debug('')
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	428 log.debug(f'{INDENT*2}-applying rule {i}')
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	429 log.debug(f'{INDENT*3} rule def lhs:')
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	430 for stmt in sorted(r.lhsGraph, reverse=True):
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	431 log.debug(f'{INDENT*4} {stmt}')
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	432 log.debug(f'{INDENT*3} rule def rhs: {graphDump(r.rhsGraph)}')
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	433
1588 0757fafbfdab WIP inferencer - partial var and function support drewp@bigasterisk.com parents: 1587 diff changeset	434
1590 327202020892 WIP inference- getting into more degenerate test cases drewp@bigasterisk.com parents: 1589 diff changeset	435 def graphDump(g: Union[Graph, List[Triple]]):
1637 ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	436 # this is very slow- debug only!
ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	437 if not log.isEnabledFor(logging.DEBUG):
ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	438 return "(skipped dump)"
1590 327202020892 WIP inference- getting into more degenerate test cases drewp@bigasterisk.com parents: 1589 diff changeset	439 if not isinstance(g, Graph):
327202020892 WIP inference- getting into more degenerate test cases drewp@bigasterisk.com parents: 1589 diff changeset	440 g2 = Graph()
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	441 g2 += g
1590 327202020892 WIP inference- getting into more degenerate test cases drewp@bigasterisk.com parents: 1589 diff changeset	442 g = g2
1589 5c1055be3c36 WIP more debugging, working towards bnode-matching support drewp@bigasterisk.com parents: 1588 diff changeset	443 g.bind('', ROOM)
5c1055be3c36 WIP more debugging, working towards bnode-matching support drewp@bigasterisk.com parents: 1588 diff changeset	444 g.bind('ex', Namespace('http://example.com/'))
5c1055be3c36 WIP more debugging, working towards bnode-matching support drewp@bigasterisk.com parents: 1588 diff changeset	445 lines = cast(bytes, g.serialize(format='n3')).decode('utf8').splitlines()
1612 272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	446 lines = [line.strip() for line in lines if not line.startswith('@prefix')]
1589 5c1055be3c36 WIP more debugging, working towards bnode-matching support drewp@bigasterisk.com parents: 1588 diff changeset	447 return ' '.join(lines)

Mercurial > code > home > repos > homeauto

annotate service/mqtt_to_rdf/inference.py @ 1640:4bb6f593ebf3