homeauto: service/mqtt_to_rdf/inference.py annotate

annotate service/mqtt_to_rdf/inference.py @ 1669:9d00adef0b22

rm used parameter

author	drewp@bigasterisk.com
date	Tue, 21 Sep 2021 23:20:16 -0700
parents	89e53cb8a01c
children	80f4e741ca4f

rev	line source
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	1 """
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	2 copied from reasoning 2021-08-29. probably same api. should
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	3 be able to lib/ this out
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	4 """
1588 0757fafbfdab WIP inferencer - partial var and function support drewp@bigasterisk.com parents: 1587 diff changeset	5 import itertools
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	6 import logging
1601 30463df12d89 infer() dumps stats drewp@bigasterisk.com parents: 1600 diff changeset	7 import time
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	8 from collections import defaultdict
1626 7b3656867185 metrics on input graph sizes drewp@bigasterisk.com parents: 1623 diff changeset	9 from dataclasses import dataclass
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	10 from typing import Dict, Iterator, List, Optional, Sequence, Union, cast
1588 0757fafbfdab WIP inferencer - partial var and function support drewp@bigasterisk.com parents: 1587 diff changeset	11
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	12 from prometheus_client import Histogram, Summary
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	13 from rdflib import BNode, Graph, Namespace
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	14 from rdflib.graph import ConjunctiveGraph
1640 4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	15 from rdflib.term import Node, URIRef, Variable
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	16
1638 0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet) drewp@bigasterisk.com parents: 1637 diff changeset	17 from candidate_binding import BindingConflict, CandidateBinding
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	18 from inference_types import BindingUnknown, Inconsistent, Triple
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	19 from lhs_evaluation import functionsFor
1650 2061df259224 move graphDump (on its way out, since reprs are getting better) drewp@bigasterisk.com parents: 1649 diff changeset	20 from rdf_debug import graphDump
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	21 from stmt_chunk import AlignedRuleChunk, Chunk, ChunkedGraph, applyChunky
1605 449746d1598f WIP move evaluation to new file drewp@bigasterisk.com parents: 1603 diff changeset	22
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	23 log = logging.getLogger('infer')
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	24 odolog = logging.getLogger('infer.odo') # the "odometer" logic
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	25 ringlog = logging.getLogger('infer.ring') # for ChunkLooper
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	26
1593 b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	27 INDENT = ' '
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	28
1626 7b3656867185 metrics on input graph sizes drewp@bigasterisk.com parents: 1623 diff changeset	29 INFER_CALLS = Summary('inference_infer_calls', 'calls')
7b3656867185 metrics on input graph sizes drewp@bigasterisk.com parents: 1623 diff changeset	30 INFER_GRAPH_SIZE = Histogram('inference_graph_size', 'statements', buckets=[2**x for x in range(2, 20, 2)])
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	31
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	32 ROOM = Namespace("http://projects.bigasterisk.com/room/")
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	33 LOG = Namespace('http://www.w3.org/2000/10/swap/log#')
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	34 MATH = Namespace('http://www.w3.org/2000/10/swap/math#')
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	35
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	36
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	37 class NoOptions(ValueError):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	38 """ChunkLooper has no possibilites to add to the binding; the whole rule must therefore not apply"""
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	39
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	40
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	41 _chunkLooperShortId = itertools.count()
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	42
bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	43
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	44 @dataclass
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	45 class ChunkLooper:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	46 """given one LHS Chunk, iterate through the possible matches for it,
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	47 returning what bindings they would imply. Only distinct bindings are
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	48 returned. The bindings build on any `prev` ChunkLooper's results.
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	49
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	50 In the odometer metaphor used below, this is one of the rings.
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	51
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	52 This iterator is restartable."""
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	53 lhsChunk: Chunk
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	54 prev: Optional['ChunkLooper']
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	55 workingSet: 'ChunkedGraph'
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	56
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	57 def __repr__(self):
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	58 return f'{self.__class__.__name__}{self._shortId}{"<pastEnd>" if self.pastEnd() else ""}'
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	59
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	60 def __post_init__(self):
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	61 self._shortId = next(_chunkLooperShortId)
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	62 self._alignedMatches = list(self.lhsChunk.ruleMatchesFrom(self.workingSet))
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	63
1668 89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	64 # only ours- do not store prev, since it could change without us
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	65 self._current = CandidateBinding({})
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	66 self._pastEnd = False
1667 a2347393b43e comments, debug, dead code drewp@bigasterisk.com parents: 1664 diff changeset	67 self._seenBindings: List[CandidateBinding] = [] # combined bindings (up to our ring) that we've returned
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	68
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	69 if ringlog.isEnabledFor(logging.DEBUG):
1667 a2347393b43e comments, debug, dead code drewp@bigasterisk.com parents: 1664 diff changeset	70 ringlog.debug('')
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	71 ringlog.debug(f'{INDENT*6} introducing {self!r}({self.lhsChunk}, {self._alignedMatches=})')
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	72
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	73 self.restart()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	74
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	75 def _prevBindings(self) -> CandidateBinding:
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	76 if not self.prev or self.prev.pastEnd():
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	77 return CandidateBinding({})
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	78
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	79 return self.prev.currentBinding()
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	80
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	81 def advance(self):
1668 89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	82 """update _current to a new set of valid bindings we haven't seen (since
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	83 last restart), or go into pastEnd mode. Note that _current is just our
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	84 contribution, but returned valid bindings include all prev rings."""
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	85 if self._pastEnd:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	86 raise NotImplementedError('need restart')
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	87 ringlog.debug('')
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	88 augmentedWorkingSet: List[AlignedRuleChunk] = []
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	89 if self.prev is None:
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	90 augmentedWorkingSet = self._alignedMatches
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	91 else:
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	92 augmentedWorkingSet = list(
1669 9d00adef0b22 rm used parameter drewp@bigasterisk.com parents: 1668 diff changeset	93 applyChunky(self.prev.currentBinding(), self._alignedMatches))
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	94
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	95 if self._advanceWithPlainMatches(augmentedWorkingSet):
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	96 ringlog.debug(f'{INDENT*6} <-- {self}.advance finished with plain matches')
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	97 return
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	98
1639 ae5ca4ba8954 rm unused computation drewp@bigasterisk.com parents: 1638 diff changeset	99 if self._advanceWithFunctions():
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	100 ringlog.debug(f'{INDENT*6} <-- {self}.advance finished with function matches')
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	101 return
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	102
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	103 ringlog.debug(f'{INDENT*6} <-- {self}.advance had nothing and is now past end')
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	104 self._pastEnd = True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	105
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	106 def _advanceWithPlainMatches(self, augmentedWorkingSet: List[AlignedRuleChunk]) -> bool:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	107 ringlog.debug(f'{INDENT*7} {self} mines {len(augmentedWorkingSet)} matching augmented statements')
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	108 for s in augmentedWorkingSet:
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	109 ringlog.debug(f'{INDENT*8} {s}')
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	110
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	111 for aligned in augmentedWorkingSet:
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	112 try:
1668 89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	113 newBinding = aligned.newBindingIfMatched(self._prevBindings())
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	114 except Inconsistent as exc:
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	115 ringlog.debug(
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	116 f'{INDENT*7} ChunkLooper{self._shortId} - {aligned} would be inconsistent with prev bindings ({exc})')
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	117 continue
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	118
1668 89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	119 if self._testAndKeepNewBinding(newBinding):
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	120 return True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	121 return False
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	122
1639 ae5ca4ba8954 rm unused computation drewp@bigasterisk.com parents: 1638 diff changeset	123 def _advanceWithFunctions(self) -> bool:
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	124 pred: Node = self.lhsChunk.predicate
1640 4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	125 if not isinstance(pred, URIRef):
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	126 raise NotImplementedError
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	127
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	128 ringlog.debug(f'{INDENT*6} advanceWithFunctions {pred!r}')
1652 dddfa09ea0b9 debug logging and comments drewp@bigasterisk.com parents: 1651 diff changeset	129
1637 ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	130 for functionType in functionsFor(pred):
1661 00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1656 diff changeset	131 fn = functionType(self.lhsChunk)
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	132 ringlog.debug(f'{INDENT*7} ChunkLooper{self._shortId} advanceWithFunctions, {functionType=}')
1652 dddfa09ea0b9 debug logging and comments drewp@bigasterisk.com parents: 1651 diff changeset	133
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	134 try:
1652 dddfa09ea0b9 debug logging and comments drewp@bigasterisk.com parents: 1651 diff changeset	135
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	136 newBinding = fn.bind(self._prevBindings())
1637 ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	137 except BindingUnknown:
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	138 pass
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	139 else:
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	140 if newBinding is not None:
1668 89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	141 if self._testAndKeepNewBinding(newBinding):
1637 ec3f98d0c1d8 refactor rules eval drewp@bigasterisk.com parents: 1636 diff changeset	142 return True
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	143
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	144 return False
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	145
1668 89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	146 def _testAndKeepNewBinding(self, newBinding):
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	147 fullBinding: CandidateBinding = self._prevBindings().copy()
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	148 fullBinding.addNewBindings(newBinding)
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	149 isNew = fullBinding not in self._seenBindings
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	150 ringlog.debug(f'{INDENT*7} {self} considering {newBinding=} to make {fullBinding}. {isNew=}')
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	151 if isNew:
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	152 self._seenBindings.append(fullBinding.copy())
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	153 self._current = newBinding
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	154 return True
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	155 return False
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	156
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	157 def _boundOperands(self, operands) -> List[Node]:
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	158 pb: CandidateBinding = self._prevBindings()
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	159
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	160 boundOperands: List[Node] = []
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	161 for op in operands:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	162 if isinstance(op, (Variable, BNode)):
1635 22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts drewp@bigasterisk.com parents: 1634 diff changeset	163 boundOperands.append(pb.applyTerm(op))
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	164 else:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	165 boundOperands.append(op)
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	166 return boundOperands
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	167
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	168 def currentBinding(self) -> CandidateBinding:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	169 if self.pastEnd():
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	170 raise NotImplementedError()
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	171 together = self._prevBindings().copy()
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	172 together.addNewBindings(self._current)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	173 return together
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	174
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	175 def pastEnd(self) -> bool:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	176 return self._pastEnd
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	177
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	178 def restart(self):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	179 self._pastEnd = False
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	180 self._seenBindings = []
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	181 self.advance()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	182 if self.pastEnd():
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	183 raise NoOptions()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	184
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	185
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	186 @dataclass
1593 b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	187 class Lhs:
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	188 graph: ChunkedGraph # our full LHS graph, as input. See below for the statements partitioned into groups.
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	189
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	190 def __post_init__(self):
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	191
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	192 self.myPreds = self.graph.allPredicatesExceptFunctions()
1602 e3c44ac6d3c5 do findEvals once at setRules time drewp@bigasterisk.com parents: 1601 diff changeset	193
1609 34f2817320cc new tests for a smaller part of the inner loop drewp@bigasterisk.com parents: 1608 diff changeset	194 def __repr__(self):
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	195 return f"Lhs({self.graph!r})"
1609 34f2817320cc new tests for a smaller part of the inner loop drewp@bigasterisk.com parents: 1608 diff changeset	196
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	197 def findCandidateBindings(self, knownTrue: ChunkedGraph, stats, ruleStatementsIterationLimit) -> Iterator['BoundLhs']:
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	198 """distinct bindings that fit the LHS of a rule, using statements from
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	199 workingSet and functions from LHS"""
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	200 if not self.graph:
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	201 # special case- no LHS!
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	202 yield BoundLhs(self, CandidateBinding({}))
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	203 return
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	204
1640 4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	205 if self._checkPredicateCounts(knownTrue):
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	206 stats['_checkPredicateCountsCulls'] += 1
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	207 return
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	208
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	209 if not all(ch in knownTrue for ch in self.graph.staticChunks):
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	210 stats['staticStmtCulls'] += 1
3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	211 return
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	212 # After this point we don't need to consider self.graph.staticChunks.
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	213
1656 d8e5b2232474 bugfix for simple (no arg lists) functions not evaluating drewp@bigasterisk.com parents: 1653 diff changeset	214 if not self.graph.patternChunks and not self.graph.chunksUsedByFuncs:
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	215 # static only
3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	216 yield BoundLhs(self, CandidateBinding({}))
3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	217 return
3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	218
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	219 log.debug('')
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	220 try:
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	221 chunkStack = self._assembleRings(knownTrue, stats)
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	222 except NoOptions:
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	223 ringlog.debug(f'{INDENT*5} start up with no options; 0 bindings')
1593 b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	224 return
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	225 self._debugChunkStack('time to spin: initial odometer is', chunkStack)
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	226 self._assertAllRingsAreValid(chunkStack)
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	227
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	228 lastRing = chunkStack[-1]
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	229 iterCount = 0
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	230 while True:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	231 iterCount += 1
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	232 if iterCount > ruleStatementsIterationLimit:
3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	233 raise ValueError('rule too complex')
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	234
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	235 log.debug(f'{INDENT*4} vv findCandBindings iteration {iterCount}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	236
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	237 yield BoundLhs(self, lastRing.currentBinding())
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	238
1667 a2347393b43e comments, debug, dead code drewp@bigasterisk.com parents: 1664 diff changeset	239 # self._debugChunkStack('odometer', chunkStack)
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	240
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	241 done = self._advanceTheStack(chunkStack)
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	242
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	243 self._debugChunkStack(f'odometer after ({done=})', chunkStack)
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	244
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	245 log.debug(f'{INDENT*4} ^^ findCandBindings iteration done')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	246 if done:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	247 break
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	248
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	249 def _debugChunkStack(self, label: str, chunkStack: List[ChunkLooper]):
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	250 odolog.debug(f'{INDENT*4} {label}:')
1652 dddfa09ea0b9 debug logging and comments drewp@bigasterisk.com parents: 1651 diff changeset	251 for i, l in enumerate(chunkStack):
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	252 odolog.debug(f'{INDENT*5} [{i}] {l} curbind={l.currentBinding() if not l.pastEnd() else "<end>"}')
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	253
1640 4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	254 def _checkPredicateCounts(self, knownTrue):
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	255 """raise NoOptions quickly in some cases"""
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	256
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	257 if self.graph.noPredicatesAppear(self.myPreds):
1652 dddfa09ea0b9 debug logging and comments drewp@bigasterisk.com parents: 1651 diff changeset	258 log.debug(f'{INDENT*3} checkPredicateCounts does cull because not all {self.myPreds=} are in knownTrue')
1640 4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	259 return True
1652 dddfa09ea0b9 debug logging and comments drewp@bigasterisk.com parents: 1651 diff changeset	260 log.debug(f'{INDENT*3} checkPredicateCounts does not cull because all {self.myPreds=} are in knownTrue')
1640 4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	261 return False
4bb6f593ebf3 speedups: abort some rules faster drewp@bigasterisk.com parents: 1639 diff changeset	262
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	263 def _assembleRings(self, knownTrue: ChunkedGraph, stats) -> List[ChunkLooper]:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	264 """make ChunkLooper for each stmt in our LHS graph, but do it in a way that they all
1652 dddfa09ea0b9 debug logging and comments drewp@bigasterisk.com parents: 1651 diff changeset	265 start out valid (or else raise NoOptions). static chunks have already been confirmed."""
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	266
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	267 log.debug(f'{INDENT*4} stats={dict(stats)}')
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	268 odolog.debug(f'{INDENT*3} build new ChunkLooper stack')
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	269 chunks = list(self.graph.patternChunks.union(self.graph.chunksUsedByFuncs))
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	270 chunks.sort(key=None)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	271 odolog.info(f' {INDENT*3} taking permutations of {len(chunks)=}')
1653 e7d594c065d4 minor refactoring drewp@bigasterisk.com parents: 1652 diff changeset	272 for i, perm in enumerate(itertools.permutations(chunks)):
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	273 looperRings: List[ChunkLooper] = []
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	274 prev: Optional[ChunkLooper] = None
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	275 if odolog.isEnabledFor(logging.DEBUG):
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	276 odolog.debug(f'{INDENT*4} [perm {i}] try rule chunks in this order: {" THEN ".join(repr(p) for p in perm)}')
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	277
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	278 for ruleChunk in perm:
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	279 try:
1661 00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1656 diff changeset	280 # These are getting rebuilt a lot which takes time. It would
00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1656 diff changeset	281 # be nice if they could accept a changing `prev` order
00a5624d1d14 cleanups and optimizations drewp@bigasterisk.com parents: 1656 diff changeset	282 # (which might already be ok).
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	283 looper = ChunkLooper(ruleChunk, prev, knownTrue)
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	284 except NoOptions:
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	285 odolog.debug(f'{INDENT*5} permutation didnt work, try another')
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	286 break
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	287 looperRings.append(looper)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	288 prev = looperRings[-1]
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	289 else:
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	290 # bug: At this point we've only shown that these are valid
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	291 # starting rings. The rules might be tricky enough that this
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	292 # permutation won't get us to the solution.
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	293 return looperRings
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	294 if i > 50000:
1652 dddfa09ea0b9 debug logging and comments drewp@bigasterisk.com parents: 1651 diff changeset	295 raise NotImplementedError(f'trying too many permutations {len(chunks)=}')
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	296
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	297 odolog.debug(f'{INDENT*5} no perms worked- rule cannot match anything')
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	298 raise NoOptions()
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	299
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	300 def _advanceTheStack(self, looperRings: List[ChunkLooper]) -> bool:
1668 89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	301 carry = True # last elem always must advance
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	302 for i, ring in reversed(list(enumerate(looperRings))):
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	303 # unlike normal odometer, advancing any earlier ring could invalidate later ones
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	304 if carry:
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	305 odolog.debug(f'{INDENT*4} advanceAll [{i}] {ring} carry/advance')
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	306 ring.advance()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	307 carry = False
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	308 if ring.pastEnd():
1668 89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos drewp@bigasterisk.com parents: 1667 diff changeset	309 if ring is looperRings[0]:
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	310 allRingsDone = [r.pastEnd() for r in looperRings]
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	311 odolog.debug(f'{INDENT*4} advanceAll [{i}] {ring} says we done {allRingsDone=}')
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	312 return True
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	313 odolog.debug(f'{INDENT*4} advanceAll [{i}] {ring} restart')
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	314 ring.restart()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	315 carry = True
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	316 return False
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	317
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	318 def _assertAllRingsAreValid(self, looperRings):
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	319 if any(ring.pastEnd() for ring in looperRings): # this is an unexpected debug assertion
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	320 odolog.warning(f'{INDENT*4} some rings started at pastEnd {looperRings}')
1633 6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	321 raise NoOptions()
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm drewp@bigasterisk.com parents: 1632 diff changeset	322
1592 d7b66234064b pure reordering of funcs to make the next diffs smaller drewp@bigasterisk.com parents: 1591 diff changeset	323
1622 38bd8ef9ef67 add CandidateTermMatches, unused so far drewp@bigasterisk.com parents: 1621 diff changeset	324 @dataclass
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	325 class BoundLhs:
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	326 lhs: Lhs
1610 6fc48ef4c696 mysteriously lost an important line drewp@bigasterisk.com parents: 1609 diff changeset	327 binding: CandidateBinding
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	328
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	329
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	330 @dataclass
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	331 class Rule:
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	332 lhsGraph: Graph
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	333 rhsGraph: Graph
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	334
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	335 def __post_init__(self):
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	336 self.lhs = Lhs(ChunkedGraph(self.lhsGraph, functionsFor))
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	337 #
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	338 self.rhsBnodeMap = {}
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	339
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	340 def applyRule(self, workingSet: Graph, implied: Graph, stats: Dict, ruleStatementsIterationLimit):
1651 20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	341 # this does not change for the current applyRule call. The rule will be
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	342 # tried again in an outer loop, in case it can produce more.
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	343 workingSetChunked = ChunkedGraph(workingSet, functionsFor)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	344
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples drewp@bigasterisk.com parents: 1650 diff changeset	345 for bound in self.lhs.findCandidateBindings(workingSetChunked, stats, ruleStatementsIterationLimit):
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	346 log.debug(f'{INDENT*5} +rule has a working binding: {bound}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	347
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	348 # rhs could have more bnodes, and they just need to be distinct per rule-firing that we do
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	349 existingRhsBnodes = set()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	350 for stmt in self.rhsGraph:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	351 for t in stmt:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	352 if isinstance(t, BNode):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	353 existingRhsBnodes.add(t)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	354 # if existingRhsBnodes:
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	355 # log.debug(f'{INDENT*6} mapping rhs bnodes {existingRhsBnodes} to new ones')
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	356
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	357 for b in existingRhsBnodes:
1612 272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	358
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	359 key = tuple(sorted(bound.binding.binding.items())), b
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	360 self.rhsBnodeMap.setdefault(key, BNode())
1638 0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet) drewp@bigasterisk.com parents: 1637 diff changeset	361 try:
0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet) drewp@bigasterisk.com parents: 1637 diff changeset	362 bound.binding.addNewBindings(CandidateBinding({b: self.rhsBnodeMap[key]}))
0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet) drewp@bigasterisk.com parents: 1637 diff changeset	363 except BindingConflict:
0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet) drewp@bigasterisk.com parents: 1637 diff changeset	364 continue
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	365
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	366 # for lhsBoundStmt in bound.binding.apply(bound.lhsStmtsWithoutEvals()):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	367 # log.debug(f'{INDENT*6} adding to workingSet {lhsBoundStmt=}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	368 # workingSet.add(lhsBoundStmt)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	369 # log.debug(f'{INDENT*6} rhsGraph is good: {list(self.rhsGraph)}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	370
1612 272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	371 for newStmt in bound.binding.apply(self.rhsGraph):
1631 2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over drewp@bigasterisk.com parents: 1627 diff changeset	372 # log.debug(f'{INDENT*6} adding {newStmt=}')
1612 272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	373 workingSet.add(newStmt)
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	374 implied.add(newStmt)
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	375
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	376
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	377 @dataclass
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	378 class Inference:
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	379 rulesIterationLimit = 3
1664 1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times drewp@bigasterisk.com parents: 1661 diff changeset	380 ruleStatementsIterationLimit = 5000
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	381
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	382 def __init__(self) -> None:
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	383 self.rules: List[Rule] = []
3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	384 self._nonRuleStmts: List[Triple] = []
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	385
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	386 def setRules(self, g: ConjunctiveGraph):
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	387 self.rules = []
3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	388 self._nonRuleStmts = []
1599 abbf0eb0e640 fix a bug with a slightly moer complicated set of rules drewp@bigasterisk.com parents: 1598 diff changeset	389 for stmt in g:
abbf0eb0e640 fix a bug with a slightly moer complicated set of rules drewp@bigasterisk.com parents: 1598 diff changeset	390 if stmt[1] == LOG['implies']:
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	391 self.rules.append(Rule(stmt[0], stmt[2]))
1649 bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit drewp@bigasterisk.com parents: 1648 diff changeset	392 else:
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit drewp@bigasterisk.com parents: 1648 diff changeset	393 self._nonRuleStmts.append(stmt)
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit drewp@bigasterisk.com parents: 1648 diff changeset	394
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit drewp@bigasterisk.com parents: 1648 diff changeset	395 def nonRuleStatements(self) -> List[Triple]:
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit drewp@bigasterisk.com parents: 1648 diff changeset	396 return self._nonRuleStmts
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	397
1601 30463df12d89 infer() dumps stats drewp@bigasterisk.com parents: 1600 diff changeset	398 @INFER_CALLS.time()
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	399 def infer(self, graph: Graph):
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	400 """
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	401 returns new graph of inferred statements.
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	402 """
1626 7b3656867185 metrics on input graph sizes drewp@bigasterisk.com parents: 1623 diff changeset	403 n = graph.__len__()
7b3656867185 metrics on input graph sizes drewp@bigasterisk.com parents: 1623 diff changeset	404 INFER_GRAPH_SIZE.observe(n)
7b3656867185 metrics on input graph sizes drewp@bigasterisk.com parents: 1623 diff changeset	405 log.info(f'{INDENT*0} Begin inference of graph len={n} with rules len={len(self.rules)}:')
1601 30463df12d89 infer() dumps stats drewp@bigasterisk.com parents: 1600 diff changeset	406 startTime = time.time()
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	407 stats: Dict[str, Union[int, float]] = defaultdict(lambda: 0)
1649 bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit drewp@bigasterisk.com parents: 1648 diff changeset	408
1589 5c1055be3c36 WIP more debugging, working towards bnode-matching support drewp@bigasterisk.com parents: 1588 diff changeset	409 # everything that is true: the input graph, plus every rule conclusion we can make
1593 b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	410 workingSet = Graph()
1649 bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit drewp@bigasterisk.com parents: 1648 diff changeset	411 workingSet += self._nonRuleStmts
1593 b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing drewp@bigasterisk.com parents: 1592 diff changeset	412 workingSet += graph
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	413
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	414 # just the statements that came from RHS's of rules that fired.
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	415 implied = ConjunctiveGraph()
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	416
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	417 rulesIterations = 0
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	418 delta = 1
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	419 stats['initWorkingSet'] = cast(int, workingSet.__len__())
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	420 while delta > 0 and rulesIterations <= self.rulesIterationLimit:
1620 92f8deb59735 log layout drewp@bigasterisk.com parents: 1616 diff changeset	421 log.debug('')
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	422 log.info(f'{INDENT1}iteration {rulesIterations}')
3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	423
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	424 delta = -len(implied)
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	425 self._iterateAllRules(workingSet, implied, stats)
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	426 delta += len(implied)
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	427 rulesIterations += 1
1597 387a9cb66517 logging adjustments drewp@bigasterisk.com parents: 1596 diff changeset	428 log.info(f'{INDENT*2} this inference iteration added {delta} more implied stmts')
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	429 stats['iterations'] = rulesIterations
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	430 stats['timeSpent'] = round(time.time() - startTime, 3)
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	431 stats['impliedStmts'] = len(implied)
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	432 log.info(f'{INDENT*0} Inference done {dict(stats)}.')
3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	433 log.debug('Implied:')
3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	434 log.debug(graphDump(implied))
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	435 return implied
9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	436
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	437 def _iterateAllRules(self, workingSet: Graph, implied: Graph, stats):
1612 272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	438 for i, rule in enumerate(self.rules):
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups. drewp@bigasterisk.com parents: 1611 diff changeset	439 self._logRuleApplicationHeader(workingSet, i, rule)
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	440 rule.applyRule(workingSet, implied, stats, self.ruleStatementsIterationLimit)
1587 9a3a18c494f9 WIP new inferencer. no vars yet. drewp@bigasterisk.com parents: diff changeset	441
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	442 def _logRuleApplicationHeader(self, workingSet, i, r: Rule):
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	443 if not log.isEnabledFor(logging.DEBUG):
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	444 return
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	445
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	446 log.debug('')
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	447 log.debug(f'{INDENT*2} workingSet:')
1648 3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	448 # for j, stmt in enumerate(sorted(workingSet)):
3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	449 # log.debug(f'{INDENT*3} ({j}) {stmt}')
3059f31b2dfa more performance work drewp@bigasterisk.com parents: 1641 diff changeset	450 log.debug(f'{INDENT*3} {graphDump(workingSet, oneLine=False)}')
1594 e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	451
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	452 log.debug('')
e58bcfa66093 cleanups and a few fixed cases drewp@bigasterisk.com parents: 1593 diff changeset	453 log.debug(f'{INDENT*2}-applying rule {i}')
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	454 log.debug(f'{INDENT*3} rule def lhs:')
1634 ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers drewp@bigasterisk.com parents: 1633 diff changeset	455 for stmt in sorted(r.lhsGraph, reverse=True):
1632 bd79a2941cab just (a lot of) debug changes drewp@bigasterisk.com parents: 1631 diff changeset	456 log.debug(f'{INDENT*4} {stmt}')
1607 b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs drewp@bigasterisk.com parents: 1605 diff changeset	457 log.debug(f'{INDENT*3} rule def rhs: {graphDump(r.rhsGraph)}')

Mercurial > code > home > repos > homeauto

annotate service/mqtt_to_rdf/inference.py @ 1669:9d00adef0b22