Mercurial > code > home > repos > homeauto

annotate service/mqtt_to_rdf/inference.py @ 1664:1a7c1261302c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
logic fix- some bindings were being returned 2+; some 0 times
author drewp@bigasterisk.com
date Mon, 20 Sep 2021 23:19:08 -0700
parents 00a5624d1d14
children a2347393b43e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
1 """
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
2 copied from reasoning 2021-08-29. probably same api. should
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
3 be able to lib/ this out
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
4 """
1588
0757fafbfdab WIP inferencer - partial var and function support
drewp@bigasterisk.com
parents: 1587
diff changeset
5 import itertools
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
6 import logging
1601
30463df12d89 infer() dumps stats
drewp@bigasterisk.com
parents: 1600
diff changeset
7 import time
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
8 from collections import defaultdict
1626
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
9 from dataclasses import dataclass
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
10 from typing import Dict, Iterator, List, Optional, Sequence, Union, cast
1588
0757fafbfdab WIP inferencer - partial var and function support
drewp@bigasterisk.com
parents: 1587
diff changeset
11
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
12 from prometheus_client import Histogram, Summary
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
13 from rdflib import BNode, Graph, Namespace
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
14 from rdflib.graph import ConjunctiveGraph
1640
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
15 from rdflib.term import Node, URIRef, Variable
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
16
1638
0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet)
drewp@bigasterisk.com
parents: 1637
diff changeset
17 from candidate_binding import BindingConflict, CandidateBinding
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
18 from inference_types import BindingUnknown, Inconsistent, Triple
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
19 from lhs_evaluation import functionsFor
1650
2061df259224 move graphDump (on its way out, since reprs are getting better)
drewp@bigasterisk.com
parents: 1649
diff changeset
20 from rdf_debug import graphDump
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
21 from stmt_chunk import AlignedRuleChunk, Chunk, ChunkedGraph, applyChunky
1605
449746d1598f WIP move evaluation to new file
drewp@bigasterisk.com
parents: 1603
diff changeset
22
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
23 log = logging.getLogger('infer')
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
24 odolog = logging.getLogger('infer.odo') # the "odometer" logic
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
25 ringlog = logging.getLogger('infer.ring') # for ChunkLooper
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
26
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
27 INDENT = ' '
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
28
1626
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
29 INFER_CALLS = Summary('inference_infer_calls', 'calls')
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
30 INFER_GRAPH_SIZE = Histogram('inference_graph_size', 'statements', buckets=[2**x for x in range(2, 20, 2)])
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
31
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
32 ROOM = Namespace("http://projects.bigasterisk.com/room/")
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
33 LOG = Namespace('http://www.w3.org/2000/10/swap/log#')
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
34 MATH = Namespace('http://www.w3.org/2000/10/swap/math#')
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
35
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
36
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
37 class NoOptions(ValueError):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
38 """ChunkLooper has no possibilites to add to the binding; the whole rule must therefore not apply"""
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
39
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
40
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
41 _chunkLooperShortId = itertools.count()
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
42
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
43
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
44 @dataclass
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
45 class ChunkLooper:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
46 """given one LHS Chunk, iterate through the possible matches for it,
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
47 returning what bindings they would imply. Only distinct bindings are
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
48 returned. The bindings build on any `prev` ChunkLooper's results.
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
49
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
50 In the odometer metaphor used below, this is one of the rings.
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
51
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
52 This iterator is restartable."""
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
53 lhsChunk: Chunk
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
54 prev: Optional['ChunkLooper']
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
55 workingSet: 'ChunkedGraph'
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
56
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
57 def __repr__(self):
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
58 return f'{self.__class__.__name__}{self._shortId}{"<pastEnd>" if self.pastEnd() else ""}'
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
59
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
60 def __post_init__(self):
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
61 self._shortId = next(_chunkLooperShortId)
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
62 self._alignedMatches = list(self.lhsChunk.ruleMatchesFrom(self.workingSet))
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
63
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
64 self._current = CandidateBinding({}) # only ours- do not store prev, since it could change without us
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
65 self._pastEnd = False
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
66 self._seenBindings: List[CandidateBinding] = [] # combined bindings (up to our ring) that we've returned
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
67
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
68 if ringlog.isEnabledFor(logging.DEBUG):
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
69 ringlog.debug(f'{INDENT*6} introducing {self!r}({self.lhsChunk}, {self._alignedMatches=})')
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
70
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
71 self.restart()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
72
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
73 def _prevBindings(self) -> CandidateBinding:
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
74 if not self.prev or self.prev.pastEnd():
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
75 return CandidateBinding({})
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
76
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
77 return self.prev.currentBinding()
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
78
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
79 def advance(self):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
80 """update to a new set of bindings we haven't seen (since last restart), or go into pastEnd mode"""
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
81 if self._pastEnd:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
82 raise NotImplementedError('need restart')
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
83 ringlog.debug('')
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
84 augmentedWorkingSet: List[AlignedRuleChunk] = []
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
85 if self.prev is None:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
86 augmentedWorkingSet = self._alignedMatches
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
87 else:
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
88 augmentedWorkingSet = list(
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
89 applyChunky(self.prev.currentBinding(), self._alignedMatches, returnBoundStatementsOnly=False))
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
90
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
91 ringlog.debug(f'{INDENT*6} --> {self}.advance has {augmentedWorkingSet=} {self._current=}')
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
92
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
93 if self._advanceWithPlainMatches(augmentedWorkingSet):
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
94 ringlog.debug(f'{INDENT*6} <-- {self}.advance finished with plain matches')
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
95 return
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
96
1639
ae5ca4ba8954 rm unused computation
drewp@bigasterisk.com
parents: 1638
diff changeset
97 if self._advanceWithFunctions():
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
98 ringlog.debug(f'{INDENT*6} <-- {self}.advance finished with function matches')
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
99 return
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
100
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
101 ringlog.debug(f'{INDENT*6} <-- {self}.advance had nothing and is now past end')
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
102 self._pastEnd = True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
103
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
104 def _advanceWithPlainMatches(self, augmentedWorkingSet: List[AlignedRuleChunk]) -> bool:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
105 ringlog.debug(f'{INDENT*7} {self} mines {len(augmentedWorkingSet)} matching augmented statements')
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
106 for s in augmentedWorkingSet:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
107 ringlog.debug(f'{INDENT*8} {s}')
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
108
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
109 for aligned in augmentedWorkingSet:
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
110 try:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
111 fullBinding = aligned.totalBindingIfThisStmtWereTrue(self._prevBindings())
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
112 except Inconsistent:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
113 ringlog.debug(f'{INDENT*7} ChunkLooper{self._shortId} - {aligned} would be inconsistent with prev bindings')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
114 continue
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
115
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
116 newBinding = fullBinding.copy()
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
117 newBinding.subtract(self._prevBindings())
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
118
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
119 ringlog.debug(f'{INDENT*7} {newBinding=} {self._seenBindings=}')
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
120 if fullBinding not in self._seenBindings:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
121 self._seenBindings.append(fullBinding.copy())
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
122 self._current = newBinding
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
123 ringlog.debug(f'{INDENT*7} new binding from {self} -> {fullBinding}')
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
124 return True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
125 return False
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
126
1639
ae5ca4ba8954 rm unused computation
drewp@bigasterisk.com
parents: 1638
diff changeset
127 def _advanceWithFunctions(self) -> bool:
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
128 pred: Node = self.lhsChunk.predicate
1640
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
129 if not isinstance(pred, URIRef):
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
130 raise NotImplementedError
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
131
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
132 ringlog.debug(f'{INDENT*6} advanceWithFunctions {pred!r}')
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
133
1637
ec3f98d0c1d8 refactor rules eval
drewp@bigasterisk.com
parents: 1636
diff changeset
134 for functionType in functionsFor(pred):
1661
00a5624d1d14 cleanups and optimizations
drewp@bigasterisk.com
parents: 1656
diff changeset
135 fn = functionType(self.lhsChunk)
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
136 ringlog.debug(f'{INDENT*7} ChunkLooper{self._shortId} advanceWithFunctions, {functionType=}')
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
137
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
138 try:
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
139
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
140 newBinding = fn.bind(self._prevBindings())
1637
ec3f98d0c1d8 refactor rules eval
drewp@bigasterisk.com
parents: 1636
diff changeset
141 except BindingUnknown:
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
142 pass
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
143 else:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
144 if newBinding is not None:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
145 fullBinding: CandidateBinding = self._prevBindings().copy()
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
146 fullBinding.addNewBindings(newBinding)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
147 if fullBinding not in self._seenBindings:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
148 self._seenBindings.append(fullBinding)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
149 self._current = newBinding
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
150 ringlog.debug(f'{INDENT*7} new binding from {self} -> {fullBinding}')
1637
ec3f98d0c1d8 refactor rules eval
drewp@bigasterisk.com
parents: 1636
diff changeset
151 return True
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
152
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
153 return False
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
154
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
155 def _boundOperands(self, operands) -> List[Node]:
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
156 pb: CandidateBinding = self._prevBindings()
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
157
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
158 boundOperands: List[Node] = []
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
159 for op in operands:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
160 if isinstance(op, (Variable, BNode)):
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
161 boundOperands.append(pb.applyTerm(op))
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
162 else:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
163 boundOperands.append(op)
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
164 return boundOperands
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
165
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
166 def currentBinding(self) -> CandidateBinding:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
167 if self.pastEnd():
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
168 raise NotImplementedError()
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
169 together = self._prevBindings().copy()
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
170 together.addNewBindings(self._current)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
171 return together
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
172
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
173 def pastEnd(self) -> bool:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
174 return self._pastEnd
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
175
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
176 def restart(self):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
177 self._pastEnd = False
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
178 self._seenBindings = []
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
179 self.advance()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
180 if self.pastEnd():
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
181 raise NoOptions()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
182
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
183
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
184 @dataclass
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
185 class Lhs:
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
186 graph: ChunkedGraph # our full LHS graph, as input. See below for the statements partitioned into groups.
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
187
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
188 def __post_init__(self):
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
189
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
190 self.myPreds = self.graph.allPredicatesExceptFunctions()
1602
e3c44ac6d3c5 do findEvals once at setRules time
drewp@bigasterisk.com
parents: 1601
diff changeset
191
1609
34f2817320cc new tests for a smaller part of the inner loop
drewp@bigasterisk.com
parents: 1608
diff changeset
192 def __repr__(self):
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
193 return f"Lhs({self.graph!r})"
1609
34f2817320cc new tests for a smaller part of the inner loop
drewp@bigasterisk.com
parents: 1608
diff changeset
194
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
195 def findCandidateBindings(self, knownTrue: ChunkedGraph, stats, ruleStatementsIterationLimit) -> Iterator['BoundLhs']:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
196 """distinct bindings that fit the LHS of a rule, using statements from
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
197 workingSet and functions from LHS"""
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
198 if not self.graph:
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
199 # special case- no LHS!
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
200 yield BoundLhs(self, CandidateBinding({}))
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
201 return
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
202
1640
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
203 if self._checkPredicateCounts(knownTrue):
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
204 stats['_checkPredicateCountsCulls'] += 1
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
205 return
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
206
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
207 if not all(ch in knownTrue for ch in self.graph.staticChunks):
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
208 stats['staticStmtCulls'] += 1
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
209 return
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
210 # After this point we don't need to consider self.graph.staticChunks.
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
211
1656
d8e5b2232474 bugfix for simple (no arg lists) functions not evaluating
drewp@bigasterisk.com
parents: 1653
diff changeset
212 if not self.graph.patternChunks and not self.graph.chunksUsedByFuncs:
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
213 # static only
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
214 yield BoundLhs(self, CandidateBinding({}))
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
215 return
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
216
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
217 log.debug('')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
218 try:
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
219 chunkStack = self._assembleRings(knownTrue, stats)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
220 except NoOptions:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
221 ringlog.debug(f'{INDENT*5} start up with no options; 0 bindings')
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
222 return
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
223 self._debugChunkStack('time to spin: initial odometer is', chunkStack)
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
224 self._assertAllRingsAreValid(chunkStack)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
225
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
226 lastRing = chunkStack[-1]
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
227 iterCount = 0
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
228 while True:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
229 iterCount += 1
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
230 if iterCount > ruleStatementsIterationLimit:
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
231 raise ValueError('rule too complex')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
232
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
233 log.debug(f'{INDENT*4} vv findCandBindings iteration {iterCount}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
234
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
235 yield BoundLhs(self, lastRing.currentBinding())
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
236
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
237 self._debugChunkStack('odometer', chunkStack)
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
238
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
239 done = self._advanceTheStack(chunkStack)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
240
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
241 self._debugChunkStack(f'odometer after ({done=})', chunkStack)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
242
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
243 log.debug(f'{INDENT*4} ^^ findCandBindings iteration done')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
244 if done:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
245 break
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
246
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
247 def _debugChunkStack(self, label: str, chunkStack: List[ChunkLooper]):
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
248 odolog.debug(f'{INDENT*4} {label}:')
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
249 for i, l in enumerate(chunkStack):
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
250 odolog.debug(f'{INDENT*5} [{i}] {l} curbind={l.currentBinding() if not l.pastEnd() else "<end>"}')
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
251
1640
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
252 def _checkPredicateCounts(self, knownTrue):
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
253 """raise NoOptions quickly in some cases"""
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
254
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
255 if self.graph.noPredicatesAppear(self.myPreds):
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
256 log.debug(f'{INDENT*3} checkPredicateCounts does cull because not all {self.myPreds=} are in knownTrue')
1640
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
257 return True
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
258 log.debug(f'{INDENT*3} checkPredicateCounts does not cull because all {self.myPreds=} are in knownTrue')
1640
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
259 return False
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
260
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
261 def _assembleRings(self, knownTrue: ChunkedGraph, stats) -> List[ChunkLooper]:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
262 """make ChunkLooper for each stmt in our LHS graph, but do it in a way that they all
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
263 start out valid (or else raise NoOptions). static chunks have already been confirmed."""
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
264
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
265 log.debug(f'{INDENT*4} stats={dict(stats)}')
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
266 odolog.debug(f'{INDENT*3} build new ChunkLooper stack')
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
267 chunks = list(self.graph.patternChunks.union(self.graph.chunksUsedByFuncs))
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
268 chunks.sort(key=None)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
269 odolog.info(f' {INDENT*3} taking permutations of {len(chunks)=}')
1653
e7d594c065d4 minor refactoring
drewp@bigasterisk.com
parents: 1652
diff changeset
270 for i, perm in enumerate(itertools.permutations(chunks)):
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
271 looperRings: List[ChunkLooper] = []
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
272 prev: Optional[ChunkLooper] = None
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
273 if odolog.isEnabledFor(logging.DEBUG):
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
274 odolog.debug(f'{INDENT*4} [perm {i}] try rule chunks in this order: {" THEN ".join(repr(p) for p in perm)}')
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
275
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
276 for ruleChunk in perm:
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
277 try:
1661
00a5624d1d14 cleanups and optimizations
drewp@bigasterisk.com
parents: 1656
diff changeset
278 # These are getting rebuilt a lot which takes time. It would
00a5624d1d14 cleanups and optimizations
drewp@bigasterisk.com
parents: 1656
diff changeset
279 # be nice if they could accept a changing `prev` order
00a5624d1d14 cleanups and optimizations
drewp@bigasterisk.com
parents: 1656
diff changeset
280 # (which might already be ok).
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
281 looper = ChunkLooper(ruleChunk, prev, knownTrue)
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
282 except NoOptions:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
283 odolog.debug(f'{INDENT*5} permutation didnt work, try another')
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
284 break
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
285 looperRings.append(looper)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
286 prev = looperRings[-1]
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
287 else:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
288 # bug: At this point we've only shown that these are valid
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
289 # starting rings. The rules might be tricky enough that this
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
290 # permutation won't get us to the solution.
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
291 return looperRings
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
292 if i > 50000:
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
293 raise NotImplementedError(f'trying too many permutations {len(chunks)=}')
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
294
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
295 odolog.debug(f'{INDENT*5} no perms worked- rule cannot match anything')
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
296 raise NoOptions()
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
297
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
298 def _advanceTheStack(self, looperRings: List[ChunkLooper]) -> bool:
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
299 carry = True # 1st elem always must advance
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
300 for i, ring in enumerate(looperRings):
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
301 # unlike normal odometer, advancing any earlier ring could invalidate later ones
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
302 if carry:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
303 odolog.debug(f'{INDENT*4} advanceAll [{i}] {ring} carry/advance')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
304 ring.advance()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
305 carry = False
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
306 if ring.pastEnd():
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
307 if ring is looperRings[-1]:
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
308 allRingsDone = [r.pastEnd() for r in looperRings]
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
309 odolog.debug(f'{INDENT*4} advanceAll [{i}] {ring} says we done {allRingsDone=}')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
310 return True
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
311 odolog.debug(f'{INDENT*4} advanceAll [{i}] {ring} restart')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
312 ring.restart()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
313 carry = True
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
314 return False
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
315
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
316 def _assertAllRingsAreValid(self, looperRings):
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
317 if any(ring.pastEnd() for ring in looperRings): # this is an unexpected debug assertion
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
318 odolog.warning(f'{INDENT*4} some rings started at pastEnd {looperRings}')
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
319 raise NoOptions()
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
320
1592
d7b66234064b pure reordering of funcs to make the next diffs smaller
drewp@bigasterisk.com
parents: 1591
diff changeset
321
1622
38bd8ef9ef67 add CandidateTermMatches, unused so far
drewp@bigasterisk.com
parents: 1621
diff changeset
322 @dataclass
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
323 class BoundLhs:
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
324 lhs: Lhs
1610
6fc48ef4c696 mysteriously lost an important line
drewp@bigasterisk.com
parents: 1609
diff changeset
325 binding: CandidateBinding
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
326
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
327
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
328 @dataclass
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
329 class Rule:
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
330 lhsGraph: Graph
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
331 rhsGraph: Graph
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
332
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
333 def __post_init__(self):
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
334 self.lhs = Lhs(ChunkedGraph(self.lhsGraph, functionsFor))
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
335 #
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
336 self.rhsBnodeMap = {}
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
337
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
338 def applyRule(self, workingSet: Graph, implied: Graph, stats: Dict, ruleStatementsIterationLimit):
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
339 # this does not change for the current applyRule call. The rule will be
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
340 # tried again in an outer loop, in case it can produce more.
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
341 workingSetChunked = ChunkedGraph(workingSet, functionsFor)
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
342
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
343 for bound in self.lhs.findCandidateBindings(workingSetChunked, stats, ruleStatementsIterationLimit):
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
344 log.debug(f'{INDENT*5} +rule has a working binding: {bound}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
345
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
346 # rhs could have more bnodes, and they just need to be distinct per rule-firing that we do
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
347 existingRhsBnodes = set()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
348 for stmt in self.rhsGraph:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
349 for t in stmt:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
350 if isinstance(t, BNode):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
351 existingRhsBnodes.add(t)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
352 # if existingRhsBnodes:
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
353 # log.debug(f'{INDENT*6} mapping rhs bnodes {existingRhsBnodes} to new ones')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
354
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
355 for b in existingRhsBnodes:
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
356
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
357 key = tuple(sorted(bound.binding.binding.items())), b
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
358 self.rhsBnodeMap.setdefault(key, BNode())
1638
0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet)
drewp@bigasterisk.com
parents: 1637
diff changeset
359 try:
0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet)
drewp@bigasterisk.com
parents: 1637
diff changeset
360 bound.binding.addNewBindings(CandidateBinding({b: self.rhsBnodeMap[key]}))
0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet)
drewp@bigasterisk.com
parents: 1637
diff changeset
361 except BindingConflict:
0ba1625037ae don't crash, just skip the rule if there's a BindingConflict (no clear test case yet)
drewp@bigasterisk.com
parents: 1637
diff changeset
362 continue
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
363
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
364 # for lhsBoundStmt in bound.binding.apply(bound.lhsStmtsWithoutEvals()):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
365 # log.debug(f'{INDENT*6} adding to workingSet {lhsBoundStmt=}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
366 # workingSet.add(lhsBoundStmt)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
367 # log.debug(f'{INDENT*6} rhsGraph is good: {list(self.rhsGraph)}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
368
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
369 for newStmt in bound.binding.apply(self.rhsGraph):
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
370 # log.debug(f'{INDENT*6} adding {newStmt=}')
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
371 workingSet.add(newStmt)
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
372 implied.add(newStmt)
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
373
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
374
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
375 @dataclass
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
376 class Inference:
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
377 rulesIterationLimit = 3
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
378 ruleStatementsIterationLimit = 5000
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
379
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
380 def __init__(self) -> None:
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
381 self.rules: List[Rule] = []
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
382 self._nonRuleStmts: List[Triple] = []
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
383
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
384 def setRules(self, g: ConjunctiveGraph):
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
385 self.rules = []
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
386 self._nonRuleStmts = []
1599
abbf0eb0e640 fix a bug with a slightly moer complicated set of rules
drewp@bigasterisk.com
parents: 1598
diff changeset
387 for stmt in g:
abbf0eb0e640 fix a bug with a slightly moer complicated set of rules
drewp@bigasterisk.com
parents: 1598
diff changeset
388 if stmt[1] == LOG['implies']:
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
389 self.rules.append(Rule(stmt[0], stmt[2]))
1649
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
390 else:
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
391 self._nonRuleStmts.append(stmt)
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
392
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
393 def nonRuleStatements(self) -> List[Triple]:
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
394 return self._nonRuleStmts
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
395
1601
30463df12d89 infer() dumps stats
drewp@bigasterisk.com
parents: 1600
diff changeset
396 @INFER_CALLS.time()
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
397 def infer(self, graph: Graph):
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
398 """
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
399 returns new graph of inferred statements.
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
400 """
1626
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
401 n = graph.__len__()
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
402 INFER_GRAPH_SIZE.observe(n)
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
403 log.info(f'{INDENT*0} Begin inference of graph len={n} with rules len={len(self.rules)}:')
1601
30463df12d89 infer() dumps stats
drewp@bigasterisk.com
parents: 1600
diff changeset
404 startTime = time.time()
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
405 stats: Dict[str, Union[int, float]] = defaultdict(lambda: 0)
1649
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
406
1589
5c1055be3c36 WIP more debugging, working towards bnode-matching support
drewp@bigasterisk.com
parents: 1588
diff changeset
407 # everything that is true: the input graph, plus every rule conclusion we can make
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
408 workingSet = Graph()
1649
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
409 workingSet += self._nonRuleStmts
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
410 workingSet += graph
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
411
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
412 # just the statements that came from RHS's of rules that fired.
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
413 implied = ConjunctiveGraph()
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
414
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
415 rulesIterations = 0
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
416 delta = 1
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
417 stats['initWorkingSet'] = cast(int, workingSet.__len__())
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
418 while delta > 0 and rulesIterations <= self.rulesIterationLimit:
1620
92f8deb59735 log layout
drewp@bigasterisk.com
parents: 1616
diff changeset
419 log.debug('')
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
420 log.info(f'{INDENT*1}*iteration {rulesIterations}')
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
421
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
422 delta = -len(implied)
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
423 self._iterateAllRules(workingSet, implied, stats)
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
424 delta += len(implied)
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
425 rulesIterations += 1
1597
387a9cb66517 logging adjustments
drewp@bigasterisk.com
parents: 1596
diff changeset
426 log.info(f'{INDENT*2} this inference iteration added {delta} more implied stmts')
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
427 stats['iterations'] = rulesIterations
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
428 stats['timeSpent'] = round(time.time() - startTime, 3)
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
429 stats['impliedStmts'] = len(implied)
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
430 log.info(f'{INDENT*0} Inference done {dict(stats)}.')
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
431 log.debug('Implied:')
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
432 log.debug(graphDump(implied))
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
433 return implied
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
434
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
435 def _iterateAllRules(self, workingSet: Graph, implied: Graph, stats):
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
436 for i, rule in enumerate(self.rules):
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
437 self._logRuleApplicationHeader(workingSet, i, rule)
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
438 rule.applyRule(workingSet, implied, stats, self.ruleStatementsIterationLimit)
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
439
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
440 def _logRuleApplicationHeader(self, workingSet, i, r: Rule):
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
441 if not log.isEnabledFor(logging.DEBUG):
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
442 return
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
443
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
444 log.debug('')
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
445 log.debug(f'{INDENT*2} workingSet:')
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
446 # for j, stmt in enumerate(sorted(workingSet)):
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
447 # log.debug(f'{INDENT*3} ({j}) {stmt}')
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
448 log.debug(f'{INDENT*3} {graphDump(workingSet, oneLine=False)}')
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
449
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
450 log.debug('')
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
451 log.debug(f'{INDENT*2}-applying rule {i}')
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
452 log.debug(f'{INDENT*3} rule def lhs:')
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
453 for stmt in sorted(r.lhsGraph, reverse=True):
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
454 log.debug(f'{INDENT*4} {stmt}')
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
455 log.debug(f'{INDENT*3} rule def rhs: {graphDump(r.rhsGraph)}')