Mercurial > code > home > repos > homeauto

annotate service/mqtt_to_rdf/inference.py @ 1694:73abfd4cf5d0

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
new html log and other refactoring as i work on the advanceTheStack problems https://bigasterisk.com/post/inference/2021-09-27_11-11.png
author drewp@bigasterisk.com
date Mon, 27 Sep 2021 11:22:09 -0700
parents 0455a1e18e4f
children 5c2565e63297
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
1 """
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
2 copied from reasoning 2021-08-29. probably same api. should
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
3 be able to lib/ this out
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
4 """
1588
0757fafbfdab WIP inferencer - partial var and function support
drewp@bigasterisk.com
parents: 1587
diff changeset
5 import itertools
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
6 import logging
1601
30463df12d89 infer() dumps stats
drewp@bigasterisk.com
parents: 1600
diff changeset
7 import time
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
8 from collections import defaultdict
1626
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
9 from dataclasses import dataclass
1692
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
10 from typing import Dict, Iterator, List, Optional, Tuple, Union, cast
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
11 from pathlib import Path
1588
0757fafbfdab WIP inferencer - partial var and function support
drewp@bigasterisk.com
parents: 1587
diff changeset
12
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
13 from prometheus_client import Histogram, Summary
1673
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
14 from rdflib import Graph, Namespace
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
15 from rdflib.graph import ConjunctiveGraph
1640
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
16 from rdflib.term import Node, URIRef, Variable
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
17
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
18 from candidate_binding import CandidateBinding
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
19 from inference_types import (BindingUnknown, Inconsistent, RhsBnode, RuleUnboundBnode, Triple, WorkingSetBnode)
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
20 from lhs_evaluation import functionsFor
1650
2061df259224 move graphDump (on its way out, since reprs are getting better)
drewp@bigasterisk.com
parents: 1649
diff changeset
21 from rdf_debug import graphDump
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
22 from stmt_chunk import AlignedRuleChunk, Chunk, ChunkedGraph, applyChunky
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
23 from structured_log import StructuredLog
1605
449746d1598f WIP move evaluation to new file
drewp@bigasterisk.com
parents: 1603
diff changeset
24
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
25 log = logging.getLogger('infer')
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
26 odolog = logging.getLogger('infer.odo') # the "odometer" logic
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
27 ringlog = logging.getLogger('infer.ring') # for ChunkLooper
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
28
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
29 INDENT = ' '
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
30
1626
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
31 INFER_CALLS = Summary('inference_infer_calls', 'calls')
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
32 INFER_GRAPH_SIZE = Histogram('inference_graph_size', 'statements', buckets=[2**x for x in range(2, 20, 2)])
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
33
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
34 ROOM = Namespace("http://projects.bigasterisk.com/room/")
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
35 LOG = Namespace('http://www.w3.org/2000/10/swap/log#')
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
36 MATH = Namespace('http://www.w3.org/2000/10/swap/math#')
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
37
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
38
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
39 class NoOptions(ValueError):
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
40 """ChunkLooper has no possibilites to add to the binding; the whole rule must therefore not apply"""
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
41
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
42
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
43 def debug(logger, slog: Optional[StructuredLog], msg):
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
44 logger.debug(msg)
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
45 if slog:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
46 slog.say(msg)
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
47
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
48
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
49 _chunkLooperShortId = itertools.count()
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
50
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
51
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
52 @dataclass
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
53 class ChunkLooper:
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
54 """given one LHS Chunk, iterate through the possible matches for it,
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
55 returning what bindings they would imply. Only distinct bindings are
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
56 returned. The bindings build on any `prev` ChunkLooper's results.
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
57
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
58 In the odometer metaphor used below, this is one of the rings.
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
59
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
60 This iterator is restartable."""
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
61 lhsChunk: Chunk
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
62 prev: Optional['ChunkLooper']
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
63 workingSet: 'ChunkedGraph'
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
64 slog: Optional[StructuredLog]
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
65
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
66 def __repr__(self):
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
67 return f'{self.__class__.__name__}{self._shortId}{"<pastEnd>" if self.pastEnd() else ""}'
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
68
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
69 def __post_init__(self):
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
70 self._shortId = next(_chunkLooperShortId)
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
71 self._alignedMatches = list(self.lhsChunk.ruleMatchesFrom(self.workingSet))
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
72 del self.workingSet
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
73
1668
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
74 # only ours- do not store prev, since it could change without us
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
75 self._current = CandidateBinding({})
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
76 self.currentSourceChunk: Optional[Chunk] = None # for debugging only
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
77 self._pastEnd = False
1667
a2347393b43e comments, debug, dead code
drewp@bigasterisk.com
parents: 1664
diff changeset
78 self._seenBindings: List[CandidateBinding] = [] # combined bindings (up to our ring) that we've returned
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
79
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
80 if ringlog.isEnabledFor(logging.DEBUG):
1667
a2347393b43e comments, debug, dead code
drewp@bigasterisk.com
parents: 1664
diff changeset
81 ringlog.debug('')
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
82 msg = f'{INDENT*6} introducing {self!r}({self.lhsChunk}, {self._alignedMatches=})'
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
83 msg = msg.replace('AlignedRuleChunk', f'\n{INDENT*12}AlignedRuleChunk')
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
84 ringlog.debug(msg)
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
85
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
86 self.restart()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
87
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
88 def _prevBindings(self) -> CandidateBinding:
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
89 if not self.prev or self.prev.pastEnd():
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
90 return CandidateBinding({})
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
91
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
92 return self.prev.currentBinding()
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
93
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
94 def advance(self):
1668
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
95 """update _current to a new set of valid bindings we haven't seen (since
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
96 last restart), or go into pastEnd mode. Note that _current is just our
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
97 contribution, but returned valid bindings include all prev rings."""
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
98 if self._pastEnd:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
99 raise NotImplementedError('need restart')
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
100 ringlog.debug('')
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
101 debug(ringlog, self.slog, f'{INDENT*6} --> {self}.advance start:')
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
102
1693
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
103 self._currentIsFromFunc = None
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
104 augmentedWorkingSet: List[AlignedRuleChunk] = []
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
105 if self.prev is None:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
106 augmentedWorkingSet = self._alignedMatches
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
107 else:
1673
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
108 augmentedWorkingSet = list(applyChunky(self.prev.currentBinding(), self._alignedMatches))
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
109
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
110 if self._advanceWithPlainMatches(augmentedWorkingSet):
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
111 debug(ringlog, self.slog, f'{INDENT*6} <-- {self}.advance finished with plain matches')
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
112 return
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
113
1639
ae5ca4ba8954 rm unused computation
drewp@bigasterisk.com
parents: 1638
diff changeset
114 if self._advanceWithFunctions():
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
115 debug(ringlog, self.slog, f'{INDENT*6} <-- {self}.advance finished with function matches')
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
116 return
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
117
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
118 debug(ringlog, self.slog, f'{INDENT*6} <-- {self}.advance had nothing and is now past end')
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
119 self._pastEnd = True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
120
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
121 def _advanceWithPlainMatches(self, augmentedWorkingSet: List[AlignedRuleChunk]) -> bool:
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
122 # if augmentedWorkingSet:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
123 # debug(ringlog, self.slog, f'{INDENT*7} {self} mines {len(augmentedWorkingSet)} matching augmented statements')
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
124 # for s in augmentedWorkingSet:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
125 # debug(ringlog, self.slog, f'{INDENT*8} {s}')
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
126
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
127 for aligned in augmentedWorkingSet:
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
128 try:
1668
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
129 newBinding = aligned.newBindingIfMatched(self._prevBindings())
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
130 except Inconsistent as exc:
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
131 debug(ringlog, self.slog,
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
132 f'{INDENT*7} ChunkLooper{self._shortId} - {aligned} would be inconsistent with prev bindings ({exc})')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
133 continue
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
134
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
135 if self._testAndKeepNewBinding(newBinding, aligned.workingSetChunk):
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
136 return True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
137 return False
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
138
1639
ae5ca4ba8954 rm unused computation
drewp@bigasterisk.com
parents: 1638
diff changeset
139 def _advanceWithFunctions(self) -> bool:
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
140 pred: Node = self.lhsChunk.predicate
1640
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
141 if not isinstance(pred, URIRef):
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
142 raise NotImplementedError
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
143
1637
ec3f98d0c1d8 refactor rules eval
drewp@bigasterisk.com
parents: 1636
diff changeset
144 for functionType in functionsFor(pred):
1661
00a5624d1d14 cleanups and optimizations
drewp@bigasterisk.com
parents: 1656
diff changeset
145 fn = functionType(self.lhsChunk)
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
146 # debug(ringlog, self.slog, f'{INDENT*7} ChunkLooper{self._shortId} advanceWithFunctions, {functionType=}')
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
147
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
148 try:
1692
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
149 log.debug(f'fn.bind {self._prevBindings()} ...')
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
150 #fullBinding = self._prevBindings().copy()
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
151 newBinding = fn.bind(self._prevBindings())
1692
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
152 log.debug(f'...makes {newBinding=}')
1637
ec3f98d0c1d8 refactor rules eval
drewp@bigasterisk.com
parents: 1636
diff changeset
153 except BindingUnknown:
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
154 pass
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
155 else:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
156 if newBinding is not None:
1693
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
157 self._currentIsFromFunc = fn
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
158 if self._testAndKeepNewBinding(newBinding, self.lhsChunk):
1637
ec3f98d0c1d8 refactor rules eval
drewp@bigasterisk.com
parents: 1636
diff changeset
159 return True
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
160
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
161 return False
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
162
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
163 def _testAndKeepNewBinding(self, newBinding: CandidateBinding, sourceChunk: Chunk):
1668
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
164 fullBinding: CandidateBinding = self._prevBindings().copy()
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
165 fullBinding.addNewBindings(newBinding)
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
166 isNew = fullBinding not in self._seenBindings
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
167 ringlog.debug(f'{INDENT*7} {self} considering {newBinding=} to make {fullBinding}. {isNew=}')
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
168 # if self.slog:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
169 # self.slog.looperConsider(self, newBinding, fullBinding, isNew)
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
170
1668
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
171 if isNew:
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
172 self._seenBindings.append(fullBinding.copy())
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
173 self._current = newBinding
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
174 self.currentSourceChunk = sourceChunk
1668
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
175 return True
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
176 return False
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
177
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
178 def localBinding(self) -> CandidateBinding:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
179 if self.pastEnd():
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
180 raise NotImplementedError()
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
181 return self._current
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
182
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
183 def currentBinding(self) -> CandidateBinding:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
184 if self.pastEnd():
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
185 raise NotImplementedError()
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
186 together = self._prevBindings().copy()
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
187 together.addNewBindings(self._current)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
188 return together
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
189
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
190 def pastEnd(self) -> bool:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
191 return self._pastEnd
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
192
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
193 def restart(self):
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
194 try:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
195 self._pastEnd = False
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
196 self._seenBindings = []
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
197 self.advance()
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
198 if self.pastEnd():
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
199 raise NoOptions()
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
200 finally:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
201 debug(ringlog, self.slog, f'{INDENT*7} ChunkLooper{self._shortId} restarts: pastEnd={self.pastEnd()}')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
202
1693
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
203 def prevMayHaveChanged(self):
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
204 pass
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
205 # self._advanceWithFunctions()
1693
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
206 # This is a total patch for a test failure. This should be generalized
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
207 # to a Looper that can keep itself correct when prev changes.
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
208 # if self.pastEnd():
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
209 # self.restart()
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
210 # else:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
211 # self.advance()
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
212 # if self._currentIsFromFunc:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
213 # self._advanceWithFunctions()
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
214 # if self.pastEnd():
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
215 # self.restart()
1693
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
216
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
217
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
218 @dataclass
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
219 class Lhs:
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
220 graph: ChunkedGraph # our full LHS graph, as input. See below for the statements partitioned into groups.
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
221
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
222 def __post_init__(self):
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
223
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
224 self.myPreds = self.graph.allPredicatesExceptFunctions()
1602
e3c44ac6d3c5 do findEvals once at setRules time
drewp@bigasterisk.com
parents: 1601
diff changeset
225
1609
34f2817320cc new tests for a smaller part of the inner loop
drewp@bigasterisk.com
parents: 1608
diff changeset
226 def __repr__(self):
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
227 return f"Lhs({self.graph!r})"
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
228 def findCandidateBindings(self, knownTrue: ChunkedGraph, stats, slog: Optional[StructuredLog],
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
229 ruleStatementsIterationLimit) -> Iterator['BoundLhs']:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
230 """distinct bindings that fit the LHS of a rule, using statements from
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
231 workingSet and functions from LHS"""
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
232 if not self.graph:
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
233 # special case- no LHS!
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
234 yield BoundLhs(self, CandidateBinding({}))
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
235 return
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
236
1640
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
237 if self._checkPredicateCounts(knownTrue):
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
238 stats['_checkPredicateCountsCulls'] += 1
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
239 return
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
240
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
241 if not all(ch in knownTrue for ch in self.graph.staticChunks):
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
242 stats['staticStmtCulls'] += 1
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
243 return
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
244 # After this point we don't need to consider self.graph.staticChunks.
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
245
1656
d8e5b2232474 bugfix for simple (no arg lists) functions not evaluating
drewp@bigasterisk.com
parents: 1653
diff changeset
246 if not self.graph.patternChunks and not self.graph.chunksUsedByFuncs:
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
247 # static only
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
248 yield BoundLhs(self, CandidateBinding({}))
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
249 return
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
250
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
251 log.debug('')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
252 try:
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
253 chunkStack = self._assembleRings(knownTrue, stats, slog)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
254 except NoOptions:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
255 ringlog.debug(f'{INDENT*5} start up with no options; 0 bindings')
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
256 return
1692
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
257 log.debug('')
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
258 log.debug('')
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
259 self._debugChunkStack('time to spin: initial odometer is', chunkStack)
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
260
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
261 if slog:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
262 slog.say('time to spin')
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
263 slog.odometer(chunkStack)
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
264 self._assertAllRingsAreValid(chunkStack)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
265
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
266 lastRing = chunkStack[-1]
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
267 iterCount = 0
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
268 while True:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
269 iterCount += 1
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
270 if iterCount > ruleStatementsIterationLimit:
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
271 raise ValueError('rule too complex')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
272
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
273 log.debug(f'{INDENT*4} vv findCandBindings iteration {iterCount}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
274
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
275 yield BoundLhs(self, lastRing.currentBinding())
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
276
1667
a2347393b43e comments, debug, dead code
drewp@bigasterisk.com
parents: 1664
diff changeset
277 # self._debugChunkStack('odometer', chunkStack)
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
278
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
279 done = self._advanceTheStack(chunkStack)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
280
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
281 self._debugChunkStack(f'odometer after ({done=})', chunkStack)
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
282 if slog:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
283 slog.odometer(chunkStack)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
284
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
285 log.debug(f'{INDENT*4} ^^ findCandBindings iteration done')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
286 if done:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
287 break
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
288
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
289 def _debugChunkStack(self, label: str, chunkStack: List[ChunkLooper]):
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
290 odolog.debug(f'{INDENT*4} {label}:')
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
291 for i, l in enumerate(chunkStack):
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
292 odolog.debug(f'{INDENT*5} [{i}] {l} curbind={l.localBinding() if not l.pastEnd() else "<end>"}')
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
293
1640
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
294 def _checkPredicateCounts(self, knownTrue):
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
295 """raise NoOptions quickly in some cases"""
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
296
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
297 if self.graph.noPredicatesAppear(self.myPreds):
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
298 log.debug(f'{INDENT*3} checkPredicateCounts does cull because not all {self.myPreds=} are in knownTrue')
1640
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
299 return True
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
300 log.debug(f'{INDENT*3} checkPredicateCounts does not cull because all {self.myPreds=} are in knownTrue')
1640
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
301 return False
4bb6f593ebf3 speedups: abort some rules faster
drewp@bigasterisk.com
parents: 1639
diff changeset
302
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
303 def _assembleRings(self, knownTrue: ChunkedGraph, stats, slog) -> List[ChunkLooper]:
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
304 """make ChunkLooper for each stmt in our LHS graph, but do it in a way that they all
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
305 start out valid (or else raise NoOptions). static chunks have already been confirmed."""
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
306
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
307 log.debug(f'{INDENT*4} stats={dict(stats)}')
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
308 odolog.debug(f'{INDENT*3} build new ChunkLooper stack')
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
309 chunks = list(self.graph.patternChunks.union(self.graph.chunksUsedByFuncs))
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
310 chunks.sort(key=None)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
311 odolog.info(f' {INDENT*3} taking permutations of {len(chunks)=}')
1692
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
312
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
313 permsTried = 0
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
314
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
315 for perm in self._partitionedGraphPermutations():
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
316 looperRings: List[ChunkLooper] = []
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
317 prev: Optional[ChunkLooper] = None
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
318 if odolog.isEnabledFor(logging.DEBUG):
1692
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
319 odolog.debug(
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
320 f'{INDENT*4} [perm {permsTried}] try rule chunks in this order: {" THEN ".join(repr(p) for p in perm)}')
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
321
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
322 for ruleChunk in perm:
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
323 try:
1661
00a5624d1d14 cleanups and optimizations
drewp@bigasterisk.com
parents: 1656
diff changeset
324 # These are getting rebuilt a lot which takes time. It would
00a5624d1d14 cleanups and optimizations
drewp@bigasterisk.com
parents: 1656
diff changeset
325 # be nice if they could accept a changing `prev` order
00a5624d1d14 cleanups and optimizations
drewp@bigasterisk.com
parents: 1656
diff changeset
326 # (which might already be ok).
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
327 looper = ChunkLooper(ruleChunk, prev, knownTrue, slog)
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
328 except NoOptions:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
329 odolog.debug(f'{INDENT*5} permutation didnt work, try another')
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
330 break
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
331 looperRings.append(looper)
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
332 prev = looperRings[-1]
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
333 else:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
334 # bug: At this point we've only shown that these are valid
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
335 # starting rings. The rules might be tricky enough that this
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
336 # permutation won't get us to the solution.
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
337 return looperRings
1692
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
338 if permsTried > 50000:
1652
dddfa09ea0b9 debug logging and comments
drewp@bigasterisk.com
parents: 1651
diff changeset
339 raise NotImplementedError(f'trying too many permutations {len(chunks)=}')
1692
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
340 permsTried += 1
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
341
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
342 odolog.debug(f'{INDENT*5} no perms worked- rule cannot match anything')
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
343 raise NoOptions()
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
344
1692
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
345 def _unpartitionedGraphPermutations(self) -> Iterator[Tuple[Chunk, ...]]:
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
346 for perm in itertools.permutations(sorted(list(self.graph.patternChunks.union(self.graph.chunksUsedByFuncs)))):
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
347 yield perm
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
348
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
349 def _partitionedGraphPermutations(self) -> Iterator[Tuple[Chunk, ...]]:
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
350 """always puts function chunks after pattern chunks
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
351
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
352 (and, if we cared, static chunks could go before that. Currently they're
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
353 culled out elsewhere, but that's done as a special case)
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
354 """
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
355 tupleOfNoChunks: Tuple[Chunk, ...] = ()
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
356 pats = sorted(self.graph.patternChunks)
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
357 funcs = sorted(self.graph.chunksUsedByFuncs)
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
358 for patternPart in itertools.permutations(pats) if pats else [tupleOfNoChunks]:
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
359 for funcPart in itertools.permutations(funcs) if funcs else [tupleOfNoChunks]:
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
360 perm = patternPart + funcPart
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
361 yield perm
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
362
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
363 def _advanceTheStack(self, looperRings: List[ChunkLooper]) -> bool:
1693
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
364
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
365 def freshenRight(i):
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
366 for ring in looperRings[i + 1:]:
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
367 ring.prevMayHaveChanged()
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
368
1668
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
369 carry = True # last elem always must advance
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
370 for i, ring in reversed(list(enumerate(looperRings))):
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
371 # unlike normal odometer, advancing any earlier ring could invalidate later ones
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
372 if carry:
1692
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
373 odolog.debug(f'{INDENT*4} advanceAll [ring={i}] {ring} carry/advance')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
374 ring.advance()
1693
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
375 freshenRight(i)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
376 carry = False
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
377 if ring.pastEnd():
1668
89e53cb8a01c fix some harder tests. Mostly, _advanceTheStack needed to spin the odometer rings starting from the other side, to get all the right combos
drewp@bigasterisk.com
parents: 1667
diff changeset
378 if ring is looperRings[0]:
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
379 allRingsDone = [r.pastEnd() for r in looperRings]
1692
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
380 odolog.debug(f'{INDENT*5} advanceAll [ring={i}] {ring} says we done {allRingsDone=}')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
381 return True
1692
2883da14847c debugging and cleanup, as i looked for a bug
drewp@bigasterisk.com
parents: 1677
diff changeset
382 odolog.debug(f'{INDENT*5} advanceAll [ring={i}] {ring} restart')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
383 ring.restart()
1693
0455a1e18e4f really rough fix to a broken test. hopefully this gets redone
drewp@bigasterisk.com
parents: 1692
diff changeset
384 freshenRight(i)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
385 carry = True
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
386 return False
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
387
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
388 def _assertAllRingsAreValid(self, looperRings):
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
389 if any(ring.pastEnd() for ring in looperRings): # this is an unexpected debug assertion
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
390 odolog.warning(f'{INDENT*4} some rings started at pastEnd {looperRings}')
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
391 raise NoOptions()
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
392
1592
d7b66234064b pure reordering of funcs to make the next diffs smaller
drewp@bigasterisk.com
parents: 1591
diff changeset
393
1622
38bd8ef9ef67 add CandidateTermMatches, unused so far
drewp@bigasterisk.com
parents: 1621
diff changeset
394 @dataclass
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
395 class BoundLhs:
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
396 lhs: Lhs
1610
6fc48ef4c696 mysteriously lost an important line
drewp@bigasterisk.com
parents: 1609
diff changeset
397 binding: CandidateBinding
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
398
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
399
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
400 @dataclass
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
401 class Rule:
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
402 lhsGraph: Graph
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
403 rhsGraph: Graph
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
404
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
405 def __post_init__(self):
1673
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
406 self.lhs = Lhs(ChunkedGraph(self.lhsGraph, RuleUnboundBnode, functionsFor))
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
407
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
408 self.maps = {}
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
409
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
410 self.rhsGraphConvert: List[Triple] = []
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
411 for s, p, o in self.rhsGraph:
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
412 from rdflib import BNode
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
413 if isinstance(s, BNode):
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
414 s = RhsBnode(s)
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
415 if isinstance(p, BNode):
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
416 p = RhsBnode(p)
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
417 if isinstance(o, BNode):
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
418 o = RhsBnode(o)
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
419 self.rhsGraphConvert.append((s, p, o))
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
420
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
421 def applyRule(self, workingSet: Graph, implied: Graph, stats: Dict, slog: Optional[StructuredLog],
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
422 ruleStatementsIterationLimit):
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
423 # this does not change for the current applyRule call. The rule will be
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
424 # tried again in an outer loop, in case it can produce more.
1673
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
425 workingSetChunked = ChunkedGraph(workingSet, WorkingSetBnode, functionsFor)
1651
20474ad4968e WIP - functions are broken as i move most layers to work in Chunks not Triples
drewp@bigasterisk.com
parents: 1650
diff changeset
426
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
427 for bound in self.lhs.findCandidateBindings(workingSetChunked, stats, slog, ruleStatementsIterationLimit):
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
428 if slog:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
429 slog.foundBinding(bound)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
430 log.debug(f'{INDENT*5} +rule has a working binding: {bound}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
431
1673
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
432 newStmts = self.generateImpliedFromRhs(bound.binding)
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
433
1673
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
434 for newStmt in newStmts:
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
435 # log.debug(f'{INDENT*6} adding {newStmt=}')
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
436 workingSet.add(newStmt)
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
437 implied.add(newStmt)
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
438
1673
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
439 def generateImpliedFromRhs(self, binding: CandidateBinding) -> List[Triple]:
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
440
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
441 out: List[Triple] = []
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
442
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
443 # Each time the RHS is used (in a rule firing), its own BNodes (which
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
444 # are subtype RhsBnode) need to be turned into distinct ones. Note that
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
445 # bnodes that come from the working set should not be remapped.
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
446 rhsBnodeMap: Dict[RhsBnode, WorkingSetBnode] = {}
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
447
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
448 # but, the iteration loop could come back with the same bindings again
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
449 key = binding.key()
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
450 rhsBnodeMap = self.maps.setdefault(key, {})
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
451
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
452 for stmt in binding.apply(self.rhsGraphConvert):
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
453
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
454 outStmt: List[Node] = []
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
455
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
456 for t in stmt:
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
457 if isinstance(t, RhsBnode):
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
458 if t not in rhsBnodeMap:
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
459 rhsBnodeMap[t] = WorkingSetBnode()
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
460 t = rhsBnodeMap[t]
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
461
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
462 outStmt.append(t)
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
463
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
464 log.debug(f'{INDENT*6} rhs stmt {stmt} became {outStmt}')
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
465 out.append((outStmt[0], outStmt[1], outStmt[2]))
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
466
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
467 return out
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
468
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
469
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
470 @dataclass
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
471 class Inference:
1673
80f4e741ca4f redo RHS bnode processing
drewp@bigasterisk.com
parents: 1669
diff changeset
472 rulesIterationLimit = 4
1664
1a7c1261302c logic fix- some bindings were being returned 2+; some 0 times
drewp@bigasterisk.com
parents: 1661
diff changeset
473 ruleStatementsIterationLimit = 5000
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
474
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
475 def __init__(self) -> None:
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
476 self.rules: List[Rule] = []
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
477 self._nonRuleStmts: List[Triple] = []
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
478
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
479 def setRules(self, g: ConjunctiveGraph):
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
480 self.rules = []
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
481 self._nonRuleStmts = []
1599
abbf0eb0e640 fix a bug with a slightly moer complicated set of rules
drewp@bigasterisk.com
parents: 1598
diff changeset
482 for stmt in g:
abbf0eb0e640 fix a bug with a slightly moer complicated set of rules
drewp@bigasterisk.com
parents: 1598
diff changeset
483 if stmt[1] == LOG['implies']:
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
484 self.rules.append(Rule(stmt[0], stmt[2]))
1649
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
485 else:
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
486 self._nonRuleStmts.append(stmt)
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
487
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
488 def nonRuleStatements(self) -> List[Triple]:
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
489 return self._nonRuleStmts
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
490
1601
30463df12d89 infer() dumps stats
drewp@bigasterisk.com
parents: 1600
diff changeset
491 @INFER_CALLS.time()
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
492 def infer(self, graph: Graph, htmlLog: Optional[Path] = None):
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
493 """
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
494 returns new graph of inferred statements.
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
495 """
1626
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
496 n = graph.__len__()
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
497 INFER_GRAPH_SIZE.observe(n)
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
498 log.info(f'{INDENT*0} Begin inference of graph len={n} with rules len={len(self.rules)}:')
1601
30463df12d89 infer() dumps stats
drewp@bigasterisk.com
parents: 1600
diff changeset
499 startTime = time.time()
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
500 stats: Dict[str, Union[int, float]] = defaultdict(lambda: 0)
1649
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
501
1589
5c1055be3c36 WIP more debugging, working towards bnode-matching support
drewp@bigasterisk.com
parents: 1588
diff changeset
502 # everything that is true: the input graph, plus every rule conclusion we can make
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
503 workingSet = Graph()
1649
bb5d2b5370ac add nonRuleStatments to Inference api. there's already a test in an eariler commit
drewp@bigasterisk.com
parents: 1648
diff changeset
504 workingSet += self._nonRuleStmts
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
505 workingSet += graph
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
506
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
507 # just the statements that came from RHS's of rules that fired.
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
508 implied = ConjunctiveGraph()
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
509
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
510 slog = StructuredLog(htmlLog) if htmlLog else None
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
511
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
512 rulesIterations = 0
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
513 delta = 1
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
514 stats['initWorkingSet'] = cast(int, workingSet.__len__())
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
515 if slog:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
516 slog.workingSet = workingSet
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
517
1674
4a15b4cd4600 don't jsut quietly succeed if iteration limit is reached
drewp@bigasterisk.com
parents: 1673
diff changeset
518 while delta > 0:
1620
92f8deb59735 log layout
drewp@bigasterisk.com
parents: 1616
diff changeset
519 log.debug('')
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
520 log.info(f'{INDENT*1}*iteration {rulesIterations}')
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
521 if slog:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
522 slog.startIteration(rulesIterations)
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
523
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
524 delta = -len(implied)
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
525 self._iterateAllRules(workingSet, implied, stats, slog)
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
526 delta += len(implied)
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
527 rulesIterations += 1
1597
387a9cb66517 logging adjustments
drewp@bigasterisk.com
parents: 1596
diff changeset
528 log.info(f'{INDENT*2} this inference iteration added {delta} more implied stmts')
1674
4a15b4cd4600 don't jsut quietly succeed if iteration limit is reached
drewp@bigasterisk.com
parents: 1673
diff changeset
529 if rulesIterations >= self.rulesIterationLimit:
4a15b4cd4600 don't jsut quietly succeed if iteration limit is reached
drewp@bigasterisk.com
parents: 1673
diff changeset
530 raise ValueError(f"rule too complex after {rulesIterations=}")
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
531 stats['iterations'] = rulesIterations
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
532 stats['timeSpent'] = round(time.time() - startTime, 3)
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
533 stats['impliedStmts'] = len(implied)
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
534 log.info(f'{INDENT*0} Inference done {dict(stats)}.')
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
535 log.debug('Implied:')
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
536 log.debug(graphDump(implied))
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
537
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
538 if slog:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
539 slog.render()
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
540 log.info(f'wrote {htmlLog}')
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
541
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
542 return implied
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
543
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
544 def _iterateAllRules(self, workingSet: Graph, implied: Graph, stats, slog: Optional[StructuredLog]):
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
545 for i, rule in enumerate(self.rules):
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
546 self._logRuleApplicationHeader(workingSet, i, rule)
1694
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
547 if slog:
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
548 slog.rule(workingSet, i, rule)
73abfd4cf5d0 new html log and other refactoring as i work on the advanceTheStack problems
drewp@bigasterisk.com
parents: 1693
diff changeset
549 rule.applyRule(workingSet, implied, stats, slog, self.ruleStatementsIterationLimit)
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
550
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
551 def _logRuleApplicationHeader(self, workingSet, i, r: Rule):
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
552 if not log.isEnabledFor(logging.DEBUG):
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
553 return
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
554
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
555 log.debug('')
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
556 log.debug(f'{INDENT*2} workingSet:')
1648
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
557 # for j, stmt in enumerate(sorted(workingSet)):
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
558 # log.debug(f'{INDENT*3} ({j}) {stmt}')
3059f31b2dfa more performance work
drewp@bigasterisk.com
parents: 1641
diff changeset
559 log.debug(f'{INDENT*3} {graphDump(workingSet, oneLine=False)}')
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
560
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
561 log.debug('')
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
562 log.debug(f'{INDENT*2}-applying rule {i}')
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
563 log.debug(f'{INDENT*3} rule def lhs:')
1677
aa35ae7a1acc add new bug test (no fix yet)
drewp@bigasterisk.com
parents: 1674
diff changeset
564 for stmt in sorted(r.lhs.graph.allChunks()):
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
565 log.debug(f'{INDENT*4} {stmt}')
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
566 log.debug(f'{INDENT*3} rule def rhs: {graphDump(r.rhsGraph)}')