Mercurial > code > home > repos > homeauto

annotate service/mqtt_to_rdf/inference.py @ 1636:3252bdc284bc

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
rm dead code from previous tries
author drewp@bigasterisk.com
date Mon, 13 Sep 2021 00:18:47 -0700
parents 22d481f0a924
children ec3f98d0c1d8
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
1 """
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
2 copied from reasoning 2021-08-29. probably same api. should
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
3 be able to lib/ this out
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
4 """
1588
0757fafbfdab WIP inferencer - partial var and function support
drewp@bigasterisk.com
parents: 1587
diff changeset
5 import itertools
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
6 import logging
1601
30463df12d89 infer() dumps stats
drewp@bigasterisk.com
parents: 1600
diff changeset
7 import time
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
8 from collections import defaultdict
1626
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
9 from dataclasses import dataclass
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
10 from typing import (Dict, Iterator, List, Optional, Sequence, Set, Tuple, Union, cast)
1588
0757fafbfdab WIP inferencer - partial var and function support
drewp@bigasterisk.com
parents: 1587
diff changeset
11
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
12 from prometheus_client import Histogram, Summary
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
13 from rdflib import RDF, BNode, Graph, Namespace
1589
5c1055be3c36 WIP more debugging, working towards bnode-matching support
drewp@bigasterisk.com
parents: 1588
diff changeset
14 from rdflib.graph import ConjunctiveGraph, ReadOnlyGraphAggregate
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
15 from rdflib.term import Literal, Node, Variable
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
16
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
17 from candidate_binding import CandidateBinding
1636
3252bdc284bc rm dead code from previous tries
drewp@bigasterisk.com
parents: 1635
diff changeset
18 from inference_types import (BindableTerm, BindingUnknown, ReadOnlyWorkingSet, Triple)
3252bdc284bc rm dead code from previous tries
drewp@bigasterisk.com
parents: 1635
diff changeset
19 from lhs_evaluation import Decimal, numericNode, parseList
1605
449746d1598f WIP move evaluation to new file
drewp@bigasterisk.com
parents: 1603
diff changeset
20
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
21 log = logging.getLogger('infer')
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
22 INDENT = ' '
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
23
1626
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
24 INFER_CALLS = Summary('inference_infer_calls', 'calls')
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
25 INFER_GRAPH_SIZE = Histogram('inference_graph_size', 'statements', buckets=[2**x for x in range(2, 20, 2)])
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
26
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
27 ROOM = Namespace("http://projects.bigasterisk.com/room/")
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
28 LOG = Namespace('http://www.w3.org/2000/10/swap/log#')
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
29 MATH = Namespace('http://www.w3.org/2000/10/swap/math#')
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
30
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
31
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
32 def stmtTemplate(stmt: Triple) -> Tuple[Optional[Node], Optional[Node], Optional[Node]]:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
33 return (
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
34 None if isinstance(stmt[0], (Variable, BNode)) else stmt[0],
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
35 None if isinstance(stmt[1], (Variable, BNode)) else stmt[1],
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
36 None if isinstance(stmt[2], (Variable, BNode)) else stmt[2],
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
37 )
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
38
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
39
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
40 class NoOptions(ValueError):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
41 """stmtlooper has no possibilites to add to the binding; the whole rule must therefore not apply"""
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
42
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
43
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
44 class Inconsistent(ValueError):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
45 """adding this stmt would be inconsistent with an existing binding"""
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
46
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
47
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
48 _stmtLooperShortId = itertools.count()
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
49
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
50
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
51 @dataclass
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
52 class StmtLooper:
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
53 """given one LHS stmt, iterate through the possible matches for it,
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
54 returning what bindings they would imply. Only distinct bindings are
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
55 returned. The bindings build on any `prev` StmtLooper's results.
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
56
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
57 This iterator is restartable."""
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
58 lhsStmt: Triple
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
59 prev: Optional['StmtLooper']
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
60 workingSet: ReadOnlyWorkingSet
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
61 parent: 'Lhs' # just for lhs.graph, really
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
62
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
63 def __repr__(self):
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
64 return f'StmtLooper{self._shortId}({graphDump([self.lhsStmt])} {"<pastEnd>" if self.pastEnd() else ""})'
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
65
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
66 def __post_init__(self):
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
67 self._shortId = next(_stmtLooperShortId)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
68 self._myWorkingSetMatches = self._myMatches(self.workingSet)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
69
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
70 self._current = CandidateBinding({})
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
71 self._pastEnd = False
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
72 self._seenBindings: List[CandidateBinding] = []
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
73 self.restart()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
74
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
75 def _myMatches(self, g: Graph) -> List[Triple]:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
76 template = stmtTemplate(self.lhsStmt)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
77
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
78 stmts = sorted(cast(Iterator[Triple], list(g.triples(template))))
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
79 # plus new lhs possibilties...
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
80 # log.debug(f'{INDENT*6} {self} find {len(stmts)=} in {len(self.workingSet)=}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
81
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
82 return stmts
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
83
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
84 def _prevBindings(self) -> CandidateBinding:
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
85 if not self.prev or self.prev.pastEnd():
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
86 return CandidateBinding({})
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
87
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
88 return self.prev.currentBinding()
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
89
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
90 def advance(self):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
91 """update to a new set of bindings we haven't seen (since last restart), or go into pastEnd mode"""
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
92 if self._pastEnd:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
93 raise NotImplementedError('need restart')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
94 log.debug('')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
95 augmentedWorkingSet: Sequence[Triple] = []
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
96 if self.prev is None:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
97 augmentedWorkingSet = self._myWorkingSetMatches
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
98 else:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
99 augmentedWorkingSet = list(self.prev.currentBinding().apply(self._myWorkingSetMatches,
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
100 returnBoundStatementsOnly=False))
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
101
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
102 log.debug(f'{INDENT*6} {self}.advance has {augmentedWorkingSet=}')
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
103
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
104 if self._advanceWithPlainMatches(augmentedWorkingSet):
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
105 return
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
106
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
107 if self._advanceWithBoolRules():
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
108 return
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
109
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
110 curBind = self.prev.currentBinding() if self.prev else CandidateBinding({})
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
111 [lhsStmtBound] = curBind.apply([self.lhsStmt], returnBoundStatementsOnly=False)
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
112
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
113 fullWorkingSet = self.workingSet + self.parent.graph
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
114 boundFullWorkingSet = list(curBind.apply(fullWorkingSet, returnBoundStatementsOnly=False))
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
115 log.debug(f'{fullWorkingSet.__len__()=} {len(boundFullWorkingSet)=}')
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
116
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
117 if self._advanceWithFunctions(augmentedWorkingSet, boundFullWorkingSet, lhsStmtBound):
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
118 return
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
119
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
120 log.debug(f'{INDENT*6} {self} is past end')
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
121 self._pastEnd = True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
122
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
123 def _advanceWithPlainMatches(self, augmentedWorkingSet: Sequence[Triple]) -> bool:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
124 log.debug(f'{INDENT*7} {self} mines {len(augmentedWorkingSet)} matching augmented statements')
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
125 for s in augmentedWorkingSet:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
126 log.debug(f'{INDENT*7} {s}')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
127
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
128 for i, stmt in enumerate(augmentedWorkingSet):
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
129 try:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
130 outBinding = self._totalBindingIfThisStmtWereTrue(stmt)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
131 except Inconsistent:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
132 log.debug(f'{INDENT*7} {self} - {stmt} would be inconsistent with prev bindings')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
133 continue
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
134
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
135 log.debug(f'{INDENT*7} {outBinding=} {self._seenBindings=}')
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
136 if outBinding not in self._seenBindings:
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
137 self._seenBindings.append(outBinding.copy())
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
138 self._current = outBinding
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
139 log.debug(f'{INDENT*7} new binding from {self} -> {outBinding}')
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
140 return True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
141 return False
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
142
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
143 def _advanceWithBoolRules(self) -> bool:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
144 log.debug(f'{INDENT*7} {self} mines bool rules')
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
145 if self.lhsStmt[1] == MATH['greaterThan']:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
146 operands = [self.lhsStmt[0], self.lhsStmt[2]]
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
147 try:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
148 boundOperands = self._boundOperands(operands)
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
149 except BindingUnknown:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
150 return False
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
151 if numericNode(boundOperands[0]) > numericNode(boundOperands[1]):
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
152 binding: CandidateBinding = self._prevBindings().copy() # no new values; just allow matching to keep going
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
153 if binding not in self._seenBindings:
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
154 self._seenBindings.append(binding)
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
155 self._current = binding
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
156 log.debug(f'{INDENT*7} new binding from {self} -> {binding}')
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
157 return True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
158 return False
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
159
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
160 def _advanceWithFunctions(self, augmentedWorkingSet: Sequence[Triple], boundFullWorkingSet, lhsStmtBound) -> bool:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
161 log.debug(f'{INDENT*7} {self} mines rules')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
162
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
163 if self.lhsStmt[1] == ROOM['asFarenheit']:
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
164 pb: CandidateBinding = self._prevBindings()
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
165 log.debug(f'{INDENT*7} {self} consider ?x faren ?y where ?x={self.lhsStmt[0]} and {pb=}')
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
166
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
167 if isinstance(self.lhsStmt[0], (Variable, BNode)) and pb.contains(self.lhsStmt[0]):
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
168 operands = [pb.applyTerm(self.lhsStmt[0])]
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
169 f = cast(Literal, Literal(Decimal(numericNode(operands[0])) * 9 / 5 + 32))
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
170 objVar = self.lhsStmt[2]
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
171 if not isinstance(objVar, Variable):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
172 raise TypeError(f'expected Variable, got {objVar!r}')
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
173 newBindings = CandidateBinding({cast(BindableTerm, objVar): cast(Node, f)})
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
174 self._current.addNewBindings(newBindings)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
175 if newBindings not in self._seenBindings:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
176 self._seenBindings.append(newBindings)
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
177 self._current = newBindings
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
178 return True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
179 elif self.lhsStmt[1] == MATH['sum']:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
180
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
181 g = Graph()
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
182 for s in boundFullWorkingSet:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
183 g.add(s)
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
184 log.debug(f' boundWorkingSet graph: {s}')
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
185 log.debug(f'_parseList subj = {lhsStmtBound[0]}')
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
186 operands, _ = parseList(g, lhsStmtBound[0])
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
187 log.debug(f'********* {INDENT*7} {self} found list {operands=}')
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
188 try:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
189 obj = Literal(sum(map(numericNode, operands)))
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
190 except TypeError:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
191 log.debug('typeerr in operands')
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
192 pass
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
193 else:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
194 objVar = lhsStmtBound[2]
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
195 log.debug(f'{objVar=}')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
196
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
197 if not isinstance(objVar, Variable):
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
198 raise TypeError(f'expected Variable, got {objVar!r}')
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
199 newBindings = CandidateBinding({objVar: obj})
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
200 log.debug(f'{newBindings=}')
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
201
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
202 self._current.addNewBindings(newBindings)
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
203 log.debug(f'{self._seenBindings=}')
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
204 if newBindings not in self._seenBindings:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
205 self._seenBindings.append(newBindings)
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
206 self._current = newBindings
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
207 return True
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
208
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
209 return False
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
210
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
211 def _boundOperands(self, operands) -> List[Node]:
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
212 pb: CandidateBinding = self._prevBindings()
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
213
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
214 boundOperands: List[Node] = []
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
215 for op in operands:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
216 if isinstance(op, (Variable, BNode)):
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
217 boundOperands.append(pb.applyTerm(op))
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
218 else:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
219 boundOperands.append(op)
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
220 return boundOperands
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
221
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
222 def _totalBindingIfThisStmtWereTrue(self, newStmt: Triple) -> CandidateBinding:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
223 outBinding = self._prevBindings().copy()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
224 for rt, ct in zip(self.lhsStmt, newStmt):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
225 if isinstance(rt, (Variable, BNode)):
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
226 if outBinding.contains(rt) and outBinding.applyTerm(rt) != ct:
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
227 raise Inconsistent(f'{rt=} {ct=} {outBinding=}')
1635
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
228 outBinding.addNewBindings(CandidateBinding({rt: ct}))
22d481f0a924 refactor: use CandidateBinding throughout, not loose dicts
drewp@bigasterisk.com
parents: 1634
diff changeset
229 return outBinding
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
230
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
231 def currentBinding(self) -> CandidateBinding:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
232 if self.pastEnd():
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
233 raise NotImplementedError()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
234 return self._current
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
235
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
236 def pastEnd(self) -> bool:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
237 return self._pastEnd
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
238
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
239 def restart(self):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
240 self._pastEnd = False
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
241 self._seenBindings = []
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
242 self.advance()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
243 if self.pastEnd():
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
244 raise NoOptions()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
245
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
246
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
247 @dataclass
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
248 class Lhs:
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
249 graph: Graph
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
250
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
251 def __post_init__(self):
1636
3252bdc284bc rm dead code from previous tries
drewp@bigasterisk.com
parents: 1635
diff changeset
252 pass
1602
e3c44ac6d3c5 do findEvals once at setRules time
drewp@bigasterisk.com
parents: 1601
diff changeset
253
1609
34f2817320cc new tests for a smaller part of the inner loop
drewp@bigasterisk.com
parents: 1608
diff changeset
254 def __repr__(self):
34f2817320cc new tests for a smaller part of the inner loop
drewp@bigasterisk.com
parents: 1608
diff changeset
255 return f"Lhs({graphDump(self.graph)})"
34f2817320cc new tests for a smaller part of the inner loop
drewp@bigasterisk.com
parents: 1608
diff changeset
256
1621
da235054caa0 rename workingSet, sometimes
drewp@bigasterisk.com
parents: 1620
diff changeset
257 def findCandidateBindings(self, knownTrue: ReadOnlyWorkingSet, stats) -> Iterator['BoundLhs']:
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
258 """bindings that fit the LHS of a rule, using statements from workingSet and functions
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
259 from LHS"""
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
260 if self.graph.__len__() == 0:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
261 # special case- no LHS!
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
262 yield BoundLhs(self, CandidateBinding({}))
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
263 return
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
264
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
265 log.debug(f'{INDENT*4} build new StmtLooper stack')
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
266
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
267 try:
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
268 stmtStack = self._assembleRings(knownTrue)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
269 except NoOptions:
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
270 log.debug(f'{INDENT*5} start up with no options; 0 bindings')
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
271 return
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
272 self._debugStmtStack('initial odometer', stmtStack)
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
273 self._assertAllRingsAreValid(stmtStack)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
274
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
275 lastRing = stmtStack[-1]
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
276 iterCount = 0
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
277 while True:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
278 iterCount += 1
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
279 if iterCount > 10:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
280 raise ValueError('stuck')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
281
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
282 log.debug(f'{INDENT*4} vv findCandBindings iteration {iterCount}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
283
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
284 yield BoundLhs(self, lastRing.currentBinding())
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
285
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
286 self._debugStmtStack('odometer', stmtStack)
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
287
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
288 done = self._advanceAll(stmtStack)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
289
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
290 self._debugStmtStack('odometer after ({done=})', stmtStack)
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
291
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
292 log.debug(f'{INDENT*4} ^^ findCandBindings iteration done')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
293 if done:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
294 break
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
295
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
296 def _debugStmtStack(self, label, stmtStack):
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
297 log.debug(f'{INDENT*5} {label}:')
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
298 for l in stmtStack:
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
299 log.debug(f'{INDENT*6} {l} curbind={l.currentBinding() if not l.pastEnd() else "<end>"}')
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
300
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
301 def _assembleRings(self, knownTrue: ReadOnlyWorkingSet) -> List[StmtLooper]:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
302 """make StmtLooper for each stmt in our LHS graph, but do it in a way that they all
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
303 start out valid (or else raise NoOptions)"""
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
304
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
305 usedByFuncs: Set[Triple] = set() # don't worry about matching these
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
306 stmtsToResolve = list(self.graph)
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
307 for i, s in enumerate(stmtsToResolve):
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
308 if s[1] == MATH['sum']:
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
309 _, used = parseList(self.graph, s[0])
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
310 usedByFuncs.update(used)
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
311
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
312 stmtsToAdd = [stmt for stmt in stmtsToResolve if not stmt in usedByFuncs]
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
313
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
314 # sort them by variable dependencies; don't just try all perms!
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
315 def lightSortKey(stmt): # Not this. Though it helps performance on the big rdf list cases.
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
316 (s, p, o) = stmt
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
317 return p == MATH['sum'], p, s, o
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
318
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
319 stmtsToAdd.sort(key=lightSortKey)
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
320
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
321 for perm in itertools.permutations(stmtsToAdd):
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
322 stmtStack: List[StmtLooper] = []
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
323 prev: Optional[StmtLooper] = None
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
324 log.debug(f'{INDENT*5} try stmts in this order: {" -> ".join(graphDump([p]) for p in perm)}')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
325
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
326 for s in perm:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
327 try:
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
328 elem = StmtLooper(s, prev, knownTrue, parent=self)
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
329 except NoOptions:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
330 log.debug(f'{INDENT*6} permutation didnt work, try another')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
331 break
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
332 stmtStack.append(elem)
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
333 prev = stmtStack[-1]
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
334 else:
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
335 return stmtStack
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
336 log.debug(f'{INDENT*6} no perms worked- rule cannot match anything')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
337
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
338 raise NoOptions()
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
339
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
340 def _advanceAll(self, stmtStack: List[StmtLooper]) -> bool:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
341 carry = True # 1st elem always must advance
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
342 for i, ring in enumerate(stmtStack):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
343 # unlike normal odometer, advancing any earlier ring could invalidate later ones
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
344 if carry:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
345 log.debug(f'{INDENT*5} advanceAll [{i}] {ring} carry/advance')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
346 ring.advance()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
347 carry = False
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
348 if ring.pastEnd():
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
349 if ring is stmtStack[-1]:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
350 log.debug(f'{INDENT*5} advanceAll [{i}] {ring} says we done')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
351 return True
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
352 log.debug(f'{INDENT*5} advanceAll [{i}] {ring} restart')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
353 ring.restart()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
354 carry = True
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
355 return False
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
356
1633
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
357 def _assertAllRingsAreValid(self, stmtStack):
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
358 if any(ring.pastEnd() for ring in stmtStack): # this is an unexpected debug assertion
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
359 log.debug(f'{INDENT*5} some rings started at pastEnd {stmtStack}')
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
360 raise NoOptions()
6107603ed455 fix farenheit rule case, fix some others that depend on rings order, but this breaks some performance because of itertools.perm
drewp@bigasterisk.com
parents: 1632
diff changeset
361
1592
d7b66234064b pure reordering of funcs to make the next diffs smaller
drewp@bigasterisk.com
parents: 1591
diff changeset
362
1622
38bd8ef9ef67 add CandidateTermMatches, unused so far
drewp@bigasterisk.com
parents: 1621
diff changeset
363 @dataclass
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
364 class BoundLhs:
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
365 lhs: Lhs
1610
6fc48ef4c696 mysteriously lost an important line
drewp@bigasterisk.com
parents: 1609
diff changeset
366 binding: CandidateBinding
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
367
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
368
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
369 @dataclass
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
370 class Rule:
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
371 lhsGraph: Graph
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
372 rhsGraph: Graph
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
373
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
374 def __post_init__(self):
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
375 self.lhs = Lhs(self.lhsGraph)
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
376 #
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
377 self.rhsBnodeMap = {}
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
378
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
379 def applyRule(self, workingSet: Graph, implied: Graph, stats: Dict):
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
380 for bound in self.lhs.findCandidateBindings(ReadOnlyGraphAggregate([workingSet]), stats):
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
381 log.debug(f'{INDENT*5} +rule has a working binding: {bound}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
382
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
383 # rhs could have more bnodes, and they just need to be distinct per rule-firing that we do
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
384 existingRhsBnodes = set()
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
385 for stmt in self.rhsGraph:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
386 for t in stmt:
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
387 if isinstance(t, BNode):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
388 existingRhsBnodes.add(t)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
389 # if existingRhsBnodes:
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
390 # log.debug(f'{INDENT*6} mapping rhs bnodes {existingRhsBnodes} to new ones')
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
391
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
392 for b in existingRhsBnodes:
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
393
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
394 key = tuple(sorted(bound.binding.binding.items())), b
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
395 self.rhsBnodeMap.setdefault(key, BNode())
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
396
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
397 bound.binding.addNewBindings(CandidateBinding({b: self.rhsBnodeMap[key]}))
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
398
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
399 # for lhsBoundStmt in bound.binding.apply(bound.lhsStmtsWithoutEvals()):
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
400 # log.debug(f'{INDENT*6} adding to workingSet {lhsBoundStmt=}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
401 # workingSet.add(lhsBoundStmt)
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
402 # log.debug(f'{INDENT*6} rhsGraph is good: {list(self.rhsGraph)}')
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
403
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
404 for newStmt in bound.binding.apply(self.rhsGraph):
1631
2c85a4f5dd9c big rewrite of infer() using statements not variables as the things to iterate over
drewp@bigasterisk.com
parents: 1627
diff changeset
405 # log.debug(f'{INDENT*6} adding {newStmt=}')
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
406 workingSet.add(newStmt)
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
407 implied.add(newStmt)
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
408
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
409
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
410 class Inference:
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
411
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
412 def __init__(self) -> None:
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
413 self.rules = []
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
414
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
415 def setRules(self, g: ConjunctiveGraph):
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
416 self.rules: List[Rule] = []
1599
abbf0eb0e640 fix a bug with a slightly moer complicated set of rules
drewp@bigasterisk.com
parents: 1598
diff changeset
417 for stmt in g:
abbf0eb0e640 fix a bug with a slightly moer complicated set of rules
drewp@bigasterisk.com
parents: 1598
diff changeset
418 if stmt[1] == LOG['implies']:
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
419 self.rules.append(Rule(stmt[0], stmt[2]))
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
420 # other stmts should go to a default working set?
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
421
1601
30463df12d89 infer() dumps stats
drewp@bigasterisk.com
parents: 1600
diff changeset
422 @INFER_CALLS.time()
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
423 def infer(self, graph: Graph):
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
424 """
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
425 returns new graph of inferred statements.
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
426 """
1626
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
427 n = graph.__len__()
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
428 INFER_GRAPH_SIZE.observe(n)
7b3656867185 metrics on input graph sizes
drewp@bigasterisk.com
parents: 1623
diff changeset
429 log.info(f'{INDENT*0} Begin inference of graph len={n} with rules len={len(self.rules)}:')
1601
30463df12d89 infer() dumps stats
drewp@bigasterisk.com
parents: 1600
diff changeset
430 startTime = time.time()
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
431 stats: Dict[str, Union[int, float]] = defaultdict(lambda: 0)
1589
5c1055be3c36 WIP more debugging, working towards bnode-matching support
drewp@bigasterisk.com
parents: 1588
diff changeset
432 # everything that is true: the input graph, plus every rule conclusion we can make
1593
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
433 workingSet = Graph()
b0df43d5494c big rewrite- more classes, smaller methods, more typesafe, all current tests passing
drewp@bigasterisk.com
parents: 1592
diff changeset
434 workingSet += graph
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
435
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
436 # just the statements that came from RHS's of rules that fired.
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
437 implied = ConjunctiveGraph()
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
438
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
439 bailout_iterations = 100
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
440 delta = 1
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
441 stats['initWorkingSet'] = cast(int, workingSet.__len__())
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
442 while delta > 0 and bailout_iterations > 0:
1620
92f8deb59735 log layout
drewp@bigasterisk.com
parents: 1616
diff changeset
443 log.debug('')
1597
387a9cb66517 logging adjustments
drewp@bigasterisk.com
parents: 1596
diff changeset
444 log.info(f'{INDENT*1}*iteration ({bailout_iterations} left)')
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
445 bailout_iterations -= 1
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
446 delta = -len(implied)
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
447 self._iterateAllRules(workingSet, implied, stats)
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
448 delta += len(implied)
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
449 stats['iterations'] += 1
1597
387a9cb66517 logging adjustments
drewp@bigasterisk.com
parents: 1596
diff changeset
450 log.info(f'{INDENT*2} this inference iteration added {delta} more implied stmts')
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
451 stats['timeSpent'] = round(time.time() - startTime, 3)
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
452 stats['impliedStmts'] = len(implied)
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
453 log.info(f'{INDENT*0} Inference done {dict(stats)}. Implied:')
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
454 for st in implied:
1597
387a9cb66517 logging adjustments
drewp@bigasterisk.com
parents: 1596
diff changeset
455 log.info(f'{INDENT*1} {st}')
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
456 return implied
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
457
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
458 def _iterateAllRules(self, workingSet: Graph, implied: Graph, stats):
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
459 for i, rule in enumerate(self.rules):
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
460 self._logRuleApplicationHeader(workingSet, i, rule)
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
461 rule.applyRule(workingSet, implied, stats)
1587
9a3a18c494f9 WIP new inferencer. no vars yet.
drewp@bigasterisk.com
parents:
diff changeset
462
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
463 def _logRuleApplicationHeader(self, workingSet, i, r: Rule):
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
464 if not log.isEnabledFor(logging.DEBUG):
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
465 return
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
466
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
467 log.debug('')
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
468 log.debug(f'{INDENT*2} workingSet:')
1597
387a9cb66517 logging adjustments
drewp@bigasterisk.com
parents: 1596
diff changeset
469 for j, stmt in enumerate(sorted(workingSet)):
387a9cb66517 logging adjustments
drewp@bigasterisk.com
parents: 1596
diff changeset
470 log.debug(f'{INDENT*3} ({j}) {stmt}')
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
471
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
472 log.debug('')
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
473 log.debug(f'{INDENT*2}-applying rule {i}')
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
474 log.debug(f'{INDENT*3} rule def lhs:')
1634
ba59cfc3c747 hack math:sum in there. Test suite is passing except some slow performers
drewp@bigasterisk.com
parents: 1633
diff changeset
475 for stmt in sorted(r.lhsGraph, reverse=True):
1632
bd79a2941cab just (a lot of) debug changes
drewp@bigasterisk.com
parents: 1631
diff changeset
476 log.debug(f'{INDENT*4} {stmt}')
1607
b21885181e35 more modules, types. Maybe less repeated computation on BoundLhs
drewp@bigasterisk.com
parents: 1605
diff changeset
477 log.debug(f'{INDENT*3} rule def rhs: {graphDump(r.rhsGraph)}')
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
478
1588
0757fafbfdab WIP inferencer - partial var and function support
drewp@bigasterisk.com
parents: 1587
diff changeset
479
1590
327202020892 WIP inference- getting into more degenerate test cases
drewp@bigasterisk.com
parents: 1589
diff changeset
480 def graphDump(g: Union[Graph, List[Triple]]):
327202020892 WIP inference- getting into more degenerate test cases
drewp@bigasterisk.com
parents: 1589
diff changeset
481 if not isinstance(g, Graph):
327202020892 WIP inference- getting into more degenerate test cases
drewp@bigasterisk.com
parents: 1589
diff changeset
482 g2 = Graph()
1594
e58bcfa66093 cleanups and a few fixed cases
drewp@bigasterisk.com
parents: 1593
diff changeset
483 g2 += g
1590
327202020892 WIP inference- getting into more degenerate test cases
drewp@bigasterisk.com
parents: 1589
diff changeset
484 g = g2
1589
5c1055be3c36 WIP more debugging, working towards bnode-matching support
drewp@bigasterisk.com
parents: 1588
diff changeset
485 g.bind('', ROOM)
5c1055be3c36 WIP more debugging, working towards bnode-matching support
drewp@bigasterisk.com
parents: 1588
diff changeset
486 g.bind('ex', Namespace('http://example.com/'))
5c1055be3c36 WIP more debugging, working towards bnode-matching support
drewp@bigasterisk.com
parents: 1588
diff changeset
487 lines = cast(bytes, g.serialize(format='n3')).decode('utf8').splitlines()
1612
272f78d4671a mark skipped tests. move applyRule into Rule. minor cleanups.
drewp@bigasterisk.com
parents: 1611
diff changeset
488 lines = [line.strip() for line in lines if not line.startswith('@prefix')]
1589
5c1055be3c36 WIP more debugging, working towards bnode-matching support
drewp@bigasterisk.com
parents: 1588
diff changeset
489 return ' '.join(lines)