annotate service/reasoning/rdflibtrig.py @ 1754:92999dfbf321 default tip

add shelly support
author drewp@bigasterisk.com
date Tue, 04 Jun 2024 13:03:43 -0700
parents 3c18b4b3b72c
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
249
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
1 import time, logging
235
5ad229334a88 rdflib can parse trig now
drewp@bigasterisk.com
parents: 129
diff changeset
2 from rdflib import ConjunctiveGraph
249
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
3 from rdflib.parser import StringInputSource
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
4 import treq
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
5 from twisted.internet.defer import inlineCallbacks, returnValue
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
6 log = logging.getLogger('fetch')
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
7
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
8 from private_ipv6_addresses import ipv6Addresses
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
9
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
10 @inlineCallbacks
129
745eff67ad40 reasoning actions: generalize them a bit but then add a bunch of special cases for mpd for now
drewp@bigasterisk.com
parents: 118
diff changeset
11 def addTrig(graph, url, timeout=2):
114
4cd065b97fa1 bugs in async http client. move trig helpers to rdflibtrig, which can work with rdflib 4
drewp@bigasterisk.com
parents:
diff changeset
12 t1 = time.time()
249
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
13 # workaround for some reason my ipv6 names don't resolve
760
3c18b4b3b72c more py3 fixes
drewp@bigasterisk.com
parents: 249
diff changeset
14 for name, addr in ipv6Addresses.items():
249
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
15 url = url.replace('/' + name + ':', '/[' + addr + ']:')
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
16 log.debug(' fetching %r', url)
760
3c18b4b3b72c more py3 fixes
drewp@bigasterisk.com
parents: 249
diff changeset
17 response = yield treq.get(url, headers={'accept': ['application/trig']},
3c18b4b3b72c more py3 fixes
drewp@bigasterisk.com
parents: 249
diff changeset
18 timeout=timeout)
249
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
19 if response.code != 200:
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
20 raise ValueError("status %s from %s" % (response.code, url))
240
0c306e76d8c5 ipv6 fetch support. refactor Actions to new class and file
drewp@bigasterisk.com
parents: 235
diff changeset
21 g = ConjunctiveGraph()
249
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
22 g.parse(StringInputSource((yield response.content())), format='trig')
114
4cd065b97fa1 bugs in async http client. move trig helpers to rdflibtrig, which can work with rdflib 4
drewp@bigasterisk.com
parents:
diff changeset
23 fetchTime = time.time() - t1
249
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
24 log.debug(' %r done in %.04f sec', url, fetchTime)
235
5ad229334a88 rdflib can parse trig now
drewp@bigasterisk.com
parents: 129
diff changeset
25 graph.addN(g.quads())
249
e5c27d2f11ab fetch all source graphs in parallel
drewp@bigasterisk.com
parents: 240
diff changeset
26 returnValue(fetchTime)