Mercurial > code > home > repos > homeauto

annotate service/reasoning/rdflibtrig.py @ 1462:2b29f14eb6bd

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
try new graph+view widget Ignore-this: d5f9c5dc52f04324368716ba2f604fdb darcs-hash:44e85a5c075ef73c34a58deaa3a3c1e8390dae52
author drewp <drewp@bigasterisk.com>
date Sun, 24 Nov 2019 00:01:00 -0800
parents bbaf0576f653
children 3c18b4b3b72c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1054
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
1 import time, logging
1040
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
2 from rdflib import ConjunctiveGraph
1054
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
3 from rdflib.parser import StringInputSource
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
4 import treq
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
5 from twisted.internet.defer import inlineCallbacks, returnValue
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
6 log = logging.getLogger('fetch')
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
7
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
8 from private_ipv6_addresses import ipv6Addresses
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
9
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
10 @inlineCallbacks
934
3bb18b7d21df reasoning actions: generalize them a bit but then add a bunch of special cases for mpd for now
drewp <drewp@bigasterisk.com>
parents: 923
diff changeset
11 def addTrig(graph, url, timeout=2):
919
6ee2a90fc816 bugs in async http client. move trig helpers to rdflibtrig, which can work with rdflib 4
drewp <drewp@bigasterisk.com>
parents:
diff changeset
12 t1 = time.time()
1054
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
13 # workaround for some reason my ipv6 names don't resolve
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
14 for name, addr in ipv6Addresses.iteritems():
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
15 url = url.replace('/' + name + ':', '/[' + addr + ']:')
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
16 log.debug(' fetching %r', url)
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
17 response = yield treq.get(url, headers={'accept': ['application/trig']}, timeout=timeout)
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
18 if response.code != 200:
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
19 raise ValueError("status %s from %s" % (response.code, url))
1045
a328cc370b22 ipv6 fetch support. refactor Actions to new class and file
drewp <drewp@bigasterisk.com>
parents: 1040
diff changeset
20 g = ConjunctiveGraph()
1054
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
21 g.parse(StringInputSource((yield response.content())), format='trig')
919
6ee2a90fc816 bugs in async http client. move trig helpers to rdflibtrig, which can work with rdflib 4
drewp <drewp@bigasterisk.com>
parents:
diff changeset
22 fetchTime = time.time() - t1
1054
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
23 log.debug(' %r done in %.04f sec', url, fetchTime)
1040
2fa7daf06f34 rdflib can parse trig now
drewp <drewp@bigasterisk.com>
parents: 934
diff changeset
24 graph.addN(g.quads())
1054
bbaf0576f653 fetch all source graphs in parallel
drewp <drewp@bigasterisk.com>
parents: 1045
diff changeset
25 returnValue(fetchTime)