gcalendarwatch: gcalendarwatch.py annotate

annotate gcalendarwatch.py @ 21:9eb6b4806272

timmzone fixes, especially on GET /events

author	drewp@bigasterisk.com
date	Mon, 08 Feb 2021 16:40:47 -0800
parents	8c7af0d1b118
children	8122ff3b0fe5

rev	line source
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	1 """
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	2 sync google calendar into mongodb, return queries from that as
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	3 JSON-LD.
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	4
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	5 gcalendarwatch.conf looks like this:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	6 {
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	7 "minutes_between_polls" : 60
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	8 "mongo" : {"host" : "h1", "port" : 27017, "database" : "dbname", "collection" : "pim"},
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	9 }
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	10
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	11 This should be updated to use
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	12 http://googledevelopers.blogspot.com/2013/07/google-calendar-api-push-notifications.html
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	13 and update faster with less polling
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	14 """
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	15 import datetime
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	16 import json
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	17 import re
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	18 import time
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	19 import traceback
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	20
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	21 import cyclone.web
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	22 import pymongo.collection
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	23 from dateutil.parser import parse
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	24 from dateutil.tz import tzlocal
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	25 from patchablegraph import (
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	26 CycloneGraphEventsHandler,
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	27 CycloneGraphHandler,
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	28 PatchableGraph,
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	29 )
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	30 from calendar_connection import getCalendarService
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	31 from typing import Dict, Any
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	32 from pymongo import MongoClient
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	33 from rdflib import Graph, Literal, Namespace, RDF, URIRef
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	34 from standardservice.logsetup import log, verboseLogging
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	35 from twisted.internet import reactor
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	36
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	37 import docopt
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	38 from prometheus_client import Summary
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	39 from prometheus_client.exposition import generate_latest
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	40 from prometheus_client.registry import REGISTRY
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	41
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	42 UPDATE = Summary('gcalendarwatch_updates', 'update loop calls')
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	43
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	44 EV = Namespace("http://bigasterisk.com/event#")
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	45 """
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	46 example:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	47 {
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	48 'id': 'l640999999999999999999999c',
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	49 'summary': 'sec.......',
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	50 'start': {'timeZone': 'America/Los_Angeles', 'dateTime': '2014-09-25T16:00:00-07:00'},
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	51 'end': {'timeZone': 'America/Los_Angeles', 'dateTime': '2014-09-25T17:00:00-07:00'},
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	52 'endTimeUnspecified': True,
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	53 'created': '2014-09-08T20:39:00.000Z',
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	54 'creator': {'self': True, 'displayName': '...', 'email': '...'},
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	55 'etag': '"2829999999999000"',
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	56 'htmlLink': 'https://www.google.com/calendar/event?eid=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbEBt',
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	57 'iCalUID': 'l640998888888888888888888888888888com',
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	58 'kind': 'calendar#event',
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	59 'organizer': {'self': True, 'displayName': '...', 'email': '...'},
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	60 'reminders': {'useDefault': True},
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	61 'sequence': 0,
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	62 'status': 'confirmed',
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	63 'updated': '2014-09-17T04:28:56.997Z',
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	64 }"""
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	65
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	66
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	67 def feedFromCalId(conf, calId):
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	68 return conf['event_uri_ns'] + 'feed/' + calId
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	69
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	70
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	71 def recordFromEv(conf: Dict, calId: str, ev: Dict):
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	72
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	73 def dateOrTime(d):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	74 if 'date' in d:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	75 return d['date']
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	76 return d['dateTime']
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	77
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	78 rec = {
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	79 'uri': conf['event_uri_ns'] + ev['id'],
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	80 'feed': feedFromCalId(conf, calId),
6 9cface2e6b11 don't break on missing summaries drewp@bigasterisk.com parents: 4 diff changeset	81 'title': ev.get('summary', '?'),
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	82 'start': dateOrTime(ev['start']),
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	83 'end': dateOrTime(ev['end']),
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	84 'endTimeUnspecified': ev.get('endTimeUnspecified', False),
11 702f74c3d742 even more optional fields drewp@bigasterisk.com parents: 10 diff changeset	85 'htmlLink': ev.get('htmlLink', ''),
702f74c3d742 even more optional fields drewp@bigasterisk.com parents: 10 diff changeset	86 'creatorEmail': ev.get('creator', {}).get('email', ''),
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	87 }
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	88
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	89 for field, val in [('start', ev['start']), ('end', ev['end'])]:
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	90 if 'date' in val:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	91 rec['%sTime' % field] = parse(val['date']).replace(tzinfo=tzlocal())
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	92 rec['%sDate' % field] = val['date']
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	93 else:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	94 rec['%sTime' % field] = parse(val['dateTime'])
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	95 rec['%sDate' % field] = parse(val['dateTime']).date().isoformat()
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	96 return rec
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	97
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	98
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	99 def asJsonLd(events):
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	100 ret = {'@graph': []} # type: Dict[Any, Any]
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	101 for ev in events:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	102 ev['startTime'] = ev['startTime'].astimezone(tzlocal()).isoformat()
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	103 ev['endTime'] = ev['endTime'].astimezone(tzlocal()).isoformat()
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	104 ev['@id'] = ev.pop('uri')
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	105 ret['@graph'].append(ev)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	106
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	107 ret['@context'] = {
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	108 "xsd": "http://www.w3.org/2001/XMLSchema#",
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	109 "ev": "http://bigasterisk.com/event#",
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	110 "startTime": {
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	111 "@id": "ev:startTime",
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	112 "@type": "xsd:dateTime"
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	113 },
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	114 "endTime": {
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	115 "@id": "ev:endTime",
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	116 "@type": "xsd:dateTime"
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	117 },
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	118 "startDate": {
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	119 "@id": "ev:startDate",
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	120 "@type": "xsd:date"
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	121 },
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	122 "endDate": {
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	123 "@id": "ev:endDate",
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	124 "@type": "xsd:date"
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	125 },
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	126 "title": "ev:title",
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	127 "feed": {
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	128 "@id": "ev:feed",
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	129 "@type": "@id"
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	130 },
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	131 "htmlLink": {
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	132 "@id": "ev:htmlLink",
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	133 "@type": "@id"
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	134 },
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	135 }
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	136 return ret
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	137
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	138
7 9cf0859441f9 another rdf class for CountdownEvent records drewp@bigasterisk.com parents: 6 diff changeset	139 def asGraph(events, extraClasses=[]):
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	140 graph = Graph()
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	141 graph.namespace_manager.bind('ev', EV)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	142 for ev in events:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	143 uri = URIRef(ev['uri'])
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	144
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	145 def add(p, o):
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	146 return graph.add((uri, p, o))
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	147
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	148 add(RDF.type, EV['Event'])
7 9cf0859441f9 another rdf class for CountdownEvent records drewp@bigasterisk.com parents: 6 diff changeset	149 for cls in extraClasses:
9cf0859441f9 another rdf class for CountdownEvent records drewp@bigasterisk.com parents: 6 diff changeset	150 add(RDF.type, cls)
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	151 add(EV['title'], Literal(ev['title']))
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	152 add(EV['start'], Literal(ev['start']))
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	153 add(EV['startDate'], Literal(ev['startDate']))
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	154 add(EV['end'], Literal(ev['end']))
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	155 add(EV['feed'], URIRef(ev['feed']))
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	156 # graph.add((feed, RDFS.label, Literal(ev['feedTitle'])))
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	157 if 'htmlLink' in ev:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	158 add(EV['htmlLink'], URIRef(ev['htmlLink']))
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	159 return graph
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	160
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	161
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	162 def getFirstPageOfCalendars(service):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	163 for row in service.calendarList().list().execute()['items']:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	164 yield row['id']
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	165
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	166
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	167 def dayRange(days):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	168 now = datetime.datetime.now(tzlocal())
4 fb02ad302f2f include more recent-past events drewp@bigasterisk.com parents: 3 diff changeset	169 start = now - datetime.timedelta(hours=12)
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	170 end = now + datetime.timedelta(days=days)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	171 return start, end
1 d77ead665ab2 whitespace drewp@bigasterisk.com parents: 0 diff changeset	172
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	173
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	174 def limitDays(recs, days):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	175 start, end = dayRange(days)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	176 start = start - datetime.timedelta(hours=12)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	177 # incomplete
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	178 return [r for r in recs if r['startTime'] < end and r['endTime'] > start]
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	179
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	180
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	181 def starred(graph, ev):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	182 title = graph.value(ev, EV['title'])
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	183 m = re.search(r'(.)\\s*$', title)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	184 if m:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	185 return m.group(1)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	186 else:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	187 return None
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	188
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	189
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	190 def filterStarred(recs, maxCount=15):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	191 recs = sorted(recs, key=lambda r: r['start'])
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	192 out = []
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	193 for rec in recs:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	194 if re.search(r'(.)\\s*$', rec['title']):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	195 out.append(rec)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	196 if len(out) >= maxCount:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	197 break
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	198 return out
1 d77ead665ab2 whitespace drewp@bigasterisk.com parents: 0 diff changeset	199
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	200
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	201 class SyncToMongo(object):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	202 """reads gcal, writes to mongodb"""
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	203 collection: pymongo.collection.Collection
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	204
20 8c7af0d1b118 format drewp@bigasterisk.com parents: 19 diff changeset	205 def __init__(self, conf, collection: pymongo.collection.Collection, agendaGraph: PatchableGraph, countdownGraph: PatchableGraph):
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	206 self.conf = conf
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	207 self.service = getCalendarService()
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	208 self.collection = collection
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	209 self.agendaGraph = agendaGraph
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	210 self.countdownGraph = countdownGraph
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	211
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	212 @UPDATE.time()
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	213 def update(self, days=10, cal=None) -> int:
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	214 start, end = dayRange(days)
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	215 spec = {"startTime": {"$gte": start, "$lte": end}}
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	216 if cal is not None:
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	217 spec['feed'] = feedFromCalId(self.conf, cal)
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	218 idsFormerlyInRange = [doc['_id'] for doc in self.collection.find(spec)]
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	219 n = self.collection.delete_many(spec)
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	220 log.info(f'cleared {n} records before reread')
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	221
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	222 totalNew = 0
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	223 currentRecords = []
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	224 for calId in getFirstPageOfCalendars(self.service):
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	225 if cal and calId != cal:
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	226 continue
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	227 print('read %s' % calId)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	228 events = self.service.events().list(
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	229 calendarId=calId,
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	230 singleEvents=True,
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	231 timeMin=start.isoformat(),
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	232 timeMax=end.isoformat(),
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	233 showDeleted=False,
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	234 maxResults=1000,
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	235 ).execute()
1 d77ead665ab2 whitespace drewp@bigasterisk.com parents: 0 diff changeset	236
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	237 for ev in events['items']:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	238 rec = recordFromEv(self.conf, calId, ev)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	239 self.upsertMongo(rec)
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	240 if rec['uri'] not in idsFormerlyInRange:
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	241 totalNew += 1
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	242 currentRecords.append(rec)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	243
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	244 self.updateGraphs(currentRecords)
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	245 return totalNew
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	246
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	247 def upsertMongo(self, rec):
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	248 if self.collection.find_one({"_id": rec['uri']}) is not None:
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	249 log.debug("existing record %s", rec['uri'])
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	250 # this is not yet noticing updates
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	251 return []
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	252 else:
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	253 log.debug("add record %s", rec)
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	254 d = rec.copy()
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	255 d['_id'] = d.pop('uri')
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	256 self.collection.insert_one(d)
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	257 return [rec]
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	258
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	259 def updateGraphs(self, currentRecords):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	260 c = EV['gcalendar']
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	261 currentRecords = list(currentRecords)
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	262 self.agendaGraph.setToGraph([(s, p, o, c) for s, p, o in asGraph(limitDays(currentRecords, days=2))])
20 8c7af0d1b118 format drewp@bigasterisk.com parents: 19 diff changeset	263 self.countdownGraph.setToGraph([(s, p, o, c) for s, p, o in asGraph(filterStarred(currentRecords, maxCount=15), extraClasses=[EV['CountdownEvent']])])
1 d77ead665ab2 whitespace drewp@bigasterisk.com parents: 0 diff changeset	264
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	265
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	266 class ReadMongoEvents(object):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	267 """read events from mongodb"""
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	268
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	269 def __init__(self, collection):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	270 self.collection = collection
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	271
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	272 def getEvents(self, t1, t2):
21 9eb6b4806272 timmzone fixes, especially on GET /events drewp@bigasterisk.com parents: 20 diff changeset	273 if t1.tzinfo is None or t2.tzinfo is None:
9eb6b4806272 timmzone fixes, especially on GET /events drewp@bigasterisk.com parents: 20 diff changeset	274 raise TypeError("tz-naive datetimes")
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	275 for doc in self.collection.find({"startTime": {"$gte": t1, "$lt": t2}}).sort([("startTime", 1)]):
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	276 doc['uri'] = doc.pop('_id')
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	277 if 'feedId' in doc:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	278 doc['feed'] = URIRef('old_event')
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	279 yield doc
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	280
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	281
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	282 class Poller(object):
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	283
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	284 def __init__(self, sync, periodSec):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	285 self.sync = sync
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	286 self.lastUpdateTime = time.time()
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	287 self.everUpdated = False
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	288 self.periodSec = periodSec
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	289 self.scheduled = reactor.callLater(self.periodSec, self._updateLoop)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	290 self.events = None
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	291
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	292 def updateNow(self, cal=None) -> int:
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	293 self.scheduled.cancel()
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	294 return self._updateLoop(cal)
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	295
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	296 def _updateLoop(self, cal=None) -> int:
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	297 log.info(f"updating {cal or 'all'}")
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	298 t1 = time.time()
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	299 try:
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	300 n = self.sync.update(cal=cal)
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	301 except Exception:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	302 traceback.print_exc()
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	303 log.error("updated failed")
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	304 n = 0
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	305 self.lastUpdateTime = t1
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	306 self.everUpdated = True
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	307 took = time.time() - t1
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	308 self.scheduled = reactor.callLater(max(3, self.periodSec - took), self._updateLoop)
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	309 return n
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	310
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	311
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	312 class PollNow(cyclone.web.RequestHandler):
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	313
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	314 def post(self):
19 0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	315 cal = json.loads(self.request.body).get('cal', None) if self.request.body else None
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	316 n = self.settings.poller.updateNow(cal)
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	317 msg = f"found {n} new records"
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	318 log.info(msg)
0af075b62e4a support reread of a single cal. read fewer default days, since who knows what will happen drewp@bigasterisk.com parents: 16 diff changeset	319 self.write(msg.encode('utf8'))
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	320
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	321
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	322 class Index(cyclone.web.RequestHandler):
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	323
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	324 def get(self):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	325 period = self.settings.conf['minutes_between_polls'] * 60
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	326 ago = time.time() - self.settings.poller.lastUpdateTime
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	327 if not self.settings.poller.everUpdated:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	328 msg = "no completed updates %d sec after startup" % ago
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	329 if ago > period * 1.1:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	330 raise ValueError(msg)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	331 else:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	332 msg = "last update was %d sec ago" % ago
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	333 if ago > period * 1.1:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	334 raise ValueError(msg)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	335 self.set_header("content-type", "text/html")
10 c3f5b0236d0f rename to index.html drewp@bigasterisk.com parents: 8 diff changeset	336 self.write(open("index.html").read().replace("MSG", msg))
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	337
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	338
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	339 class EventsPage(cyclone.web.RequestHandler):
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	340
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	341 def get(self):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	342 """
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	343 upcoming events as JSON-LD
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	344 """
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	345 arg = self.get_argument
21 9eb6b4806272 timmzone fixes, especially on GET /events drewp@bigasterisk.com parents: 20 diff changeset	346 t1, t2 = dayRange(int(arg('days', default='2')))
9eb6b4806272 timmzone fixes, especially on GET /events drewp@bigasterisk.com parents: 20 diff changeset	347 if arg('t1', default=None):
9eb6b4806272 timmzone fixes, especially on GET /events drewp@bigasterisk.com parents: 20 diff changeset	348 t1 = parse(arg('t1'), tzinfo=tzlocal())
9eb6b4806272 timmzone fixes, especially on GET /events drewp@bigasterisk.com parents: 20 diff changeset	349 if arg('t2', default=None):
9eb6b4806272 timmzone fixes, especially on GET /events drewp@bigasterisk.com parents: 20 diff changeset	350 t2 = parse(arg('t2'), tzinfo=tzlocal())
9eb6b4806272 timmzone fixes, especially on GET /events drewp@bigasterisk.com parents: 20 diff changeset	351 log.info(f'get /events local t1={t1} t2={t2}')
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	352 if 0:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	353 self.set_header("content-type", "application/ld+json")
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	354 self.write(asJsonLd(self.settings.read.getEvents(t1, t2)))
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	355 else:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	356 self.set_header("content-type", "text/n3")
7 9cf0859441f9 another rdf class for CountdownEvent records drewp@bigasterisk.com parents: 6 diff changeset	357 self.write(asGraph(self.settings.read.getEvents(t1, t2)).serialize(format='n3'))
1 d77ead665ab2 whitespace drewp@bigasterisk.com parents: 0 diff changeset	358
d77ead665ab2 whitespace drewp@bigasterisk.com parents: 0 diff changeset	359
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	360 class Countdowns(cyclone.web.RequestHandler):
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	361
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	362 def get(self):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	363 rows = []
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	364 graph = self.settings.countdownGraph._graph
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	365 for ev in graph.subjects(RDF.type, EV['Event']):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	366 starLabel = starred(graph, ev)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	367 if starLabel is not None:
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	368 rows.append({'@type': 'countdown', 'time': graph.value(ev, EV['start']), 'label': starLabel})
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	369 self.set_header("content-type", "application/ld+json")
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	370 self.write(
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	371 json.dumps({
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	372 "@context": {
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	373 "countdown": "http://bigasterisk.com/countdown#CountdownEvent",
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	374 "label": "http://www.w3.org/2000/01/rdf-schema#label",
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	375 "time": {
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	376 "@id": "http://bigasterisk.com/event#time",
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	377 "@type": "xsd:dateTime"
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	378 },
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	379 "xsd": "http://www.w3.org/2001/XMLSchema#",
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	380 "rdfs": "http://www.w3.org/2000/01/rdf-schema#"
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	381 },
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	382 "@graph": rows,
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	383 }))
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	384
1 d77ead665ab2 whitespace drewp@bigasterisk.com parents: 0 diff changeset	385
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	386 class Metrics(cyclone.web.RequestHandler):
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	387
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	388 def get(self):
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	389 self.add_header('content-type', 'text/plain')
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	390 self.write(generate_latest(REGISTRY))
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	391
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	392
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	393 def main():
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	394 args = docopt.docopt('''
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	395 Usage:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	396 gcalendarwatch [options]
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	397
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	398 Options:
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	399 -v, --verbose more logging
3 acb378b62dfd --now feature for testing drewp@bigasterisk.com parents: 2 diff changeset	400 --now don't wait for first update
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	401 ''')
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	402
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	403 verboseLogging(args['--verbose'])
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	404
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	405 agendaGraph = PatchableGraph() # next few days
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	406 countdownGraph = PatchableGraph() # next n of starred events
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	407 conf = json.load(open("gcalendarwatch.conf"))
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	408 m = conf['mongo']
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	409 mongoOut = MongoClient(m['host'], m['port'], tz_aware=True)[m['database']][m['collection']]
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	410 sync = SyncToMongo(conf, mongoOut, agendaGraph, countdownGraph)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	411 read = ReadMongoEvents(mongoOut)
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	412
4 fb02ad302f2f include more recent-past events drewp@bigasterisk.com parents: 3 diff changeset	413 s, e = dayRange(60)
fb02ad302f2f include more recent-past events drewp@bigasterisk.com parents: 3 diff changeset	414 sync.updateGraphs(read.getEvents(s, e))
1 d77ead665ab2 whitespace drewp@bigasterisk.com parents: 0 diff changeset	415
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	416 poller = Poller(sync, conf['minutes_between_polls'] * 60)
3 acb378b62dfd --now feature for testing drewp@bigasterisk.com parents: 2 diff changeset	417 if args['--now']:
acb378b62dfd --now feature for testing drewp@bigasterisk.com parents: 2 diff changeset	418 poller.updateNow()
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	419
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	420 class Application(cyclone.web.Application):
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	421
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	422 def __init__(self):
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	423 handlers = [
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	424 (r"/", Index),
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	425 (r'/events', EventsPage),
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	426 (r'/pollNow', PollNow),
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	427 (r'/graph/calendar/upcoming', CycloneGraphHandler, {
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	428 'masterGraph': agendaGraph
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	429 }),
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	430 (r'/graph/calendar/upcoming/events', CycloneGraphEventsHandler, {
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	431 'masterGraph': agendaGraph
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	432 }),
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	433 (r'/graph/calendar/countdown', CycloneGraphHandler, {
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	434 'masterGraph': countdownGraph
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	435 }),
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	436 (r'/graph/calendar/countdown/events', CycloneGraphEventsHandler, {
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	437 'masterGraph': countdownGraph
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	438 }),
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	439 (r'/countdowns.json', Countdowns),
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	440 (r'/metrics', Metrics),
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	441 ]
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	442 cyclone.web.Application.__init__(
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	443 self,
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	444 handlers,
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	445 conf=conf,
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	446 read=read,
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	447 poller=poller,
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	448 agendaGraph=agendaGraph,
a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	449 countdownGraph=countdownGraph,
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	450 )
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	451
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	452 reactor.listenTCP(conf['serve_port'], Application())
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	453 reactor.run()
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	454
16 a87969972d85 lib updates, auth update and cleanup, reformat drewp@bigasterisk.com parents: 15 diff changeset	455
0 e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	456 if __name__ == '__main__':
e40034f22c69 moved from pimscreen, upgrade to py3. Redo google auth. drewp@bigasterisk.com parents: diff changeset	457 main()

Mercurial > code > home > repos > gcalendarwatch

annotate gcalendarwatch.py @ 21:9eb6b4806272