Mercurial > code > home > repos > victoriametrics

annotate tasks.py @ 70:68164135ae66

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
refactor tasks.py to smaller steps
author drewp@bigasterisk.com
date Fri, 03 May 2024 11:56:59 -0700
parents c5e98d891638
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
62
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
1 from pathlib import Path
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
2
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
3 import yaml
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
4 from invoke import task
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
5 from kubernetes import config
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
6
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
7 import alert_rules
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
8 from k8s_ops import firstPodName, refreshPodCmaps, replaceCmap
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
9
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
10 config.load_kube_config()
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
11
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
12
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
13 def scrapeConfig(fn):
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
14 return yaml.load(open(fn), yaml.FullLoader)
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
15
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
16
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
17 @task
70
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
18 def build_k8s_config(ctx):
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
19 ctx.run('pdm run python create_k8s.py')
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
20
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
21
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
22 @task
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
23 def build_scrape_config(ctx):
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
24 ctx.run('pdm run python create_scrape_configs.py')
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
25
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
26
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
27 @task(pre=[build_k8s_config, build_scrape_config])
66
429bfd62e6ba clean out old config; move remaining bits into next/
drewp@bigasterisk.com
parents: 64
diff changeset
28 def push_config(ctx):
70
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
29 cmap: dict[str, object] = {
62
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
30 'rules': alert_rules.allRules(ctx),
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
31 }
8134cd480817 make next/ a complete standalone setup dir- no deps on ./
drewp@bigasterisk.com
parents:
diff changeset
32 top = Path('build/scrape_config')
64
def1aa2bfa3f more targets polish. reorg code into next/
drewp@bigasterisk.com
parents: 62
diff changeset
33 for p in top.glob('*.yaml'):
70
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
34 cmap[str(p.relative_to(top))] = scrapeConfig(p)
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
35 replaceCmap("victoriametrics-config", cmap)
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
36
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
37 # note: these will fail on first run, since you have to make config before
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
38 # you can launch the service for the first time. Also, these won't error if
68164135ae66 refactor tasks.py to smaller steps
drewp@bigasterisk.com
parents: 68
diff changeset
39 # there are problems with the config. Watch the vmagent (etc) logs.
68
c5e98d891638 remove 'next' prefix
drewp@bigasterisk.com
parents: 67
diff changeset
40 refreshPodCmaps(firstPodName("app=victoriametrics-forever-vmagent"))
c5e98d891638 remove 'next' prefix
drewp@bigasterisk.com
parents: 67
diff changeset
41 refreshPodCmaps(firstPodName("app=victoriametrics-recent-vmagent"))