Mercurial > code > home > repos > pomerium
annotate tasks.py @ 29:bd2cbc36bc65
reorder `delete` task
| author | drewp@bigasterisk.com |
|---|---|
| date | Mon, 19 Jun 2023 22:20:00 -0700 |
| parents | b53ab97e8979 |
| children | b1f75b0584f3 |
| rev | line source |
|---|---|
|
24
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
1 import sys |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
2 import time |
| 6 | 3 from invoke import task |
| 4 from invoke.exceptions import UnexpectedExit | |
| 5 | |
|
24
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
6 |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
7 def authCert(ctx): |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
8 for tries in range(100): |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
9 try: |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
10 ctx.run("kubectl apply -f config/60-auth-cert.yaml", echo=True, ) |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
11 sys.stderr.write("worked") |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
12 return |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
13 except UnexpectedExit: |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
14 time.sleep(2) |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
15 sys.stderr.write('.') |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
16 sys.stderr.flush() |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
17 raise ValueError |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
18 |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
19 |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
20 |
| 6 | 21 @task |
| 22 def run(ctx): | |
|
24
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
23 ctx.run("kubectl delete -n pomerium job/pomerium-gen-secrets --ignore-not-found", echo=True) |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
24 ctx.run("skaffold run -f use-invoke-not-skaffold.yaml", echo=True) |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
25 authCert(ctx) |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
26 ctx.run("./make_global.py | kubectl apply -f -", echo=True) |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
27 ctx.run("kubectl apply -f config/51-pomerium-production-issuer.yaml", echo=True) |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
28 ctx.run("kubectl apply -f config/51-pomerium-staging-issuer.yaml", echo=True) |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
29 |
| 6 | 30 |
|
24
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
31 @task |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
32 def delete(ctx): |
|
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
33 # todo don't delete certs that have big timeouts to remake |
| 29 | 34 ctx.run("kubectl delete -f config/51-pomerium-staging-issuer.yaml --ignore-not-found", echo=True) |
| 35 ctx.run("kubectl delete -f config/51-pomerium-production-issuer.yaml --ignore-not-found", echo=True) | |
| 36 ctx.run("kubectl delete -f config/60-auth-cert.yaml --ignore-not-found", echo=True) | |
|
24
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
37 ctx.run("kubectl delete pomerium/global --ignore-not-found", echo=True) |
| 29 | 38 ctx.run("skaffold delete -f use-invoke-not-skaffold.yaml ", echo=True) |
| 39 ctx.run("kubectl delete -n pomerium job/pomerium-gen-secrets --ignore-not-found", echo=True) | |
|
24
b53ab97e8979
reorganize, and add two retry loops to try to get everything to startup in one 'inv run'
drewp@bigasterisk.com
parents:
23
diff
changeset
|
40 |
| 11 | 41 |
| 42 ''' | |
| 43 troubleshooting, based on | |
| 44 https://cert-manager.io/docs/troubleshooting/ | |
| 45 then | |
| 46 https://cert-manager.io/docs/concepts/acme-orders-challenges/ | |
| 47 | |
| 48 I had these open: | |
| 49 ✨ dash(pts/31):~% watch 'kubectl describe -n pomerium issuers.cert-manager.io letsencrypt-staging' | |
| 50 ✨ dash(pts/31):~% watch 'kubectl describe -n pomerium issuers.cert-manager.io letsencrypt-prod' | |
| 51 ✨ dash(pts/29):~% watch "kubectl get -n pomerium certificates.cert-manager.io -o wide" | |
| 52 ✨ dash(pts/36):~% watch 'kubectl describe -n pomerium certificaterequests.cert-manager.io' | |
| 53 ✨ dash(pts/37):~% watch 'kubectl describe -n pomerium orders.acme.cert-manager.io' | |
| 54 ✨ dash(pts/38):~% watch 'kubectl describe -n pomerium challenges.acme.cert-manager.io ' | |
| 55 | |
| 56 then i checked clusterissuer vs issuer, the ns of the 60-auth-cert.yaml resources, | |
| 57 and i often restarted cert-manager and eventually pomerium too. 10-pom-pom.yaml last line | |
| 58 may need to be toggled. | |
| 59 | |
| 60 The 'cm-acme-http-solver' ingress for LE comes and goes but i didn't have to force it to exist. | |
| 61 | |
| 62 Didn't need 04-gen-secrets-job.yaml | |
| 63 | |
| 64 Also, CM says this a lot which means it may be afraid to renew bigasterisk.com | |
| 65 | |
| 66 I1213 07:00:01.946799 1 sync.go:394] cert-manager/controller/ingress-shim "msg"="certificate resource is not owned by this object. refusing to update non-owned certificate resource for object" "related_resource_kind"="Certificate" "related_resource_name"="bigasterisk.com-tls" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Ingress" "resource_name"="registry" "resource_namespace"="default" "resource_version"="v1" | |
| 67 | |
| 68 ''' |