Difference between revisions of "StarlingX/Containers/StarlingXAppFrameworkInternals"
(→Robustness Tests) |
(Clarify target public) |
||
Line 1: | Line 1: | ||
− | This page describes details, use cases and checks that developers should be aware of when testing code changes to AppFwk | + | This page describes details, use cases and checks that Application Framework Team developers should be aware of when testing code changes to AppFwk. |
= Metadata Correctness Tests = | = Metadata Correctness Tests = |
Latest revision as of 12:28, 17 February 2023
This page describes details, use cases and checks that Application Framework Team developers should be aware of when testing code changes to AppFwk.
Contents
Metadata Correctness Tests
The data provided on the metadata.yaml file is loaded into the sysinv database. The data from the file can be checked against the database for correctness. For instance:
- Check if "maintain_user_overrides" database value matches the value from metadata.yaml
- Check if "maintain_attributes" database value matches the value from metadata.yaml
- Check if "plataform_managed_apps" database value matches the value from metadata.yaml
- Check if "desired_state" database value matches the value from metadata.yaml
- Check if evaluate reapply trigger values match the ones from metadata.yaml
- Check if "auto_update" value matches the one from metadata.yaml
Example 1: snmp
Extract the metadata.yaml from the application tarball:
sysadmin@controller-0:~$ mkdir /home/sysadmin/snmp && tar -xvf /usr/local/share/applications/helm/snmp-1.0-1.tgz -C /home/sysadmin/snmp
Check the metadata.yaml content:
sysadmin@controller-0:~$ cat /home/sysadmin/snmp/metadata.yaml maintain_user_overrides: true app_name: snmp app_version: 1.0-1 helm_repo: stx-platform
Compare against the database (data is json formatted):
sysadmin@controller-0:~$ sudo -u postgres psql -d sysinv -c "select name,app_metadata from kube_app where name='snmp';" snmp | {"maintain_user_overrides": true, "app_name": "snmp", "app_version": "1.0-1", "helm_repo": "stx-platform"}
Example 2: platform-integ-apps
Extract the metadata.yaml from the application tarball:
sysadmin@controller-0:~$ mkdir /home/sysadmin/platform-integ-apps && tar -xvf /usr/local/share/applications/helm/platform-integ-apps-1.0-56.tgz -C /home/sysadmin/platform-integ-apps
Check the metadata.yaml content:
sysadmin@controller-0:~$ cat /home/sysadmin/platform-integ-apps/metadata.yaml app_name: platform-integ-apps app_version: 1.0-56 helm_repo: stx-platform maintain_user_overrides: true behavior: platform_managed_app: yes desired_state: applied evaluate_reapply: triggers: - type: runtime-apply-puppet - type: host-availability-updated - type: kube-upgrade-complete filters: - availability: services-enabled - type: host-delete filters: - personality: controller
Compare against the database (data is json formatted):
sysadmin@controller-0:~$ sudo -u postgres psql -d sysinv -c "select name,app_metadata from kube_app where name='platform-integ-apps'" platform-integ-apps | {"app_name": "platform-integ-apps", "app_version": "1.0-56", "helm_repo": "stx-platform", "maintain_user_overrides": true, "behavior": {"platform_managed_app": "yes", "desired_state": "applied", "evaluate_reapply": {"triggers": [{"type": "runtime-apply-puppet"}, {"type": "host-availability-updated"}, {"type": "kube-upgrade-complete", "filters": [{"availability": "services-enabled"}]}, {"type": "host-delete", "filters": [{"personality": "controller"}]}]}}}
Metadata Functional Tests
- Update app and check whether overrides are preserved when "maintain_user_overrides" is true
- Update app and check whether attributes are preserved when "maintain_attributes" is true
- Check if app was automatically updated when "auto_update" is set to true
- Check if application apply is triggered by automatic re-evaluation
Legacy Code Related Tests
Armada was replaced by Flux since stx-8 but Armada-based apps may still be around when updating from previous versions
- Update Armada app version to Flux app version. For instance during upgrades from stx-7 to stx-8.
- Make sure the upgrade-activate step is successful
Helm Release Tests
- Check if helm release version is being correctly updated on sysinv database
- Raise error when helm chart is invalid
Robustness Tests
- Flux pods:
- Check helm-controller and source-controller pods are running and ready
- Check heml-controller and source controller pods memory and cpu usage
- Test application upload/apply/update/remove/delete across helm-controller and source-controller Kubernetes deployments restarts
- Test application upload/apply/update/remove/delete during and after sysinv restarts
Example 1
Check if pods are running and ready
sysadmin@controller-0:~$ kubectl get pods -n flux-helm -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES helm-controller-7d89fdbc47-2z7j2 1/1 Running 0 5h14m 172.16.192.95 controller-0 <none> <none> source-controller-67f6d76b4b-jj8bb 1/1 Running 0 5h14m 172.16.192.96 controller-0 <none> <none>
Check pods CPU time. This is given in nanoseconds:
sysadmin@controller-0:~$ kubectl exec -it -n flux-helm helm-controller-7d89fdbc47-2z7j2 -- cat /sys/fs/cgroup/cpu/cpuacct.usage 96434038481
sysadmin@controller-0:~$ kubectl exec -it -n flux-helm source-controller-67f6d76b4b-jj8bb -- cat /sys/fs/cgroup/cpu/cpuacct.usage 60297801826
Check pods memory. Output converted to Megabytes:
sysadmin@controller-0:~$ kubectl exec -it -n flux-helm helm-controller-7d89fdbc47-2z7j2 -- cat /sys/fs/cgroup/memory/memory.usage_in_bytes | awk '{ foo = $1 / 1024 / 1024 ; print foo "MB" }' 34.8594MB
kubectl exec -it -n flux-helm source-controller-67f6d76b4b-jj8bb -- cat /sys/fs/cgroup/memory/memory.usage_in_bytes | awk '{ foo = $1 / 1024 / 1024 ; print foo "MB" }' 29.5742MB
Those values can be assessed in comparison to the pods requests and limits:
kubectl describe node controller-0
Check the "Non-terminated Pods" section:
Non-terminated Pods: (21 in total) Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age --------- ---- ------------ ---------- --------------- ------------- --- [...] flux-helm helm-controller-68775d8db6-9s5nh 0 (0%) 1 (50%) 64Mi (0%) 1Gi (5%) 83m flux-helm source-controller-64d9d88f74-lf9js 0 (0%) 1 (50%) 64Mi (0%) 1Gi (5%) 83m [..]
Example 2
Restart all Flux pods:
sysadmin@controller-0:~$ kubectl rollout restart deployment -n flux-helm deployment.apps/helm-controller restarted deployment.apps/source-controller restarted
Test application operations:
Upload application:
system application-upload system application-upload /usr/local/share/applications/helm/snmp-1.0-1.tgz
Apply application:
system application-apply snmp
Remove application:
system application-remove snmp
Delete application:
system application-delete snmp
Example 3
Restart all sysinv related services:
sysadmin@controller-0:~$ sudo systemctl restart sysinv-agent.service sysinv-api.service sysinv-conductor.service
Test application operations:
Upload application:
system application-upload /usr/local/share/applications/helm/platform-integ-apps-1.0-56.tgz
Apply/reapply application:
system application-apply platform-integ-apps
Remove application:
system application-remove platform-integ-apps
Delete application:
system application-delete platform-integ-apps