From a5a83e5d07f0fe29d7cbd19e4bcf38af5e4c0e99 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Luk=C3=A1=C5=A1=20Trkan?= Date: Fri, 14 Nov 2025 00:20:19 +0100 Subject: [PATCH] update docs --- 7project/report.md | 86 ++++++++++++++--------- 7project/src/charts/README.md | 30 ++++++++ 7project/src/tofu/README.md | 20 ++++++ 7project/src/tofu/modules/storage/main.tf | 2 +- 7project/src/tofu/variables.tf | 7 -- 5 files changed, 103 insertions(+), 42 deletions(-) create mode 100644 7project/src/charts/README.md create mode 100644 7project/src/tofu/README.md diff --git a/7project/report.md b/7project/report.md index eaf56fb..2e0cd94 100644 --- a/7project/report.md +++ b/7project/report.md @@ -91,10 +91,15 @@ The workflow works in the following way: ## Prerequisites +Here are software and hardware prerequisites for the development and production environments. This section also +describes +necessary environment variables and key dependencies used in the project. + ### System Requirements #### Development +- OS: Tested on MacOS, Linux and Windows should work as well - Minimum RAM: 8 GB - Storage: 10 GB+ free @@ -138,8 +143,8 @@ The workflow works in the following way: #### Backend - `MOJEID_CLIENT_ID`, `MOJEID_CLIENT_SECRET` \- OAuth client ID and secret for - [MojeID](https://www.mojeid.cz/en/provider/) -- `BANKID_CLIENT_ID`, `BANKID_CLIENT_SECRET` \- OAuth client ID and secret for [BankID](https://developer.bankid.cz/) + [MojeID](https://www.mojeid.cz/en/provider/) +- `BANKID_CLIENT_ID`, `BANKID_CLIENT_SECRET` \- OAuth client ID and secret for [BankID](https://developer.bankid.cz/) - `CSAS_CLIENT_ID`, `CSAS_CLIENT_SECRET` \- OAuth client ID and secret for [Česká spořitelna](https://developers.erstegroup.com/docs/apis/bank.csas) - `DATABASE_URL`(or `MARIADB_HOST`, `MARIADB_PORT`, `MARIADB_DB`, `MARIADB_USER`, `MARIADB_PASSWORD`) \- MariaDB @@ -227,6 +232,9 @@ npm run dev ### Backend +App is separated into backend and frontend so it also needs to be built separately. Backend is build into docker image +and frontend is deployed as static files. + ```bash cd 7project/backend # Dont forget to set correct image tag with your registry and name @@ -244,6 +252,10 @@ npm run build ## Deployment Instructions +Deployment is tested on TalosOS cluster with 1 control plane and 4 workers, cluster needs to be setup and configured +manually. Terraform/OpenTofu is then used to deploy base services to the cluster. App itself is deployed automatically +via GitHub actions and Helm chart. Frontend files are deployed to Cloudflare pages. + ### Setup Cluster Deployment should work on any Kubernetes cluster. However, we are using 4 TalosOS virtual machines (1 control plane, 3 @@ -427,6 +439,7 @@ See the workflow [here](../.github/workflows/run-tests.yml). If you want to run the tests locally, the preferred way is to use a [bash script](backend/test_locally.sh) that will start a test DB container with [docker compose](backend/docker-compose.test.yml) and remove it afterwards. + ```bash cd 7project/src/backend bash test_locally.sh @@ -486,10 +499,12 @@ curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8000/authenticated-route ### Frontend -- Start with: +- Start with: + ```bash npm run dev in 7project/src/frontend ``` + - Ensure VITE_BACKEND_URL is set to the backend URL (e.g., http://127.0.0.1:8000) - Open http://localhost:5173 - Login, view latest transactions, filter, and add new transactions from the UI. @@ -511,20 +526,27 @@ npm run dev in 7project/src/frontend ## Troubleshooting -running on arm -tofu apply error - ### Common Issues -#### Issue 1: [Common problem] +#### Issue 1: Unable to apply Cloudflare terraform module -**Symptoms**: [What the user sees] -**Solution**: [Step-by-step fix] +**Symptoms**: Terraform/OpenTofu apply fails during Cloudflare module deployment. +This is caused by unknown variable not known beforehand. -#### Issue 2: [Another common problem] +**Solution**: Apply first without Cloudflare module and then apply again. -**Symptoms**: [What the user sees] -**Solution**: [Step-by-step fix] +```bash +tofu apply -exclude modules.cloudflare +tofu apply +``` + +#### Issue 2: Pods are unable to start + +**Symptoms**: Pods are unable to start with ImagePullBackOff error. This could be caused +by either hitting docker hub rate limits or by docker hub being down. + +**Solution**: Make sure you updated the cluster config to use registry mirror as described in +"Setup Cluster" section. ### Debug Commands @@ -587,22 +609,22 @@ tofu apply error ### Dejan -| Date | Activity | Hours | Description | Representative Commit / PR | -|:----------------|:-------------------------|:-------|:--------------------------------------------------------------|:---------------------------------------------------------| -| 25.9. | Design | 2 | 6design | | -| 9.10 to 11.10. | Backend APIs | 14 | Implemented Backend APIs | `PR #26`, `20-create-a-controller-layer-on-backend-side` | -| 13.10 to 15.10. | Frontend Development | 8 | Created user interface mockups | `PR #28`, `frontend basics` | -| Continually | Documentation | 7 | Documenting the dev process | | -| 21.10 to 23.10 | Tests, frontend | 10 | Test basics, balance charts, and frontend improvement | `PR #31`, `30 create tests and set up a GitHub pipeline` | -| 28.10 to 30.10 | CI | 6 | Integrated tests with test database setup on github workflows | `PR #28`, `frontend basics` | -| 28.10 to 30.10 | Frontend | 8 | UI improvements and exchange rate API integration | `PR #28`, `frontend basics` | -| 4.11 to 6.11 | Tests | 6 | Test fixes improvement, more integration and e2e | `PR #28`, `frontend basics` | -| 4.11 to 6.11 | Frontend | 6 | Fixes, Improved UI, added support for mobile devices | `PR #28`, `frontend basics` | -| 11.11 | Backend APIs | 4 | Moved rates API, mock bank to Backend, few fixes | `PR #28`, `frontend basics` | -| 11.11 to 12.11 | Tests | 3 | Local testing DB container, few fixes | `PR #28`, `frontend basics` | -| 12.11 | Frontend | 3 | Enabled multiple transaction edits at once, CSAS button state | `PR #28`, `frontend basics` | -| 13.11 | Video | 3 | Video | | -| **Total** | | **80** | | | +| Date | Activity | Hours | Description | Representative Commit / PR | +|:----------------|:---------------------|:-------|:--------------------------------------------------------------|:---------------------------------------------------------| +| 25.9. | Design | 2 | 6design | | +| 9.10 to 11.10. | Backend APIs | 14 | Implemented Backend APIs | `PR #26`, `20-create-a-controller-layer-on-backend-side` | +| 13.10 to 15.10. | Frontend Development | 8 | Created user interface mockups | `PR #28`, `frontend basics` | +| Continually | Documentation | 7 | Documenting the dev process | | +| 21.10 to 23.10 | Tests, frontend | 10 | Test basics, balance charts, and frontend improvement | `PR #31`, `30 create tests and set up a GitHub pipeline` | +| 28.10 to 30.10 | CI | 6 | Integrated tests with test database setup on github workflows | `PR #28`, `frontend basics` | +| 28.10 to 30.10 | Frontend | 8 | UI improvements and exchange rate API integration | `PR #28`, `frontend basics` | +| 4.11 to 6.11 | Tests | 6 | Test fixes improvement, more integration and e2e | `PR #28`, `frontend basics` | +| 4.11 to 6.11 | Frontend | 6 | Fixes, Improved UI, added support for mobile devices | `PR #28`, `frontend basics` | +| 11.11 | Backend APIs | 4 | Moved rates API, mock bank to Backend, few fixes | `PR #28`, `frontend basics` | +| 11.11 to 12.11 | Tests | 3 | Local testing DB container, few fixes | `PR #28`, `frontend basics` | +| 12.11 | Frontend | 3 | Enabled multiple transaction edits at once, CSAS button state | `PR #28`, `frontend basics` | +| 13.11 | Video | 3 | Video | | +| **Total** | | **80** | | | ### Group Total: [XXX.X] hours @@ -649,15 +671,11 @@ The biggest challenge for me was time tracking since I am used to tracking to pr It was also interesting experience to be the one responsible for the initial project structure/design/setup used not only by myself. -[Personal reflection on growth, challenges, and learning] - #### [Dejan] -Since I do not have a job, this project was probably the most complex one I have ever worked on. + +Since I do not have a job, this project was probably the most complex one I have ever worked on. It was also the first school project where I was encouraged to use AI. -Lukas - -[Personal reflection on growth, challenges, and learning] --- diff --git a/7project/src/charts/README.md b/7project/src/charts/README.md new file mode 100644 index 0000000..3c7d7d5 --- /dev/null +++ b/7project/src/charts/README.md @@ -0,0 +1,30 @@ +# Helm chart deployment + +This directory contains a Helm chart for deploying the app to a cluster, it support bot production and preview +deployment. + +## Directory Structure + +- `myapp-chart/` + - `templates/` + - `app-deployment.yaml` - Kubernetes Deployment for the application + - `cron.yaml` - cronjob for periodic tasks - periodically calls app endpoint + - `database.yaml` - Creates database using MariaDB operator. Production database is kept, but preview/dev + database is dropped after uninstalling the chart. + - `database-grant.yaml` - Defines rights for the database user + - `database-user.yaml` - Creates database user + - `monitoring.yaml` - Adds /metrics endpoint to Prometheus scraping + - `prod.yaml` - Application secrets + - `rabbitmq-cluster.yaml` - Defines RabbitMQ cluster for this deployment + - `rabbitmq-permission.yalm` - Defines RabbitMQ user permissions + - `rabbitmq-queue.yaml` - Defines RabbitMQ queue + - `rabbitmq-user.yaml` - Defines RabbitMQ user + - `rabbitmq-user-secret.yaml` - Defines RabbitMQ user secret + - `service.yaml` - Kubernetes Service for the application + - `tunnel.yaml` - Cloudflare tunnel for accessing the application¨ + - `worker-deployment.yaml` - Kubernetes Deployment for the Celery worker, uses same image as the app-deployment, + but with different entrypoint + - `Chart.yaml` - Helm chart metadata + - `values.yaml` - list of all configurable values + - `values-dev.yaml` - default values for development/preview deployment + - `values-prod.yaml` - default values for production deployment \ No newline at end of file diff --git a/7project/src/tofu/README.md b/7project/src/tofu/README.md new file mode 100644 index 0000000..bfd57e6 --- /dev/null +++ b/7project/src/tofu/README.md @@ -0,0 +1,20 @@ +# Terraform/OpenTofu IaC + +This directory contains infrastructure configuration files and modules for underlying services. + +## Directory Structure + +- `modules/` - separated modules for different services + - `cert-manager/` - module for deploying Cert-Manager + - `cloudflare/` - module for Cloudflare tunnels operator - allows to expose services via Cloudflare tunnels + - `maxscale/` - module for MariaDB Operator set up for MaxScale and 3 nodes, backups and phpMyAdmin. PHPMyAdmin is + available at mysql.YOU_DOMAIN, MaxScale UI at maxscale.YOU_DOMAIN + - `metallb/` - module for MetalLB load balancer + - `metrics-server/` - module for Kubernetes Metrics Server + - `prometheus/` - module for Prometheus and Grafana - monitoring stack with grafana available at grafana.YOU_DOMAIN + - `rabbitmq/` - module for RabbitMQ message queue cluster + - `redis/` - module for Redis cluster, not used currently + - `storage/` - module for Longhorn file storage +- `main.tf` - main deployment configuration +- `variables.tf` - deployment variables +- `terraform.tfvars.example` - example variables file \ No newline at end of file diff --git a/7project/src/tofu/modules/storage/main.tf b/7project/src/tofu/modules/storage/main.tf index db69b92..c48ea7c 100644 --- a/7project/src/tofu/modules/storage/main.tf +++ b/7project/src/tofu/modules/storage/main.tf @@ -31,5 +31,5 @@ resource "helm_release" "longhorn" { chart = "longhorn" namespace = "longhorn-system" version = "1.9.1" - timeout = 3600 + timeout = 300 } diff --git a/7project/src/tofu/variables.tf b/7project/src/tofu/variables.tf index fe42b76..0cce4db 100644 --- a/7project/src/tofu/variables.tf +++ b/7project/src/tofu/variables.tf @@ -95,13 +95,6 @@ variable "cloudflare_account_id" { nullable = false } -variable "argocd_admin_password" { - type = string - nullable = false - sensitive = true - description = "ArgoCD admin password" -} - variable "rabbitmq-password" { type = string nullable = false