Compare commits

7 Commits

Author SHA1 Message Date
c689caea88 refactor(core): fix tests 2025-11-14 16:51:21 +01:00
8c20deb690 refactor(core): simplify core module 2025-11-14 16:42:35 +01:00
39979b51ee update report 2025-11-14 15:20:16 +01:00
da0c77101d Merge pull request #53 from dat515-2025/test_arm_build
Some checks failed
Deploy Prod / Run Python Tests (push) Has been cancelled
Deploy Prod / Build and push image (reusable) (push) Has been cancelled
Deploy Prod / Generate Production URLs (push) Has been cancelled
Deploy Prod / Frontend - Build and Deploy to Cloudflare Pages (prod) (push) Has been cancelled
Deploy Prod / Helm upgrade/install (prod) (push) Has been cancelled
build arm64 image
2025-11-14 00:58:33 +01:00
a5a83e5d07 update docs 2025-11-14 00:20:19 +01:00
3749aa4525 Also add amd64 2025-11-14 00:16:36 +01:00
94aa64addc build arm64 image 2025-11-14 00:03:06 +01:00
10 changed files with 120 additions and 60 deletions

View File

@@ -94,7 +94,7 @@ jobs:
tags: | tags: |
${{ env.IMAGE_REPO }}:${{ env.TAG1 }} ${{ env.IMAGE_REPO }}:${{ env.TAG1 }}
${{ env.IMAGE_REPO }}:${{ env.TAG2 }} ${{ env.IMAGE_REPO }}:${{ env.TAG2 }}
platforms: linux/amd64 platforms: linux/arm64,linux/amd64
- name: Set outputs - name: Set outputs
id: set id: set

View File

@@ -91,10 +91,15 @@ The workflow works in the following way:
## Prerequisites ## Prerequisites
Here are software and hardware prerequisites for the development and production environments. This section also
describes
necessary environment variables and key dependencies used in the project.
### System Requirements ### System Requirements
#### Development #### Development
- OS: Tested on MacOS, Linux and Windows should work as well
- Minimum RAM: 8 GB - Minimum RAM: 8 GB
- Storage: 10 GB+ free - Storage: 10 GB+ free
@@ -138,8 +143,8 @@ The workflow works in the following way:
#### Backend #### Backend
- `MOJEID_CLIENT_ID`, `MOJEID_CLIENT_SECRET` \- OAuth client ID and secret for - `MOJEID_CLIENT_ID`, `MOJEID_CLIENT_SECRET` \- OAuth client ID and secret for
[MojeID](https://www.mojeid.cz/en/provider/) [MojeID](https://www.mojeid.cz/en/provider/)
- `BANKID_CLIENT_ID`, `BANKID_CLIENT_SECRET` \- OAuth client ID and secret for [BankID](https://developer.bankid.cz/) - `BANKID_CLIENT_ID`, `BANKID_CLIENT_SECRET` \- OAuth client ID and secret for [BankID](https://developer.bankid.cz/)
- `CSAS_CLIENT_ID`, `CSAS_CLIENT_SECRET` \- OAuth client ID and secret for [Česká - `CSAS_CLIENT_ID`, `CSAS_CLIENT_SECRET` \- OAuth client ID and secret for [Česká
spořitelna](https://developers.erstegroup.com/docs/apis/bank.csas) spořitelna](https://developers.erstegroup.com/docs/apis/bank.csas)
- `DATABASE_URL`(or `MARIADB_HOST`, `MARIADB_PORT`, `MARIADB_DB`, `MARIADB_USER`, `MARIADB_PASSWORD`) \- MariaDB - `DATABASE_URL`(or `MARIADB_HOST`, `MARIADB_PORT`, `MARIADB_DB`, `MARIADB_USER`, `MARIADB_PASSWORD`) \- MariaDB
@@ -227,6 +232,9 @@ npm run dev
### Backend ### Backend
App is separated into backend and frontend so it also needs to be built separately. Backend is build into docker image
and frontend is deployed as static files.
```bash ```bash
cd 7project/backend cd 7project/backend
# Dont forget to set correct image tag with your registry and name # Dont forget to set correct image tag with your registry and name
@@ -244,6 +252,10 @@ npm run build
## Deployment Instructions ## Deployment Instructions
Deployment is tested on TalosOS cluster with 1 control plane and 4 workers, cluster needs to be setup and configured
manually. Terraform/OpenTofu is then used to deploy base services to the cluster. App itself is deployed automatically
via GitHub actions and Helm chart. Frontend files are deployed to Cloudflare pages.
### Setup Cluster ### Setup Cluster
Deployment should work on any Kubernetes cluster. However, we are using 4 TalosOS virtual machines (1 control plane, 3 Deployment should work on any Kubernetes cluster. However, we are using 4 TalosOS virtual machines (1 control plane, 3
@@ -427,6 +439,7 @@ See the workflow [here](../.github/workflows/run-tests.yml).
If you want to run the tests locally, the preferred way is to use a [bash script](backend/test_locally.sh) If you want to run the tests locally, the preferred way is to use a [bash script](backend/test_locally.sh)
that will start a test DB container with [docker compose](backend/docker-compose.test.yml) and remove it afterwards. that will start a test DB container with [docker compose](backend/docker-compose.test.yml) and remove it afterwards.
```bash ```bash
cd 7project/src/backend cd 7project/src/backend
bash test_locally.sh bash test_locally.sh
@@ -486,10 +499,12 @@ curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8000/authenticated-route
### Frontend ### Frontend
- Start with: - Start with:
```bash ```bash
npm run dev in 7project/src/frontend npm run dev in 7project/src/frontend
``` ```
- Ensure VITE_BACKEND_URL is set to the backend URL (e.g., http://127.0.0.1:8000) - Ensure VITE_BACKEND_URL is set to the backend URL (e.g., http://127.0.0.1:8000)
- Open http://localhost:5173 - Open http://localhost:5173
- Login, view latest transactions, filter, and add new transactions from the UI. - Login, view latest transactions, filter, and add new transactions from the UI.
@@ -511,20 +526,27 @@ npm run dev in 7project/src/frontend
## Troubleshooting ## Troubleshooting
running on arm
tofu apply error
### Common Issues ### Common Issues
#### Issue 1: [Common problem] #### Issue 1: Unable to apply Cloudflare terraform module
**Symptoms**: [What the user sees] **Symptoms**: Terraform/OpenTofu apply fails during Cloudflare module deployment.
**Solution**: [Step-by-step fix] This is caused by unknown variable not known beforehand.
#### Issue 2: [Another common problem] **Solution**: Apply first without Cloudflare module and then apply again.
**Symptoms**: [What the user sees] ```bash
**Solution**: [Step-by-step fix] tofu apply -exclude modules.cloudflare
tofu apply
```
#### Issue 2: Pods are unable to start
**Symptoms**: Pods are unable to start with ImagePullBackOff error. This could be caused
by either hitting docker hub rate limits or by docker hub being down.
**Solution**: Make sure you updated the cluster config to use registry mirror as described in
"Setup Cluster" section.
### Debug Commands ### Debug Commands
@@ -587,22 +609,22 @@ tofu apply error
### Dejan ### Dejan
| Date | Activity | Hours | Description | Representative Commit / PR | | Date | Activity | Hours | Description | Representative Commit / PR |
|:----------------|:-------------------------|:-------|:--------------------------------------------------------------|:---------------------------------------------------------| |:----------------|:---------------------|:-------|:--------------------------------------------------------------|:---------------------------------------------------------|
| 25.9. | Design | 2 | 6design | | | 25.9. | Design | 2 | 6design | |
| 9.10 to 11.10. | Backend APIs | 14 | Implemented Backend APIs | `PR #26`, `20-create-a-controller-layer-on-backend-side` | | 9.10 to 11.10. | Backend APIs | 14 | Implemented Backend APIs | `PR #26`, `20-create-a-controller-layer-on-backend-side` |
| 13.10 to 15.10. | Frontend Development | 8 | Created user interface mockups | `PR #28`, `frontend basics` | | 13.10 to 15.10. | Frontend Development | 8 | Created user interface mockups | `PR #28`, `frontend basics` |
| Continually | Documentation | 7 | Documenting the dev process | | | Continually | Documentation | 7 | Documenting the dev process | |
| 21.10 to 23.10 | Tests, frontend | 10 | Test basics, balance charts, and frontend improvement | `PR #31`, `30 create tests and set up a GitHub pipeline` | | 21.10 to 23.10 | Tests, frontend | 10 | Test basics, balance charts, and frontend improvement | `PR #31`, `30 create tests and set up a GitHub pipeline` |
| 28.10 to 30.10 | CI | 6 | Integrated tests with test database setup on github workflows | `PR #28`, `frontend basics` | | 28.10 to 30.10 | CI | 6 | Integrated tests with test database setup on github workflows | `PR #28`, `frontend basics` |
| 28.10 to 30.10 | Frontend | 8 | UI improvements and exchange rate API integration | `PR #28`, `frontend basics` | | 28.10 to 30.10 | Frontend | 8 | UI improvements and exchange rate API integration | `PR #28`, `frontend basics` |
| 4.11 to 6.11 | Tests | 6 | Test fixes improvement, more integration and e2e | `PR #28`, `frontend basics` | | 4.11 to 6.11 | Tests | 6 | Test fixes improvement, more integration and e2e | `PR #28`, `frontend basics` |
| 4.11 to 6.11 | Frontend | 6 | Fixes, Improved UI, added support for mobile devices | `PR #28`, `frontend basics` | | 4.11 to 6.11 | Frontend | 6 | Fixes, Improved UI, added support for mobile devices | `PR #28`, `frontend basics` |
| 11.11 | Backend APIs | 4 | Moved rates API, mock bank to Backend, few fixes | `PR #28`, `frontend basics` | | 11.11 | Backend APIs | 4 | Moved rates API, mock bank to Backend, few fixes | `PR #28`, `frontend basics` |
| 11.11 to 12.11 | Tests | 3 | Local testing DB container, few fixes | `PR #28`, `frontend basics` | | 11.11 to 12.11 | Tests | 3 | Local testing DB container, few fixes | `PR #28`, `frontend basics` |
| 12.11 | Frontend | 3 | Enabled multiple transaction edits at once, CSAS button state | `PR #28`, `frontend basics` | | 12.11 | Frontend | 3 | Enabled multiple transaction edits at once, CSAS button state | `PR #28`, `frontend basics` |
| 13.11 | Video | 3 | Video | | | 13.11 | Video | 3 | Video | |
| **Total** | | **80** | | | | **Total** | | **80** | | |
### Group Total: [XXX.X] hours ### Group Total: [XXX.X] hours
@@ -634,6 +656,11 @@ FastAPI lacks usable build in support for database migrations and implementing A
Tricky was also integrating FastAPI auth system with React frontend, since there is no official project template. Tricky was also integrating FastAPI auth system with React frontend, since there is no official project template.
Using .NET (which we considered initially) would probably solve these issues. Using .NET (which we considered initially) would probably solve these issues.
#### Private container registry
Using private container registry would allow us to include environment variables directly in the image during build.
This would simplify deployment and CI/CD setup.
[What would you do differently? What worked well that you'd keep?] [What would you do differently? What worked well that you'd keep?]
### Individual Growth ### Individual Growth
@@ -649,15 +676,11 @@ The biggest challenge for me was time tracking since I am used to tracking to pr
It was also interesting experience to be the one responsible for the initial project structure/design/setup It was also interesting experience to be the one responsible for the initial project structure/design/setup
used not only by myself. used not only by myself.
[Personal reflection on growth, challenges, and learning]
#### [Dejan] #### [Dejan]
Since I do not have a job, this project was probably the most complex one I have ever worked on.
Since I do not have a job, this project was probably the most complex one I have ever worked on.
It was also the first school project where I was encouraged to use AI. It was also the first school project where I was encouraged to use AI.
Lukas
[Personal reflection on growth, challenges, and learning]
--- ---

View File

@@ -1,6 +0,0 @@
import app.celery_app # noqa: F401
from app.workers.celery_tasks import send_email
def enqueue_email(to: str, subject: str, body: str) -> None:
send_email.delay(to, subject, body)

View File

@@ -49,4 +49,4 @@ def decode_and_verify_jwt(token: str, secret: str) -> dict:
secret, secret,
algorithms=["HS256"], algorithms=["HS256"],
options={"verify_aud": False}, options={"verify_aud": False},
) # verify_exp is True by default ) # verify_exp is True by default

View File

@@ -14,11 +14,10 @@ from httpx_oauth.oauth2 import BaseOAuth2
from app.models.user import User from app.models.user import User
from app.oauth.bank_id import BankID from app.oauth.bank_id import BankID
from app.oauth.csas import CSASOAuth from app.workers.celery_tasks import send_email
from app.oauth.custom_openid import CustomOpenID from app.oauth.custom_openid import CustomOpenID
from app.oauth.moje_id import MojeIDOAuth from app.oauth.moje_id import MojeIDOAuth
from app.services.db import get_user_db from app.services.db import get_user_db
from app.core.queue import enqueue_email
SECRET = os.getenv("SECRET", "CHANGE_ME_SECRET") SECRET = os.getenv("SECRET", "CHANGE_ME_SECRET")
@@ -87,7 +86,7 @@ class UserManager(UUIDIDMixin, BaseUserManager[User, uuid.UUID]):
"Pokud jsi registraci neprováděl(a), tento email ignoruj.\n" "Pokud jsi registraci neprováděl(a), tento email ignoruj.\n"
) )
try: try:
enqueue_email(to=user.email, subject=subject, body=body) send_email.delay(user.email, subject, body)
except Exception as e: except Exception as e:
print("[Email Fallback] To:", user.email) print("[Email Fallback] To:", user.email)
print("[Email Fallback] Subject:", subject) print("[Email Fallback] Subject:", subject)

View File

@@ -34,15 +34,16 @@ def test_authenticated_route_requires_auth(client):
async def test_on_after_request_verify_enqueues_email(monkeypatch): async def test_on_after_request_verify_enqueues_email(monkeypatch):
calls = {} calls = {}
def fake_enqueue_email(to: str, subject: str, body: str): class FakeCeleryTask:
calls.setdefault("emails", []).append({ def delay(to: str, subject: str, body: str):
"to": to, calls.setdefault("emails", []).append({
"subject": subject, "to": to,
"body": body, "subject": subject,
}) "body": body,
})
# Patch the enqueue_email used inside user_service # Patch the enqueue_email used inside user_service
monkeypatch.setattr(user_service, "enqueue_email", fake_enqueue_email) monkeypatch.setattr(user_service, "send_email", FakeCeleryTask)
class DummyUser: class DummyUser:
def __init__(self, email): def __init__(self, email):

View File

@@ -0,0 +1,30 @@
# Helm chart deployment
This directory contains a Helm chart for deploying the app to a cluster, it support bot production and preview
deployment.
## Directory Structure
- `myapp-chart/`
- `templates/`
- `app-deployment.yaml` - Kubernetes Deployment for the application
- `cron.yaml` - cronjob for periodic tasks - periodically calls app endpoint
- `database.yaml` - Creates database using MariaDB operator. Production database is kept, but preview/dev
database is dropped after uninstalling the chart.
- `database-grant.yaml` - Defines rights for the database user
- `database-user.yaml` - Creates database user
- `monitoring.yaml` - Adds /metrics endpoint to Prometheus scraping
- `prod.yaml` - Application secrets
- `rabbitmq-cluster.yaml` - Defines RabbitMQ cluster for this deployment
- `rabbitmq-permission.yalm` - Defines RabbitMQ user permissions
- `rabbitmq-queue.yaml` - Defines RabbitMQ queue
- `rabbitmq-user.yaml` - Defines RabbitMQ user
- `rabbitmq-user-secret.yaml` - Defines RabbitMQ user secret
- `service.yaml` - Kubernetes Service for the application
- `tunnel.yaml` - Cloudflare tunnel for accessing the application¨
- `worker-deployment.yaml` - Kubernetes Deployment for the Celery worker, uses same image as the app-deployment,
but with different entrypoint
- `Chart.yaml` - Helm chart metadata
- `values.yaml` - list of all configurable values
- `values-dev.yaml` - default values for development/preview deployment
- `values-prod.yaml` - default values for production deployment

View File

@@ -0,0 +1,20 @@
# Terraform/OpenTofu IaC
This directory contains infrastructure configuration files and modules for underlying services.
## Directory Structure
- `modules/` - separated modules for different services
- `cert-manager/` - module for deploying Cert-Manager
- `cloudflare/` - module for Cloudflare tunnels operator - allows to expose services via Cloudflare tunnels
- `maxscale/` - module for MariaDB Operator set up for MaxScale and 3 nodes, backups and phpMyAdmin. PHPMyAdmin is
available at mysql.YOU_DOMAIN, MaxScale UI at maxscale.YOU_DOMAIN
- `metallb/` - module for MetalLB load balancer
- `metrics-server/` - module for Kubernetes Metrics Server
- `prometheus/` - module for Prometheus and Grafana - monitoring stack with grafana available at grafana.YOU_DOMAIN
- `rabbitmq/` - module for RabbitMQ message queue cluster
- `redis/` - module for Redis cluster, not used currently
- `storage/` - module for Longhorn file storage
- `main.tf` - main deployment configuration
- `variables.tf` - deployment variables
- `terraform.tfvars.example` - example variables file

View File

@@ -31,5 +31,5 @@ resource "helm_release" "longhorn" {
chart = "longhorn" chart = "longhorn"
namespace = "longhorn-system" namespace = "longhorn-system"
version = "1.9.1" version = "1.9.1"
timeout = 3600 timeout = 300
} }

View File

@@ -95,13 +95,6 @@ variable "cloudflare_account_id" {
nullable = false nullable = false
} }
variable "argocd_admin_password" {
type = string
nullable = false
sensitive = true
description = "ArgoCD admin password"
}
variable "rabbitmq-password" { variable "rabbitmq-password" {
type = string type = string
nullable = false nullable = false