Compare commits

...

14 Commits

Author SHA1 Message Date
da0c77101d Merge pull request #53 from dat515-2025/test_arm_build
Some checks failed
Deploy Prod / Run Python Tests (push) Has been cancelled
Deploy Prod / Build and push image (reusable) (push) Has been cancelled
Deploy Prod / Generate Production URLs (push) Has been cancelled
Deploy Prod / Frontend - Build and Deploy to Cloudflare Pages (prod) (push) Has been cancelled
Deploy Prod / Helm upgrade/install (prod) (push) Has been cancelled
build arm64 image
2025-11-14 00:58:33 +01:00
a5a83e5d07 update docs 2025-11-14 00:20:19 +01:00
3749aa4525 Also add amd64 2025-11-14 00:16:36 +01:00
94aa64addc build arm64 image 2025-11-14 00:03:06 +01:00
ba1677b2d3 add README.md
Some checks are pending
Deploy Prod / Run Python Tests (push) Waiting to run
Deploy Prod / Build and push image (reusable) (push) Blocked by required conditions
Deploy Prod / Generate Production URLs (push) Blocked by required conditions
Deploy Prod / Frontend - Build and Deploy to Cloudflare Pages (prod) (push) Blocked by required conditions
Deploy Prod / Helm upgrade/install (prod) (push) Blocked by required conditions
2025-11-13 15:50:19 +01:00
ribardej
8ea1ef9eea Merge remote-tracking branch 'origin/main' 2025-11-13 14:33:50 +01:00
ribardej
4b614902b2 feat(docs): report.md update 2025-11-13 14:33:42 +01:00
a152ecbe4d fix main.py 2025-11-13 14:30:31 +01:00
7d7dd98d0f Merge remote-tracking branch 'origin/main' 2025-11-13 14:16:30 +01:00
5aca071ac2 update report 2025-11-13 14:16:21 +01:00
ribardej
80991c7390 Merge remote-tracking branch 'origin/main' 2025-11-13 14:09:04 +01:00
ribardej
1403e0029b feat(docs): report.md update 2025-11-13 14:08:52 +01:00
aa63e51e6a update report 2025-11-13 14:06:35 +01:00
Dejan Ribarovski
4aaaba3956 Merge pull request #52 from dat515-2025/51-refactor-project-structure
feat(docs): codebase refactor - added src directory
2025-11-13 13:58:24 +01:00
11 changed files with 209 additions and 152 deletions

View File

@@ -94,7 +94,7 @@ jobs:
tags: |
${{ env.IMAGE_REPO }}:${{ env.TAG1 }}
${{ env.IMAGE_REPO }}:${{ env.TAG2 }}
platforms: linux/amd64
platforms: linux/arm64,linux/amd64
- name: Set outputs
id: set

View File

@@ -1,43 +1,6 @@
# Lab 6: Design Document for Course Project
| Lab 6: | Design Document for Course Project |
| ----------- | ---------------------------------- |
| Subject: | DAT515 Cloud Computing |
| Deadline: | **September 19, 2025 23:59** |
| Grading: | No Grade |
| Submission: | Group |
## Table of Contents
- [Table of Contents](#table-of-contents)
- [1. Design Document (design.md)](#1-design-document-designmd)
The design document is the first deliverable for your project.
We separated this out as a separate deliverable, with its own deadline, to ensure that you have a clear plan before you start coding.
This part only needs a cursory review by the teaching staff to ensure it is sufficiently comprehensive, while still realistic.
The teaching staff will assign you to a project mentor who will provide guidance and support throughout the development process.
## 1. Design Document (design.md)
You are required to prepare a design document for your application.
The design doc should be brief, well-organized and easy to understand.
The design doc should be prepared in markdown format and named `design.md` and submitted in the project group's repository.
Remember that you can use [mermaid diagrams](https://github.com/mermaid-js/mermaid#readme) in markdown files.
The design doc **should include** the following sections:
- **Overview**: A brief description of the application and its purpose.
- **Architecture**: The high-level architecture of the application, including components, interactions, and data flow.
- **Technologies**: The cloud computing technologies or services used in the application.
- **Deployment**: The deployment strategy for the application, including any infrastructure requirements.
The design document should be updated throughout the development process and reflect the final implementation of your project.
Optional sections may include:
- Security: The security measures implemented in the application to protect data and resources.
- Scalability: The scalability considerations for the application, including load balancing and auto-scaling.
- Monitoring: The monitoring and logging strategy for the application to track performance and detect issues.
- Disaster Recovery: The disaster recovery plan for the application to ensure business continuity in case of failures.
- Cost Analysis: The cost analysis of running the application on the cloud, including pricing models and cost-saving strategies.
- References: Any external sources or references used in the design document.
# Personal Finance Tracker
## Folder Structure
- meetings: Contains note from meetings
- scr: Source code for the project
- checklist: Project checklist and self assessment tracking
- report.md: Detailed report of the project

View File

@@ -7,64 +7,64 @@ Focus on areas that align with your project goals and interests.
The core deliverables are required.
This means that you must get at least 2 points for each item in this category.
| **Category** | **Item** | **Max Points** | **Points** |
|----------------------------------| --------------------------------------- | -------------- |-------------------------------------------------|
| **Core Deliverables (Required)** | | | |
| Codebase & Organization | Well-organized project structure | 5 | 5 |
| | Clean, readable code | 5 | 4 |
| | Use planning tool (e.g., GitHub issues) | 5 | 4 |
| | Proper version control usage | 5 | 5 |
| 23 | Complete source code | 5 | 5 |
| Documentation | Comprehensive reproducibility report | 10 | 4-5 |
| | Updated design document | 5 | 2 |
| | Clear build/deployment instructions | 5 | 2 |
| | Troubleshooting guide | 5 | 1 |
| | Completed self-assessment table | 5 | 2 |
| 14 | Hour sheets for all members | 5 | 3 |
| Presentation Video | Project demonstration | 5 | 0 |
| | Code walk-through | 5 | 0 |
| 0 | Deployment showcase | 5 | 0 |
| **Technical Implementation** | | | |
| Application Functionality | Basic functionality works | 10 | 8 |
| | Advanced features implemented | 10 | 0 |
| | Error handling & robustness | 10 | 4 |
| 16 | User-friendly interface | 5 | 4 |
| Backend & Architecture | Stateless web server | 5 | 5 |
| | Stateful application | 10 | ? WHAT DOES THIS MEAN |
| | Database integration | 10 | 10 |
| | API design | 5 | 5 |
| 20 | Microservices architecture | 10 | 0 |
| Cloud Integration | Basic cloud deployment | 10 | 10 |
| | Cloud APIs usage | 10 | ? WHAT DOES THIS MEAN |
| | Serverless components | 10 | 0 |
| 10 | Advanced cloud services | 5 | 0 |
| **DevOps & Deployment** | | | |
| Containerization | Basic Dockerfile | 5 | 5 |
| | Optimized Dockerfile | 5 | 0 |
| | Docker Compose | 5 | 5 - dev only |
| 15 | Persistent storage | 5 | 5 |
| Deployment & Scaling | Manual deployment | 5 | 5 |
| | Automated deployment | 5 | 5 |
| | Multiple replicas | 5 | 5 |
| 20 | Kubernetes deployment | 10 | 10 |
| **Quality Assurance** | | | |
| Testing | Unit tests | 5 | 2 |
| | Integration tests | 5 | 2 |
| | End-to-end tests | 5 | 5 |
| 9 | Performance testing | 5 | 0 |
| Monitoring & Operations | Health checks | 5 | 5 |
| | Logging | 5 | 2 - only to terminal add logstash |
| 9 | Metrics/Monitoring | 5 | 2 - only DB, need to create Prometheus endpoint |
| Security | HTTPS/TLS | 5 | 5 |
| | Authentication | 5 | 5 |
| 15 | Authorization | 5 | 5 |
| **Innovation & Excellence** | | | |
| Advanced Features and | AI/ML Integration | 10 | 0 |
| Technical Excellence | Real-time features | 10 | 0 |
| | Creative problem solving | 10 | ? |
| | Performance optimization | 5 | 2 |
| 2 | Exceptional user experience | 5 | 0 |
| **Total** | | **255** | **153** |
| **Category** | **Item** | **Max Points** | **Points** | **Comments** |
|----------------------------------| --------------------------------------- | -------------- |-------------------------------------------------| |
| **Core Deliverables (Required)** | | | | |
| Codebase & Organization | Well-organized project structure | 5 | 5 | |
| | Clean, readable code | 5 | 4 | |
| | Use planning tool (e.g., GitHub issues) | 5 | 4 | |
| | Proper version control usage | 5 | 5 | |
| 23 | Complete source code | 5 | 5 | |
| Documentation | Comprehensive reproducibility report | 10 | 4-5 | |
| | Updated design document | 5 | 2 | |
| | Clear build/deployment instructions | 5 | 2 | |
| | Troubleshooting guide | 5 | 1 | |
| | Completed self-assessment table | 5 | 2 | |
| 14 | Hour sheets for all members | 5 | 3 | |
| Presentation Video | Project demonstration | 5 | 0 | |
| | Code walk-through | 5 | 0 | |
| 0 | Deployment showcase | 5 | 0 | |
| **Technical Implementation** | | | | |
| Application Functionality | Basic functionality works | 10 | 8 | |
| | Advanced features implemented | 10 | 0 | |
| | Error handling & robustness | 10 | 4 | |
| 16 | User-friendly interface | 5 | 4 | |
| Backend & Architecture | Stateless web server | 5 | 5 | |
| | Stateful application | 10 | ? WHAT DOES THIS MEAN | |
| | Database integration | 10 | 10 | |
| | API design | 5 | 5 | |
| 20 | Microservices architecture | 10 | 0 | |
| Cloud Integration | Basic cloud deployment | 10 | 10 | |
| | Cloud APIs usage | 10 | ? WHAT DOES THIS MEAN | |
| | Serverless components | 10 | 0 | |
| 10 | Advanced cloud services | 5 | 0 | |
| **DevOps & Deployment** | | | | |
| Containerization | Basic Dockerfile | 5 | 5 | |
| | Optimized Dockerfile | 5 | 0 | |
| | Docker Compose | 5 | 5 - dev only | |
| 15 | Persistent storage | 5 | 5 | |
| Deployment & Scaling | Manual deployment | 5 | 5 | |
| | Automated deployment | 5 | 5 | |
| | Multiple replicas | 5 | 5 | |
| 20 | Kubernetes deployment | 10 | 10 | |
| **Quality Assurance** | | | | |
| Testing | Unit tests | 5 | 2 | |
| | Integration tests | 5 | 2 | |
| | End-to-end tests | 5 | 5 | |
| 9 | Performance testing | 5 | 0 | |
| Monitoring & Operations | Health checks | 5 | 5 | |
| | Logging | 5 | 2 - only to terminal add logstash | |
| 9 | Metrics/Monitoring | 5 | 2 - only DB, need to create Prometheus endpoint | |
| Security | HTTPS/TLS | 5 | 5 | |
| | Authentication | 5 | 5 | |
| 15 | Authorization | 5 | 5 | |
| **Innovation & Excellence** | | | | |
| Advanced Features and | AI/ML Integration | 10 | 0 | |
| Technical Excellence | Real-time features | 10 | 0 | |
| | Creative problem solving | 10 | ? | |
| | Performance optimization | 5 | 2 | |
| 2 | Exceptional user experience | 5 | 0 | |
| **Total** | | **255** | **153** | |
## Grading Scale

View File

@@ -9,6 +9,8 @@
**Project Name**: Personal Finance Tracker
**Deployment URL**: https://finance.ltrk.cz/
**Group Members**:
- 289229, Lukáš Trkan, lukastrkan
@@ -89,10 +91,15 @@ The workflow works in the following way:
## Prerequisites
Here are software and hardware prerequisites for the development and production environments. This section also
describes
necessary environment variables and key dependencies used in the project.
### System Requirements
#### Development
- OS: Tested on MacOS, Linux and Windows should work as well
- Minimum RAM: 8 GB
- Storage: 10 GB+ free
@@ -136,10 +143,10 @@ The workflow works in the following way:
#### Backend
- `MOJEID_CLIENT_ID`, `MOJEID_CLIENT_SECRET` \- OAuth client ID and secret for
MojeID - https://www.mojeid.cz/en/provider/
- `BANKID_CLIENT_ID`, `BANKID_CLIENT_SECRET` \- OAuth client ID and secret for BankID - https://developer.bankid.cz/
- `CSAS_CLIENT_ID`, `CSAS_CLIENT_SECRET` \- OAuth client ID and secret for Česká
spořitelna - https://developers.erstegroup.com/docs/apis/bank.csas
[MojeID](https://www.mojeid.cz/en/provider/)
- `BANKID_CLIENT_ID`, `BANKID_CLIENT_SECRET` \- OAuth client ID and secret for [BankID](https://developer.bankid.cz/)
- `CSAS_CLIENT_ID`, `CSAS_CLIENT_SECRET` \- OAuth client ID and secret for [Česká
spořitelna](https://developers.erstegroup.com/docs/apis/bank.csas)
- `DATABASE_URL`(or `MARIADB_HOST`, `MARIADB_PORT`, `MARIADB_DB`, `MARIADB_USER`, `MARIADB_PASSWORD`) \- MariaDB
connection details
- `RABBITMQ_USERNAME`, `RABBITMQ_PASSWORD` \- credentials for RabbitMQ
@@ -166,7 +173,7 @@ You can run the project with Docker Compose and Python virtual environment for t
```bash
git clone https://github.com/dat515-2025/Group-8.git
cd Group-8/7project/src
cd Group-8/7project
```
### 2) Install dependencies
@@ -205,7 +212,7 @@ uvicorn app.app:fastApi --reload --host 0.0.0.0 --port 8000
### 6) Run Celery worker (optional, in another terminal)
```bash
cd Group-8/7project/backend
cd Group-8/7project/src/backend
source .venv/bin/activate
celery -A app.celery_app.celery_app worker -l info
```
@@ -225,6 +232,9 @@ npm run dev
### Backend
App is separated into backend and frontend so it also needs to be built separately. Backend is build into docker image
and frontend is deployed as static files.
```bash
cd 7project/backend
# Dont forget to set correct image tag with your registry and name
@@ -235,13 +245,17 @@ docker buildx build --platform linux/amd64,linux/arm64 -t CHANGE_ME --push .
### Frontend
```bash
cd project7/frontend
cd project7/src/frontend
npm ci
npm run build
```
## Deployment Instructions
Deployment is tested on TalosOS cluster with 1 control plane and 4 workers, cluster needs to be setup and configured
manually. Terraform/OpenTofu is then used to deploy base services to the cluster. App itself is deployed automatically
via GitHub actions and Helm chart. Frontend files are deployed to Cloudflare pages.
### Setup Cluster
Deployment should work on any Kubernetes cluster. However, we are using 4 TalosOS virtual machines (1 control plane, 3
@@ -254,7 +268,7 @@ running on top of Proxmox VE.
4) Navigate to tofu directory
```bash
cd 7project/tofu
cd 7project/src/tofu
````
5) Set IP addresses in environment variables
@@ -423,16 +437,17 @@ The tests are located in 7project/backend/tests directory. All tests are run by
push to main.
See the workflow [here](../.github/workflows/run-tests.yml).
If you want to run the tests locally, the preferred way is to use a [bash script](src/backend/test_locally.sh)
that will start a test DB container with [docker compose](src/backend/docker-compose.test.yml) and remove it afterwards.
If you want to run the tests locally, the preferred way is to use a [bash script](backend/test_locally.sh)
that will start a test DB container with [docker compose](backend/docker-compose.test.yml) and remove it afterwards.
```bash
cd 7project/backend
cd 7project/src/backend
bash test_locally.sh
```
### Unit Tests
There are 5 basic unit tests, since our services logic is very simple
There are only 5 basic unit tests, since our services logic is very simple
```bash
bash test_locally.sh --only-unit
@@ -484,7 +499,12 @@ curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8000/authenticated-route
### Frontend
- Start with: npm run dev in 7project/frontend
- Start with:
```bash
npm run dev in 7project/src/frontend
```
- Ensure VITE_BACKEND_URL is set to the backend URL (e.g., http://127.0.0.1:8000)
- Open http://localhost:5173
- Login, view latest transactions, filter, and add new transactions from the UI.
@@ -508,15 +528,25 @@ curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8000/authenticated-route
### Common Issues
#### Issue 1: [Common problem]
#### Issue 1: Unable to apply Cloudflare terraform module
**Symptoms**: [What the user sees]
**Solution**: [Step-by-step fix]
**Symptoms**: Terraform/OpenTofu apply fails during Cloudflare module deployment.
This is caused by unknown variable not known beforehand.
#### Issue 2: [Another common problem]
**Solution**: Apply first without Cloudflare module and then apply again.
**Symptoms**: [What the user sees]
**Solution**: [Step-by-step fix]
```bash
tofu apply -exclude modules.cloudflare
tofu apply
```
#### Issue 2: Pods are unable to start
**Symptoms**: Pods are unable to start with ImagePullBackOff error. This could be caused
by either hitting docker hub rate limits or by docker hub being down.
**Solution**: Make sure you updated the cluster config to use registry mirror as described in
"Setup Cluster" section.
### Debug Commands
@@ -580,11 +610,11 @@ curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8000/authenticated-route
### Dejan
| Date | Activity | Hours | Description | Representative Commit / PR |
|:----------------|:-------------------------|:-------|:--------------------------------------------------------------|:---------------------------------------------------------|
|:----------------|:---------------------|:-------|:--------------------------------------------------------------|:---------------------------------------------------------|
| 25.9. | Design | 2 | 6design | |
| 9.10 to 11.10. | Backend APIs | 14 | Implemented Backend APIs | `PR #26`, `20-create-a-controller-layer-on-backend-side` |
| 13.10 to 15.10. | Frontend Development | 8 | Created user interface mockups | `PR #28`, `frontend basics` |
| Continually | Documentation | 8 | Documenting the dev process | |
| Continually | Documentation | 7 | Documenting the dev process | |
| 21.10 to 23.10 | Tests, frontend | 10 | Test basics, balance charts, and frontend improvement | `PR #31`, `30 create tests and set up a GitHub pipeline` |
| 28.10 to 30.10 | CI | 6 | Integrated tests with test database setup on github workflows | `PR #28`, `frontend basics` |
| 28.10 to 30.10 | Frontend | 8 | UI improvements and exchange rate API integration | `PR #28`, `frontend basics` |
@@ -594,7 +624,7 @@ curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8000/authenticated-route
| 11.11 to 12.11 | Tests | 3 | Local testing DB container, few fixes | `PR #28`, `frontend basics` |
| 12.11 | Frontend | 3 | Enabled multiple transaction edits at once, CSAS button state | `PR #28`, `frontend basics` |
| 13.11 | Video | 3 | Video | |
| **Total** | | **81** | | |
| **Total** | | **80** | | |
### Group Total: [XXX.X] hours
@@ -641,15 +671,11 @@ The biggest challenge for me was time tracking since I am used to tracking to pr
It was also interesting experience to be the one responsible for the initial project structure/design/setup
used not only by myself.
[Personal reflection on growth, challenges, and learning]
#### [Dejan]
Since I do not have a job, this project was probably the most complex one I have ever worked on.
It was also the first school project where I was encouraged to use AI.
Lukas
[Personal reflection on growth, challenges, and learning]
---

23
7project/src/README.md Normal file
View File

@@ -0,0 +1,23 @@
## Folder structure
- `src/`
- `backend/`
- `alembic/` - database migrations
- `app/` - main application code
- `tests/` - tests
- `docker-compose.test.yml` - docker compose for testing database
- `Dockerfile` - production Dockerfile
- `main.py` - App entrypoint
- `requirements.txt` - Python dependencies
- `test_locally.sh` - script to run tests with temporary database
- `charts/`
- `myapp-chart/` - Helm chart for deploying the application, supports prod and dev environments
- `frontend/` - React frontend application
- `tofu/` - Terraform/OpenTofu services deployment configurations
- `modules/` - separated modules for different services
- `main.tf` - main deployment configuration
- `variables.tf` - deployment variables
- `terraform.tfvars.example` - example variables file
- `compose.yaml` - Docker Compose file for local development
- `create_migration.sh` - script to create new Alembic database migration
- `upgrade_database.sh` - script to upgrade database to latest Alembic revision

View File

@@ -1,4 +1,4 @@
import uvicorn
if __name__ == "__main__":
uvicorn.run("app.app:app", host="0.0.0.0", log_level="info")
uvicorn.run("app.app:fastApi", host="0.0.0.0", log_level="info")

View File

@@ -0,0 +1,30 @@
# Helm chart deployment
This directory contains a Helm chart for deploying the app to a cluster, it support bot production and preview
deployment.
## Directory Structure
- `myapp-chart/`
- `templates/`
- `app-deployment.yaml` - Kubernetes Deployment for the application
- `cron.yaml` - cronjob for periodic tasks - periodically calls app endpoint
- `database.yaml` - Creates database using MariaDB operator. Production database is kept, but preview/dev
database is dropped after uninstalling the chart.
- `database-grant.yaml` - Defines rights for the database user
- `database-user.yaml` - Creates database user
- `monitoring.yaml` - Adds /metrics endpoint to Prometheus scraping
- `prod.yaml` - Application secrets
- `rabbitmq-cluster.yaml` - Defines RabbitMQ cluster for this deployment
- `rabbitmq-permission.yalm` - Defines RabbitMQ user permissions
- `rabbitmq-queue.yaml` - Defines RabbitMQ queue
- `rabbitmq-user.yaml` - Defines RabbitMQ user
- `rabbitmq-user-secret.yaml` - Defines RabbitMQ user secret
- `service.yaml` - Kubernetes Service for the application
- `tunnel.yaml` - Cloudflare tunnel for accessing the application¨
- `worker-deployment.yaml` - Kubernetes Deployment for the Celery worker, uses same image as the app-deployment,
but with different entrypoint
- `Chart.yaml` - Helm chart metadata
- `values.yaml` - list of all configurable values
- `values-dev.yaml` - default values for development/preview deployment
- `values-prod.yaml` - default values for production deployment

View File

@@ -0,0 +1,20 @@
# Terraform/OpenTofu IaC
This directory contains infrastructure configuration files and modules for underlying services.
## Directory Structure
- `modules/` - separated modules for different services
- `cert-manager/` - module for deploying Cert-Manager
- `cloudflare/` - module for Cloudflare tunnels operator - allows to expose services via Cloudflare tunnels
- `maxscale/` - module for MariaDB Operator set up for MaxScale and 3 nodes, backups and phpMyAdmin. PHPMyAdmin is
available at mysql.YOU_DOMAIN, MaxScale UI at maxscale.YOU_DOMAIN
- `metallb/` - module for MetalLB load balancer
- `metrics-server/` - module for Kubernetes Metrics Server
- `prometheus/` - module for Prometheus and Grafana - monitoring stack with grafana available at grafana.YOU_DOMAIN
- `rabbitmq/` - module for RabbitMQ message queue cluster
- `redis/` - module for Redis cluster, not used currently
- `storage/` - module for Longhorn file storage
- `main.tf` - main deployment configuration
- `variables.tf` - deployment variables
- `terraform.tfvars.example` - example variables file

View File

@@ -31,5 +31,5 @@ resource "helm_release" "longhorn" {
chart = "longhorn"
namespace = "longhorn-system"
version = "1.9.1"
timeout = 3600
timeout = 300
}

View File

@@ -95,13 +95,6 @@ variable "cloudflare_account_id" {
nullable = false
}
variable "argocd_admin_password" {
type = string
nullable = false
sensitive = true
description = "ArgoCD admin password"
}
variable "rabbitmq-password" {
type = string
nullable = false

View File

@@ -1 +1,3 @@
# Group-8 project
https://finance.ltrk.cz/