uis-cloud-computing/7project/report.md

# Personal finance tracker

 <!--- **Instructions**:
> This template provides the structure for your project report.
> Replace the placeholder text with your actual content.
> Remove instructions that are not relevant for your project, but leave the headings along with a (NA) label. -->

## Project Overview

**Project Name**: Personal Finance Tracker

**Group Members**:

- 289229, Lukáš Trkan, lukastrkan
- 289258, Dejan Ribarovski, ribardej (derib2613)

**Brief Description**:
Our application allows users to easily track their cash flow
through multiple bank accounts. Users can label their transactions with custom categories that can be later used for 
filtering and visualization. New transactions are automatically fetched in the background.

## Architecture Overview
Our system is a full‑stack web application composed of a React frontend, a FastAPI backend, 
a PostgreSQL database, and asynchronous background workers powered by Celery with RabbitMQ. 
Redis is available for caching/kv and may be used by Celery as a result backend. The backend 
exposes REST endpoints for authentication (email/password and OAuth), users, categories, 
transactions, exchange rates and bank APIs. A thin controller layer (FastAPI routers) lives under app/api. 
Infrastructure for Kubernetes is provided via OpenTofu (Terraform‑compatible) modules and 
the application is packaged via a Helm chart.

### High-Level Architecture

```mermaid
flowchart LR
    proc_queue[Message Queue] --> proc_queue_worker[Worker Service]
    proc_queue_worker --> ext_mail[(Email Service)]
    proc_cron[Task planner] --> proc_queue
    proc_queue_worker --> ext_bank[(Bank API)]
    proc_queue_worker --> db
    client[Client/Frontend] <--> svc[Backend API]
    svc --> proc_queue
    svc <--> db[(Database)]
    svc <--> api[(UniRate API)]
```

The workflow works in the following way:
- Client connects to the frontend. After login, frontend automatically fetches the stored transactions from 
the database via the backend API and currency rates from UniRate API.
- When the client opts for fetching new transactions via the Bank API, the backend delegates the task 
to a background worker service via the Message queue.
- After successful load, these transactions are stored to the database and displayed to the client
- There is also a Task planner, that executes periodic tasks, like fetching new transactions automatically from the Bank APIs

### Features
- The stored transactions are encrypted in the DB for security reasons.
- For every pull request the full APP is deployed on a separate URL and the tests are run by github CI/CD
- On every push to main, the production app is automatically updated
- 

### Components

- Frontend (frontend/): React + TypeScript app built with Vite. Talks to the backend via REST, handles login/registration, shows latest transactions, filtering, and allows adding transactions.
- Backend API (backend/app): FastAPI app with routers under app/api for auth, users, categories, transactions, exchange rates and bankAPI. Uses FastAPI Users for auth (JWT + OAuth), SQLAlchemy ORM, and Pydantic v2 schemas.
- Worker service (backend/app/workers): Celery worker handling asynchronous tasks (e.g., sending verification emails, future background processing).
- Database (PostgreSQL): Persists users, categories, transactions; schema managed by Alembic migrations.
- Message Queue (RabbitMQ): Transports background jobs from the API to the worker.
- Cache/Result Store (Redis): Available for caching or Celery result backend.
- Infrastructure as Code (tofu/): OpenTofu modules provisioning cluster services (RabbitMQ, Redis, Argo CD, cert-manager, Cloudflare tunnel, etc.).
- Deployment Chart (charts/myapp-chart/): Helm chart to deploy the application to Kubernetes.

### Technologies Used

- Backend: Python, FastAPI, FastAPI Users, SQLAlchemy, Pydantic, Alembic, Celery
- Frontend: React, TypeScript, Vite
- Database: MariaDB with Maxscale
- Background jobs: RabbitMQ, Celery
- Containerization/Orchestration: Docker, Docker Compose (dev), Kubernetes, Helm
- IaC/Platform: Proxmox, Talos, Cloudflare pages, OpenTofu (Terraform), cert-manager, MetalLB, Cloudflare Tunnel, Prometheus, Loki

## Prerequisites

### System Requirements

- Operating System (dev): Linux, macOS, or Windows with Docker support
- Operating System (prod): Linux with kubernetes
- Minimum RAM: 4 GB (8 GB recommended for running backend, frontend, and database together)
- Storage: 4 GB free (Docker images may require additional space)

### Required Software

- Docker Desktop or Docker Engine
- Docker Compose 
- Node.js and npm
- Python 3.12+ 
- MariaDB 11
- Helm 3.12+ and kubectl 1.29+
- OpenTofu

### Environment Variables (common)

# TODO: UPDATE
- Backend: SECRET, FRONTEND_URL, BACKEND_URL, DATABASE_URL, RABBITMQ_URL, REDIS_URL, UNIRATE_API_KEY

- OAuth vars (Backend): MOJEID_CLIENT_ID/SECRET, BANKID_CLIENT_ID/SECRET (optional)
- Frontend: VITE_BACKEND_URL

### Dependencies (key libraries)
Backend: FastAPI, fastapi-users, SQLAlchemy, pydantic v2, Alembic, Celery, uvicorn  
Frontend: React, TypeScript, Vite

## Local development

You can run the project with Docker Compose and Python virtual environment for testing and dev purposes

### 1) Clone the Repository

```bash
git clone https://github.com/dat515-2025/Group-8.git
cd 7project
```

### 2) Install dependencies
Backend
```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
Frontend
```bash
# In 7project/frontend
npm install
```

### 3) Manual Local Run

Backend
```bash
# From the 7project/ directory
docker compose up --build
# This starts: MariaDB, RabbitMQ

# Set environment variables (or create .env file)
# TODO: fix
export SECRET=CHANGE_ME_SECRET
export FRONTEND_DOMAIN_SCHEME=http://localhost:5173
export BANKID_CLIENT_ID=CHANGE_ME
export BANKID_CLIENT_SECRET=CHANGE_ME
export CSAS_CLIENT_ID=CHANGE_ME
export CSAS_CLIENT_SECRET=CHANGE_ME
export MOJEID_CLIENT_ID=CHANGE_ME
export MOJEID_CLIENT_SECRET=CHANGE_ME
# Apply DB migrations (Alembic)
# From 7project
bash upgrade_database.sh

# Run API
uvicorn app.app:fastApi --reload --host 0.0.0.0 --port 8000

celery -A app.celery_app.celery_app worker -l info
```

Frontend
```bash
# Configure backend URL for dev
echo 'VITE_BACKEND_URL=http://127.0.0.1:8000' > .env
npm run dev
# Open http://localhost:5173
```

- Backend default: http://127.0.0.1:8000 (OpenAPI at /docs)
- Frontend default: http://localhost:5173

## Build Instructions
### Backend
```bash
# run in project7/backend
docker buildx build --platform linux/amd64,linux/arm64 -t your_container_registry/your_name --push .
```
### Frontend
```bash
# run in project7/frontend
npm ci
npm run build
```

## Deployment Instructions
### Setup Cluster
Deployment should work on any Kubernetes cluster. However, we are using 4 TalosOS virtual machines (1 control plane, 3 workers) 
running on top of Proxmox VE. 

1) Create 4 VMs with TalosOS
2) Install talosctl for your OS: https://docs.siderolabs.com/talos/v1.10/getting-started/talosctl
3) Generate Talos config
```bash
# TODO: add commands
```
4) Edit the generated worker.yaml
- add google container registry mirror
- add modules from config generator 
- add extramounts for persistent storage
- add kernel modules

5) Apply the config to the VMs
```bash
#TODO: add config apply commands
```

6) Verify the cluster is up
```bash
```

7) Export kubeconfig
```bash
# TODO: add export command
```


### Install
1) Install base services to cluster
```bash
cd tofu
# copy and edit variables
cp terraform.tfvars.example terraform.tfvars
# authenticate to your cluster/cloud as needed, then:
tofu init
tofu apply -exclude modules.cloudflare
tofu apply  
```

2) Deploy the app using Helm
```bash
# Set the namespace
kubectl create namespace myapp || true

# Install/upgrade the chart with required values
helm upgrade --install myapp charts/myapp-chart \
  -n myapp \
  -f charts/myapp-chart/values.yaml \
  --set image.backend.repository=myorg/myapp-backend \
  --set image.backend.tag=latest \
  --set env.BACKEND_URL="https://myapp.example.com" \
  --set env.FRONTEND_URL="https://myapp.example.com" \
  --set env.SECRET="CHANGE_ME_SECRET"
```
Adjust values to your registry and domain. The chart’s NOTES.txt includes additional examples.

3) Expose and access
- If using Cloudflare Tunnel or an ingress, configure DNS accordingly (see tofu/modules/cloudflare and deployment/tunnel.yaml).
- For quick testing without ingress:
```bash
kubectl -n myapp port-forward deploy/myapp-backend 8000:8000
kubectl -n myapp port-forward deploy/myapp-frontend 5173:80
```

### Verification

```bash
# Check pods
kubectl -n myapp get pods

# Backend health
curl -i http://127.0.0.1:8000/
# OpenAPI
open http://127.0.0.1:8000/docs

# Frontend (if port-forwarded)
open http://localhost:5173
```

## Testing Instructions
The tests are located in 7project/backend/tests directory. All tests are run by GitHub actions on every pull request and push to main.
See the workflow [here](../.github/workflows/run-tests.yml).

If you want to run the tests locally, the preferred is to use a [bash script](backend/test-with-ephemeral-mariadb.sh)
that will start a [test DB container](backend/docker-compose.test.yml) and remove it afterward.
```bash
cd 7project/backend 
bash test-with-ephemeral-mariadb.sh
```


### Unit Tests
There are only 3 basic unit tests, since our services logic is very simple
```bash
bash test-with-ephemeral-mariadb.sh --only-unit
```

### Integration Tests
There are 11 basic unit tests, testing the individual backend API logic
```bash
bash test-with-ephemeral-mariadb.sh --only-integration
```

### End-to-End Tests
There are 7 e2e tests, testing more complex app logic
```bash
bash test-with-ephemeral-mariadb.sh --only-e2e
```

## Usage Examples

All endpoints are documented at OpenAPI: http://127.0.0.1:8000/docs

### Auth: Register and Login (JWT)

```bash
# Register
curl -X POST http://127.0.0.1:8000/auth/register \
  -H 'Content-Type: application/json' \
  -d '{
    "email": "user@example.com",
    "password": "StrongPassw0rd",
    "first_name": "Jane",
    "last_name": "Doe"
  }'

# Login (JWT)
TOKEN=$(curl -s -X POST http://127.0.0.1:8000/auth/jwt/login \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'username=user@example.com&password=StrongPassw0rd' | jq -r .access_token)

echo $TOKEN

# Call a protected route
curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8000/authenticated-route
```

### Frontend

- Start with: npm run dev in 7project/frontend
- Ensure VITE_BACKEND_URL is set to the backend URL (e.g., http://127.0.0.1:8000)
- Open http://localhost:5173
- Login, view latest transactions, filter, and add new transactions from the UI.

---

## Presentation Video

**YouTube Link**: [Insert your YouTube link here]

**Duration**: [X minutes Y seconds]

**Video Includes**:

- [ ] Project overview and architecture
- [ ] Live demonstration of key features
- [ ] Code walkthrough
- [ ] Build and deployment showcase

## Troubleshooting

### Common Issues

#### Issue 1: [Common problem]

**Symptoms**: [What the user sees]
**Solution**: [Step-by-step fix]

#### Issue 2: [Another common problem]

**Symptoms**: [What the user sees]
**Solution**: [Step-by-step fix]

### Debug Commands

```bash
# Useful commands for debugging
# Log viewing commands
# Service status checks
```

---

## Progress Table

> Be honest and detailed in your assessments.
> This information is used for individual grading.
> Link to the specific commit on GitHub for each contribution.

| Task/Component                                                        | Assigned To | Status        | Time Spent | Difficulty | Notes       |
|-----------------------------------------------------------------------|-------------| ------------- |------------|------------| ----------- |
| [Project Setup & Repository](https://github.com/dat515-2025/Group-8#) | Lukas       | ✅ Complete    | [X hours]  | Medium     | [Any notes] |
| [Design Document](https://github.com/dat515-2025/Group-8/blob/main/6design/design.md)          | Both        | ✅ Complete    | 4 Hours    | Easy       | [Any notes] |
| [Backend API Development](https://github.com/dat515-2025/Group-8/tree/main/7project/backend/app/api)  | Dejan       | ✅ Complete    | 12 hours   | Medium     | [Any notes] |
| [Database Setup & Models](https://github.com/dat515-2025/Group-8/tree/main/7project/backend/app/models)  | Lukas       | 🔄 In Progress    | [X hours]  | Medium     | [Any notes] |
| [Frontend Development](https://github.com/dat515-2025/Group-8/tree/main/7project/frontend)     | Dejan       | ✅ Complete | 17 hours   | Medium     | [Any notes] |
| [Docker Configuration](https://github.com/dat515-2025/Group-8/blob/main/7project/compose.yml)     | Lukas       | ✅ Complete    | [X hours]  | Easy       | [Any notes] |
| [Cloud Deployment](https://github.com/dat515-2025/Group-8/blob/main/7project/deployment/app-demo-deployment.yaml)         | Lukas       | ✅ Complete    | [X hours]  | Hard       | [Any notes] |
| [Testing Implementation](https://github.com/dat515-2025/group-name)   | Dejan       | ✅ Complete    | 16 hours   | Medium     | [Any notes] |
| [Documentation](https://github.com/dat515-2025/group-name)            | Both        | 🔄 In Progress   | [X hours]  | Easy       | [Any notes] |
| [Presentation Video](https://github.com/dat515-2025/group-name)       | Both        | ❌ Not Started     | [X hours]  | Medium     | [Any notes] |

**Legend**: ✅ Complete | 🔄 In Progress | ⏳ Pending | ❌ Not Started

## Hour Sheet

> Link to the specific commit on GitHub for each contribution.

### [Lukáš]

| Date           | Activity            | Hours      | Description                                        |
|----------------|---------------------|------------|----------------------------------------------------|
| 4.10 to 10.10  | Initial Setup       | 40         | Repository setup, project structure, cluster setup |
| 14.10 to 16.10 | Backend Development | 12         | Implemented user authentication - oauth            |
| 8.10 to 12.10  | CI/CD               | 10         | Created database schema and models                 |
| [Date]         | Testing             | [X.X]      | Unit tests for API endpoints                       |
| [Date]         | Documentation       | [X.X]      | Updated README and design doc                      |
| **Total**      |                     | **[XX.X]** |                                                    |

### Dejan

| Date            | Activity             | Hours  | Description                                                   |
|-----------------|----------------------|--------|---------------------------------------------------------------|
| 25.9.           | Design               | 2      | 6design                                                       |
| 9.10 to 11.10.  | Backend APIs         | 12     | Implemented Backend APIs                                      |
| 13.10 to 15.10. | Frontend Development | 8      | Created user interface mockups                                |
| Continually     | Documentation        | 6      | Documenting the dev process                                   |
| 21.10 to 23.10  | Tests, frontend      | 10     | Test basics, balance charts, and frontend improvement         |
| 28.10 to 30.10  | CI                   | 6      | Integrated tests with test database setup on github workflows |
| 28.10 to 30.10  | Frontend             | 7      | UI improvements and exchange rate API integration             |
| 4.11 to 6.11    | Tests                | 6      | Test fixes improvement, more integration and e2e              |
| 4.11 to 6.11    | Frontend             | 6      | Fixes, Improved UI, added support for mobile devices          |
| **Total**       |                      | **63** |                                                               |


### Group Total: [XXX.X] hours

---

## Final Reflection

### What We Learned

[Reflect on the key technical and collaboration skills learned during this project]

### Challenges Faced

[Describe the main challenges and how you overcame them]

### If We Did This Again

[What would you do differently? What worked well that you'd keep?]

### Individual Growth

#### [Team Member 1 Name]

[Personal reflection on growth, challenges, and learning]

#### [Team Member 2 Name]

[Personal reflection on growth, challenges, and learning]


---

**Report Completion Date**: [Date]
**Last Updated**: 15.10.2025