# Personal finance tracker ## Project Overview **Project Name**: Personal Finance Tracker **Group Members**: - 289229, Lukáš Trkan, lukastrkan - 289258, Dejan Ribarovski, ribardej (derib2613) **Brief Description**: Our application allows users to easily track their cash flow through multiple bank accounts. Users can label their transactions with custom categories that can be later used for filtering and visualization. New transactions are automatically fetched in the background. ## Architecture Overview Our system is a full‑stack web application composed of a React frontend, a FastAPI backend, a PostgreSQL database, and asynchronous background workers powered by Celery with RabbitMQ. Redis is available for caching/kv and may be used by Celery as a result backend. The backend exposes REST endpoints for authentication (email/password and OAuth), users, categories, transactions, exchange rates and bank APIs. A thin controller layer (FastAPI routers) lives under app/api. Infrastructure for Kubernetes is provided via OpenTofu (Terraform‑compatible) modules and the application is packaged via a Helm chart. ### High-Level Architecture ```mermaid flowchart LR proc_queue[Message Queue] --> proc_queue_worker[Worker Service] proc_queue_worker --> ext_mail[(Email Service)] proc_cron[Cron] --> svc proc_queue_worker --> ext_bank[(Bank API)] proc_queue_worker --> db client[Client/Frontend] <--> svc[Backend API] svc --> proc_queue svc <--> db[(Database)] svc <--> api[(UniRate API)] ``` The workflow works in the following way: - Client connects to the frontend. After login, frontend automatically fetches the stored transactions from the database via the backend API and currency rates from UniRate API. - When the client opts for fetching new transactions via the Bank API, the backend delegates the task to a background worker service via the Message queue. - After successful load, these transactions are stored to the database and displayed to the client - There is also a Task planner, that executes periodic tasks, like fetching new transactions automatically from the Bank APIs ### Features - The stored transactions are encrypted in the DB for security reasons. - For every pull request the full APP is deployed on a separate URL and the tests are run by github CI/CD - On every push to main, the production app is automatically updated - UI is responsive for mobile devices ### Components - Frontend (frontend/): React + TypeScript app built with Vite. Talks to the backend via REST, handles login/registration, shows latest transactions, filtering, and allows adding transactions. - Backend API (backend/app): FastAPI app with routers under app/api for auth, users, categories, transactions, exchange rates and bankAPI. Uses FastAPI Users for auth (JWT + OAuth), SQLAlchemy ORM, and Pydantic v2 schemas. - Worker service (backend/app/workers): Celery worker handling asynchronous tasks (e.g., sending verification emails, future background processing). - Database (PostgreSQL): Persists users, categories, transactions; schema managed by Alembic migrations. - Message Queue (RabbitMQ): Transports background jobs from the API to the worker. - Cache/Result Store (Redis): Available for caching or Celery result backend. - Infrastructure as Code (tofu/): OpenTofu modules provisioning cluster services (RabbitMQ, Redis, Argo CD, cert-manager, Cloudflare tunnel, etc.). - Deployment Chart (charts/myapp-chart/): Helm chart to deploy the application to Kubernetes. ### Technologies Used - Backend: Python, FastAPI, FastAPI Users, SQLAlchemy, Pydantic, Alembic, Celery - Frontend: React, TypeScript, Vite - Database: MariaDB with Maxscale - Background jobs: RabbitMQ, Celery - Containerization/Orchestration: Docker, Docker Compose (dev), Kubernetes, Helm - IaC/Platform: Proxmox, Talos, Cloudflare pages, OpenTofu (Terraform), cert-manager, MetalLB, Cloudflare Tunnel, Prometheus, Loki ## Prerequisites ### System Requirements - Operating System (dev): Linux, macOS, or Windows with Docker support - Operating System (prod): Linux with kubernetes - Minimum RAM: 4 GB (8 GB recommended for running backend, frontend, and database together) - Storage: 4 GB free (Docker images may require additional space) ### Required Software - Docker Desktop or Docker Engine - Docker Compose - Node.js and npm - Python 3.12+ - MariaDB 11 - Helm 3.12+ and kubectl 1.29+ - OpenTofu ### Environment Variables (common) # TODO: UPDATE - Backend: SECRET, FRONTEND_URL, BACKEND_URL, DATABASE_URL, RABBITMQ_URL, REDIS_URL, UNIRATE_API_KEY - OAuth vars (Backend): MOJEID_CLIENT_ID/SECRET, BANKID_CLIENT_ID/SECRET (optional) - Frontend: VITE_BACKEND_URL ### Dependencies (key libraries) Backend: FastAPI, fastapi-users, SQLAlchemy, pydantic v2, Alembic, Celery, uvicorn Frontend: React, TypeScript, Vite ## Local development You can run the project with Docker Compose and Python virtual environment for testing and dev purposes ### 1) Clone the Repository ```bash git clone https://github.com/dat515-2025/Group-8.git cd 7project ``` ### 2) Install dependencies Backend ```bash python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt ``` Frontend ```bash # In 7project/frontend npm install ``` ### 3) Manual Local Run Backend ```bash # From the 7project/ directory docker compose up --build # This starts: MariaDB, RabbitMQ # Set environment variables (or create .env file) # TODO: fix export SECRET=CHANGE_ME_SECRET export FRONTEND_DOMAIN_SCHEME=http://localhost:5173 export BANKID_CLIENT_ID=CHANGE_ME export BANKID_CLIENT_SECRET=CHANGE_ME export CSAS_CLIENT_ID=CHANGE_ME export CSAS_CLIENT_SECRET=CHANGE_ME export MOJEID_CLIENT_ID=CHANGE_ME export MOJEID_CLIENT_SECRET=CHANGE_ME # Apply DB migrations (Alembic) # From 7project bash upgrade_database.sh # Run API uvicorn app.app:fastApi --reload --host 0.0.0.0 --port 8000 celery -A app.celery_app.celery_app worker -l info ``` Frontend ```bash # Configure backend URL for dev echo 'VITE_BACKEND_URL=http://127.0.0.1:8000' > .env npm run dev # Open http://localhost:5173 ``` - Backend default: http://127.0.0.1:8000 (OpenAPI at /docs) - Frontend default: http://localhost:5173 ## Build Instructions ### Backend ```bash # run in project7/backend docker buildx build --platform linux/amd64,linux/arm64 -t your_container_registry/your_name --push . ``` ### Frontend ```bash # run in project7/frontend npm ci npm run build ``` ## Deployment Instructions ### Setup Cluster Deployment should work on any Kubernetes cluster. However, we are using 4 TalosOS virtual machines (1 control plane, 3 workers) running on top of Proxmox VE. 1) Create 4 VMs with TalosOS 2) Install talosctl for your OS: https://docs.siderolabs.com/talos/v1.10/getting-started/talosctl 3) Generate Talos config ```bash # TODO: add commands ``` 4) Edit the generated worker.yaml - add google container registry mirror - add modules from config generator - add extramounts for persistent storage - add kernel modules 5) Apply the config to the VMs ```bash #TODO: add config apply commands ``` 6) Verify the cluster is up ```bash ``` 7) Export kubeconfig ```bash # TODO: add export command ``` ### Install 1) Install base services to cluster ```bash cd tofu # copy and edit variables cp terraform.tfvars.example terraform.tfvars # authenticate to your cluster/cloud as needed, then: tofu init tofu apply -exclude modules.cloudflare tofu apply ``` 2) Deploy the app using Helm ```bash # Set the namespace kubectl create namespace myapp || true # Install/upgrade the chart with required values helm upgrade --install myapp charts/myapp-chart \ -n myapp \ -f charts/myapp-chart/values.yaml \ --set image.backend.repository=myorg/myapp-backend \ --set image.backend.tag=latest \ --set env.BACKEND_URL="https://myapp.example.com" \ --set env.FRONTEND_URL="https://myapp.example.com" \ --set env.SECRET="CHANGE_ME_SECRET" ``` Adjust values to your registry and domain. The chart’s NOTES.txt includes additional examples. 3) Expose and access - If using Cloudflare Tunnel or an ingress, configure DNS accordingly (see tofu/modules/cloudflare and deployment/tunnel.yaml). - For quick testing without ingress: ```bash kubectl -n myapp port-forward deploy/myapp-backend 8000:8000 kubectl -n myapp port-forward deploy/myapp-frontend 5173:80 ``` ### Verification ```bash # Check pods kubectl -n myapp get pods # Backend health curl -i http://127.0.0.1:8000/ # OpenAPI open http://127.0.0.1:8000/docs # Frontend (if port-forwarded) open http://localhost:5173 ``` ## Testing Instructions The tests are located in 7project/backend/tests directory. All tests are run by GitHub actions on every pull request and push to main. See the workflow [here](../.github/workflows/run-tests.yml). If you want to run the tests locally, the preferred is to use a [bash script](backend/test-with-ephemeral-mariadb.sh) that will start a [test DB container](backend/docker-compose.test.yml) and remove it afterward. ```bash cd 7project/backend bash test-with-ephemeral-mariadb.sh ``` ### Unit Tests There are only 5 basic unit tests, since our services logic is very simple ```bash bash test-with-ephemeral-mariadb.sh --only-unit ``` ### Integration Tests There are 9 basic unit tests, testing the individual backend API logic ```bash bash test-with-ephemeral-mariadb.sh --only-integration ``` ### End-to-End Tests There are 7 e2e tests, testing more complex app logic ```bash bash test-with-ephemeral-mariadb.sh --only-e2e ``` ## Usage Examples All endpoints are documented at OpenAPI: http://127.0.0.1:8000/docs ### Auth: Register and Login (JWT) ```bash # Register curl -X POST http://127.0.0.1:8000/auth/register \ -H 'Content-Type: application/json' \ -d '{ "email": "user@example.com", "password": "StrongPassw0rd", "first_name": "Jane", "last_name": "Doe" }' # Login (JWT) TOKEN=$(curl -s -X POST http://127.0.0.1:8000/auth/jwt/login \ -H 'Content-Type: application/x-www-form-urlencoded' \ -d 'username=user@example.com&password=StrongPassw0rd' | jq -r .access_token) echo $TOKEN # Call a protected route curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8000/authenticated-route ``` ### Frontend - Start with: npm run dev in 7project/frontend - Ensure VITE_BACKEND_URL is set to the backend URL (e.g., http://127.0.0.1:8000) - Open http://localhost:5173 - Login, view latest transactions, filter, and add new transactions from the UI. --- ## Presentation Video **YouTube Link**: [Insert your YouTube link here] **Duration**: [X minutes Y seconds] **Video Includes**: - [ ] Project overview and architecture - [ ] Live demonstration of key features - [ ] Code walkthrough - [ ] Build and deployment showcase ## Troubleshooting ### Common Issues #### Issue 1: [Common problem] **Symptoms**: [What the user sees] **Solution**: [Step-by-step fix] #### Issue 2: [Another common problem] **Symptoms**: [What the user sees] **Solution**: [Step-by-step fix] ### Debug Commands ```bash # Useful commands for debugging # Log viewing commands # Service status checks ``` --- ## Progress Table > Be honest and detailed in your assessments. > This information is used for individual grading. > Link to the specific commit on GitHub for each contribution. | Task/Component | Assigned To | Status | Time Spent | Difficulty | Notes | |-----------------------------------------------------------------------|-------------| ------------- |------------|------------| ----------- | | [Project Setup & Repository](https://github.com/dat515-2025/Group-8#) | Lukas | ✅ Complete | [X hours] | Medium | [Any notes] | | [Design Document](https://github.com/dat515-2025/Group-8/blob/main/6design/design.md) | Both | ✅ Complete | 4 Hours | Easy | [Any notes] | | [Backend API Development](https://github.com/dat515-2025/Group-8/tree/main/7project/backend/app/api) | Dejan | ✅ Complete | 12 hours | Medium | [Any notes] | | [Database Setup & Models](https://github.com/dat515-2025/Group-8/tree/main/7project/backend/app/models) | Lukas | 🔄 In Progress | [X hours] | Medium | [Any notes] | | [Frontend Development](https://github.com/dat515-2025/Group-8/tree/main/7project/frontend) | Dejan | ✅ Complete | 17 hours | Medium | [Any notes] | | [Docker Configuration](https://github.com/dat515-2025/Group-8/blob/main/7project/compose.yml) | Lukas | ✅ Complete | [X hours] | Easy | [Any notes] | | [Cloud Deployment](https://github.com/dat515-2025/Group-8/blob/main/7project/deployment/app-demo-deployment.yaml) | Lukas | ✅ Complete | [X hours] | Hard | [Any notes] | | [Testing Implementation](https://github.com/dat515-2025/group-name) | Dejan | ✅ Complete | 16 hours | Medium | [Any notes] | | [Documentation](https://github.com/dat515-2025/group-name) | Both | 🔄 In Progress | [X hours] | Easy | [Any notes] | | [Presentation Video](https://github.com/dat515-2025/group-name) | Both | ❌ Not Started | [X hours] | Medium | [Any notes] | **Legend**: ✅ Complete | 🔄 In Progress | ⏳ Pending | ❌ Not Started ## Hour Sheet > Link to the specific commit on GitHub for each contribution. ### [Lukáš] | Date | Activity | Hours | Description | |----------------|---------------------|------------|----------------------------------------------------| | 4.10 to 10.10 | Initial Setup | 40 | Repository setup, project structure, cluster setup | | 14.10 to 16.10 | Backend Development | 12 | Implemented user authentication - oauth | | 8.10 to 12.10 | CI/CD | 10 | Created database schema and models | | [Date] | Testing | [X.X] | Unit tests for API endpoints | | [Date] | Documentation | [X.X] | Updated README and design doc | | **Total** | | **[XX.X]** | | ### Dejan | Date | Activity | Hours | Description | |-----------------|----------------------|--------|---------------------------------------------------------------| | 25.9. | Design | 2 | 6design | | 9.10 to 11.10. | Backend APIs | 12 | Implemented Backend APIs | | 13.10 to 15.10. | Frontend Development | 8 | Created user interface mockups | | Continually | Documentation | 6 | Documenting the dev process | | 21.10 to 23.10 | Tests, frontend | 10 | Test basics, balance charts, and frontend improvement | | 28.10 to 30.10 | CI | 6 | Integrated tests with test database setup on github workflows | | 28.10 to 30.10 | Frontend | 7 | UI improvements and exchange rate API integration | | 4.11 to 6.11 | Tests | 6 | Test fixes improvement, more integration and e2e | | 4.11 to 6.11 | Frontend | 6 | Fixes, Improved UI, added support for mobile devices | | **Total** | | **63** | | ### Group Total: [XXX.X] hours --- ## Final Reflection ### What We Learned [Reflect on the key technical and collaboration skills learned during this project] ### Challenges Faced [Describe the main challenges and how you overcame them] ### If We Did This Again [What would you do differently? What worked well that you'd keep?] ### Individual Growth #### [Team Member 1 Name] [Personal reflection on growth, challenges, and learning] #### [Team Member 2 Name] [Personal reflection on growth, challenges, and learning] --- **Report Completion Date**: [Date] **Last Updated**: 15.10.2025