Compare commits

...

8 Commits

Author SHA1 Message Date
ribardej
b5290119e9 feat(docs): report.md update 2025-11-14 17:56:08 +01:00
0beb889f5e updated docs 2025-11-14 17:32:11 +01:00
ribardej
0a96c32c93 Merge remote-tracking branch 'origin/main' 2025-11-14 17:24:04 +01:00
ribardej
f1034f6ed5 feat(docs): report.md update 2025-11-14 17:23:55 +01:00
0f729d28d1 Merge pull request #54 from dat515-2025/merge/core_simplificcation
refactor(core): simplify core module
2025-11-14 17:14:47 +01:00
c689caea88 refactor(core): fix tests 2025-11-14 16:51:21 +01:00
8c20deb690 refactor(core): simplify core module 2025-11-14 16:42:35 +01:00
39979b51ee update report 2025-11-14 15:20:16 +01:00
9 changed files with 253 additions and 188 deletions

View File

@@ -8,7 +8,7 @@ The core deliverables are required.
This means that you must get at least 2 points for each item in this category. This means that you must get at least 2 points for each item in this category.
| **Category** | **Item** | **Max Points** | **Points** | **Comments** | | **Category** | **Item** | **Max Points** | **Points** | **Comments** |
|----------------------------------| --------------------------------------- | -------------- |-------------------------------------------------| | |----------------------------------| --------------------------------------- | -------------- |-------------------------------------------------|---------------|
| **Core Deliverables (Required)** | | | | | | **Core Deliverables (Required)** | | | | |
| Codebase & Organization | Well-organized project structure | 5 | 5 | | | Codebase & Organization | Well-organized project structure | 5 | 5 | |
| | Clean, readable code | 5 | 4 | | | | Clean, readable code | 5 | 4 | |

View File

@@ -48,6 +48,60 @@ flowchart TB
client <-- HTTP request/response --> n2 client <-- HTTP request/response --> n2
``` ```
### Database Schema
```mermaid
classDiagram
direction BT
class alembic_version {
varchar(32) version_num
}
class categories {
varchar(100) name
varchar(255) description
char(36) user_id
int(11) id
}
class category_transaction {
int(11) category_id
int(11) transaction_id
}
class oauth_account {
char(36) user_id
varchar(100) oauth_name
varchar(4096) access_token
int(11) expires_at
varchar(1024) refresh_token
varchar(320) account_id
varchar(320) account_email
char(36) id
}
class transaction {
blob amount
blob description
char(36) user_id
date date
int(11) id
}
class user {
varchar(100) first_name
varchar(100) last_name
varchar(320) email
varchar(1024) hashed_password
tinyint(1) is_active
tinyint(1) is_superuser
tinyint(1) is_verified
longtext config
char(36) id
}
categories --> user : user_id -> id
category_transaction --> categories : category_id -> id
category_transaction --> transaction : transaction_id -> id
oauth_account --> user : user_id -> id
transaction --> user : user_id -> id
```
The workflow works in the following way: The workflow works in the following way:
- Client connects to the frontend. After login, frontend automatically fetches the stored transactions from - Client connects to the frontend. After login, frontend automatically fetches the stored transactions from
@@ -79,6 +133,14 @@ The workflow works in the following way:
etc.). etc.).
- Deployment Chart (charts/myapp-chart/): Helm chart to deploy the application to Kubernetes. - Deployment Chart (charts/myapp-chart/): Helm chart to deploy the application to Kubernetes.
### Other services deployed in the cluster
- Longhorn: distributed storage system providing persistent volumes for the database and other services
- Prometheus + Grafana: monitoring stack collecting metrics from the app and cluster, visualized in Grafana dashboards
- MariaDB operator: manages the MariaDB cluster based on Custom resources, creates Databases, users, handles backups
- RabbitMQ operator: manages RabbitMQ cluster based on Custom resources
- Cloudflare Tunnel: allows public access to backend API running in the private cluster, providing HTTPS
### Technologies Used ### Technologies Used
- Backend: Python, FastAPI, FastAPI Users, SQLAlchemy, Pydantic, Alembic, Celery - Backend: Python, FastAPI, FastAPI Users, SQLAlchemy, Pydantic, Alembic, Celery
@@ -202,10 +264,10 @@ bash upgrade_database.sh
### 5) Run backend ### 5) Run backend
Before running the backend, make sure to set the necessary environment variables. Either by setting them in your shell
or by setting them in run configuration in your IDE.
```bash ```bash
cd backend cd backend
#TODO: set env variables
uvicorn app.app:fastApi --reload --host 0.0.0.0 --port 8000 uvicorn app.app:fastApi --reload --host 0.0.0.0 --port 8000
``` ```
@@ -258,7 +320,7 @@ via GitHub actions and Helm chart. Frontend files are deployed to Cloudflare pag
### Setup Cluster ### Setup Cluster
Deployment should work on any Kubernetes cluster. However, we are using 4 TalosOS virtual machines (1 control plane, 3 Deployment should work on any Kubernetes cluster. However, we are using 5 TalosOS virtual machines (1 control plane, 4
workers) workers)
running on top of Proxmox VE. running on top of Proxmox VE.
@@ -526,6 +588,7 @@ npm run dev in 7project/src/frontend
## Troubleshooting ## Troubleshooting
### Common Issues ### Common Issues
#### Issue 1: Unable to apply Cloudflare terraform module #### Issue 1: Unable to apply Cloudflare terraform module
@@ -550,10 +613,29 @@ by either hitting docker hub rate limits or by docker hub being down.
### Debug Commands ### Debug Commands
Get a detailed description of the Deployment:
```bash ```bash
# Useful commands for debugging kubectl describe deployment finance-tracker -n prod
# Log viewing commands ```
# Service status checks
Get a list of pods in the Deployment:
```bash
kubectl get pods -n prod
```
Check the logs of a specific pod copy value for <pod-name> from the command above (--previous flag shows logs of a failing pod, remove it if the pod is not failing):
```bash
kubectl logs <pod-name> -n prod --previous
```
See the service description:
```bash
kubectl describe service finance-tracker -n prod
```
Connect to the pod and run a bash shell:
```bash
kubectl exec -it <pod-name> -n prod -- /bin/bash
``` ```
--- ---
@@ -566,9 +648,9 @@ by either hitting docker hub rate limits or by docker hub being down.
| Task/Component | Assigned To | Status | Time Spent | Difficulty | Notes | | Task/Component | Assigned To | Status | Time Spent | Difficulty | Notes |
|-------------------------------------------------------------------------------------------------------------------|-------------|----------------|------------|------------|-----------------------------------------------------------------------------------------------------| |-------------------------------------------------------------------------------------------------------------------|-------------|----------------|------------|------------|-----------------------------------------------------------------------------------------------------|
| [Project Setup & Repository](https://github.com/dat515-2025/Group-8#) | Lukas | ✅ Complete | [X hours] | Medium | [Any notes] | | [Project Setup & Repository](https://github.com/dat515-2025/Group-8#) | Lukas | ✅ Complete | 40 Hours | Medium | [Any notes] |
| [Design Document](https://github.com/dat515-2025/Group-8/blob/main/6design/design.md) | Both | ✅ Complete | 4 Hours | Easy | [Any notes] | | [Design Document](https://github.com/dat515-2025/Group-8/blob/main/6design/design.md) | Both | ✅ Complete | 4 Hours | Easy | [Any notes] |
| [Backend API Development](https://github.com/dat515-2025/Group-8/tree/main/7project/backend/app/api) | Dejan | ✅ Complete | 12 hours | Medium | [Any notes] | | [Backend API Development](https://github.com/dat515-2025/Group-8/tree/main/7project/backend/app/api) | Dejan | ✅ Complete | 14 hours | Medium | [Any notes] |
| [Database Setup & Models](https://github.com/dat515-2025/Group-8/tree/main/7project/backend/app/models) | Lukas | ✅ Complete | [X hours] | Medium | [Any notes] | | [Database Setup & Models](https://github.com/dat515-2025/Group-8/tree/main/7project/backend/app/models) | Lukas | ✅ Complete | [X hours] | Medium | [Any notes] |
| [Frontend Development](https://github.com/dat515-2025/Group-8/tree/main/7project/frontend) | Dejan | ✅ Complete | 17 hours | Medium | [Any notes] | | [Frontend Development](https://github.com/dat515-2025/Group-8/tree/main/7project/frontend) | Dejan | ✅ Complete | 17 hours | Medium | [Any notes] |
| [Docker Configuration](https://github.com/dat515-2025/Group-8/blob/main/7project/compose.yml) | Lukas | ✅ Complete | 3 hours | Easy | [Any notes] | | [Docker Configuration](https://github.com/dat515-2025/Group-8/blob/main/7project/compose.yml) | Lukas | ✅ Complete | 3 hours | Easy | [Any notes] |
@@ -610,23 +692,25 @@ by either hitting docker hub rate limits or by docker hub being down.
### Dejan ### Dejan
| Date | Activity | Hours | Description | Representative Commit / PR | | Date | Activity | Hours | Description | Representative Commit / PR |
|:----------------|:---------------------|:-------|:--------------------------------------------------------------|:---------------------------------------------------------| |:-----------------|:---------------------|:-------|:----------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------|
| 25.9. | Design | 2 | 6design | | | 25.9. | Design | 2 | 6design | |
| 9.10 to 11.10. | Backend APIs | 14 | Implemented Backend APIs | `PR #26`, `20-create-a-controller-layer-on-backend-side` | | 9.10. to 11.10. | Backend APIs | 14 | Implemented Backend APIs | `PR #26`, `20-create-a-controller-layer-on-backend-side` |
| 13.10 to 15.10. | Frontend Development | 8 | Created user interface mockups | `PR #28`, `frontend basics` | | 13.10. to 15.10. | Frontend Development | 8 | Created user interface mockups | `PR #28`, `frontend basics` |
| Continually | Documentation | 7 | Documenting the dev process | | | 21.10. to 23.10. | Tests, frontend | 10 | Test basics, balance charts, and frontend improvement | `PR #31`, `30 create tests and set up a GitHub pipeline` |
| 21.10 to 23.10 | Tests, frontend | 10 | Test basics, balance charts, and frontend improvement | `PR #31`, `30 create tests and set up a GitHub pipeline` | | 28.10. to 30.10. | CI/CD | 6 | Integrated tests with test database setup on github workflows | `PR #31`, `30 create tests and set up a GitHub pipeline` |
| 28.10 to 30.10 | CI | 6 | Integrated tests with test database setup on github workflows | `PR #28`, `frontend basics` | | 28.10. to 30.10. | Frontend | 8 | UI improvements and exchange rate API integration | `PR #35`, `34 improve frontend functionality` |
| 28.10 to 30.10 | Frontend | 8 | UI improvements and exchange rate API integration | `PR #28`, `frontend basics` | | 29.10. | Backend | 4 | Token invalidation, few fixes | `PR #38`, `fix(backend): implemented jwt token invalidation so users cannot use …` |
| 4.11 to 6.11 | Tests | 6 | Test fixes improvement, more integration and e2e | `PR #28`, `frontend basics` | | 4.11. to 6.11. | Tests | 6 | Test fixes improvement, more integration and e2e | `PR #45`, `feat(test): added more tests ` |
| 4.11 to 6.11 | Frontend | 6 | Fixes, Improved UI, added support for mobile devices | `PR #28`, `frontend basics` | | 4.11. to 6.11. | Frontend | 8 | Fixes, rates API, Improved UI, added support for mobile devices | `PR #41, #44`, `feat(frontend): added CNB API and moved management into a new tab`, `43 fix the UI layout in chrome ` |
| 11.11 | Backend APIs | 4 | Moved rates API, mock bank to Backend, few fixes | `PR #28`, `frontend basics` | | 11.11. | Backend APIs | 4 | Moved rates API, mock bank to Backend, few fixes | `feat(backend): Moved the unirate API to the backend `, `feat(backend): moved mock bank to backend` |
| 11.11 to 12.11 | Tests | 3 | Local testing DB container, few fixes | `PR #28`, `frontend basics` | | 11.11. to 12.11. | Tests | 3 | Local testing DB container, few fixes | `PR #48`, `fix(tests): fixed test runtime errors regarding database connection ` |
| 12.11 | Frontend | 3 | Enabled multiple transaction edits at once, CSAS button state | `PR #28`, `frontend basics` | | 12.11. | Frontend | 3 | Enabled multiple transaction edits at once, CSAS button state | `feat(frontend): implemented multiple transaction selections in UI` |
| 13.11 | Video | 3 | Video | | | 13.11. | Video | 3 | Video | |
| **Total** | | **80** | | | | 25.9. to 14.11. | Documentation | 8 | Documenting the dev process | multiple `feat(docs): report.md update` |
| **Total** | | **87** | | |
### Group Total: [XXX.X] hours
### Group Total: 192 hours
--- ---
@@ -634,7 +718,13 @@ by either hitting docker hub rate limits or by docker hub being down.
### What We Learned ### What We Learned
[Reflect on the key technical and collaboration skills learned during this project] #### Technical
- We learned how to use AI to help us with our project.
- We learned how to use Copilot for PR reviews.
- We learned how to troubleshoot issues with our project in different areas.
#### Collaboration
- Weekly meetings with the TA were great for syncing up on progress, discussing issues, planning future work.
- Using GitHub issues and pull requests was very helpful for keeping track of progress.
### Challenges Faced ### Challenges Faced
@@ -648,6 +738,10 @@ If the deployed module (helm chart for example) was not configured properly, it
namespace that cannot be deleted. namespace that cannot be deleted.
This was solved by using snapshots in Proxmox and restoring if this happened. This was solved by using snapshots in Proxmox and restoring if this happened.
#### Not enough time to implement all features
Since this course is worth only 5 credits, we often had to prioritize other courses we were attending over this project.
In the end, we were able to implement all necessary features.
### If We Did This Again ### If We Did This Again
#### Different framework #### Different framework
@@ -656,6 +750,15 @@ FastAPI lacks usable build in support for database migrations and implementing A
Tricky was also integrating FastAPI auth system with React frontend, since there is no official project template. Tricky was also integrating FastAPI auth system with React frontend, since there is no official project template.
Using .NET (which we considered initially) would probably solve these issues. Using .NET (which we considered initially) would probably solve these issues.
#### Private container registry
Using private container registry would allow us to include environment variables directly in the image during build.
This would simplify deployment and CI/CD setup.
#### Start sooner
The weekly meetings helped us to start planning the project earlier and avoid spending too much time on details,
but we could have started earlier if we had more time.
[What would you do differently? What worked well that you'd keep?] [What would you do differently? What worked well that you'd keep?]
### Individual Growth ### Individual Growth
@@ -673,8 +776,14 @@ used not only by myself.
#### [Dejan] #### [Dejan]
Since I do not have a job, this project was probably the most complex one I have ever worked on. Since I do not have a job and I am more theoretically oriented student (I am more into math, algorithms, cryptography), this project was probably the most complex one I have ever worked on.
It was also the first school project where I was encouraged to use AI. For me, it was a great experience to work on an actually deployed fullstack app and not only local development, that I was used to from the past.
It was also a great experience to collaborate with Lukas who has prior experience with app deployment and infrastructure.
Thanks to this, I learned a lot new technologies and how to work in a team (First time reviewing PRs).
It was challenging to wrap my head around the project structure and how everything was connected (And I still think I have some gaps in my knowledge).
But I think that if I decide to create my own demo project in the future, I will definitely be able to work on it much more efficiently.

View File

@@ -1,23 +1,14 @@
## Folder structure ## Folder structure
- `src/` - `src/`
- `backend/` - `backend/` - Python FastAPI backend application. Described in separate [README](./backend/README.md).
- `alembic/` - database migrations
- `app/` - main application code
- `tests/` - tests
- `docker-compose.test.yml` - docker compose for testing database
- `Dockerfile` - production Dockerfile
- `main.py` - App entrypoint
- `requirements.txt` - Python dependencies
- `test_locally.sh` - script to run tests with temporary database
- `charts/` - `charts/`
- `myapp-chart/` - Helm chart for deploying the application, supports prod and dev environments - `myapp-chart/` - Helm chart for deploying the application, supports prod and dev environments. Described in
- `frontend/` - React frontend application separate [README](./charts/README.md).
- `tofu/` - Terraform/OpenTofu services deployment configurations - `frontend/` - React frontend application. Described in separate
- `modules/` - separated modules for different services [README](./frontend/README.md).
- `main.tf` - main deployment configuration - `tofu/` - Terraform/OpenTofu services deployment configurations. Described in separate
- `variables.tf` - deployment variables [README](./tofu/README.md).
- `terraform.tfvars.example` - example variables file
- `compose.yaml` - Docker Compose file for local development - `compose.yaml` - Docker Compose file for local development
- `create_migration.sh` - script to create new Alembic database migration - `create_migration.sh` - script to create new Alembic database migration
- `upgrade_database.sh` - script to upgrade database to latest Alembic revision - `upgrade_database.sh` - script to upgrade database to latest Alembic revision

View File

@@ -0,0 +1,23 @@
# Backend
This directory contains the backend code for the project. It is built using Python and FastAPI framework and with
database migrations support using Alembic.
## Directory structure
- `alembic/` - database migrations
- `app/` - main application code
- `api/` - API endpoints - routers/controllers with request handling logic
- `core/` - core application logic - database session management, security
- `models/` - database models
- `schemas/` - Endpoint schemas
- `services/` - utilities for various tasks
- `workers/` - background tasks
- `app.py` - FastAPI startup script
- `celery_app.py` - Celery startup script
- `tests/` - tests
- `docker-compose.test.yml` - docker compose for testing database
- `Dockerfile` - production Dockerfile
- `main.py` - App entrypoint
- `requirements.txt` - Python dependencies
- `test_locally.sh` - script to run tests with temporary database

View File

@@ -1,6 +0,0 @@
import app.celery_app # noqa: F401
from app.workers.celery_tasks import send_email
def enqueue_email(to: str, subject: str, body: str) -> None:
send_email.delay(to, subject, body)

View File

@@ -14,11 +14,10 @@ from httpx_oauth.oauth2 import BaseOAuth2
from app.models.user import User from app.models.user import User
from app.oauth.bank_id import BankID from app.oauth.bank_id import BankID
from app.oauth.csas import CSASOAuth from app.workers.celery_tasks import send_email
from app.oauth.custom_openid import CustomOpenID from app.oauth.custom_openid import CustomOpenID
from app.oauth.moje_id import MojeIDOAuth from app.oauth.moje_id import MojeIDOAuth
from app.services.db import get_user_db from app.services.db import get_user_db
from app.core.queue import enqueue_email
SECRET = os.getenv("SECRET", "CHANGE_ME_SECRET") SECRET = os.getenv("SECRET", "CHANGE_ME_SECRET")
@@ -87,7 +86,7 @@ class UserManager(UUIDIDMixin, BaseUserManager[User, uuid.UUID]):
"Pokud jsi registraci neprováděl(a), tento email ignoruj.\n" "Pokud jsi registraci neprováděl(a), tento email ignoruj.\n"
) )
try: try:
enqueue_email(to=user.email, subject=subject, body=body) send_email.delay(user.email, subject, body)
except Exception as e: except Exception as e:
print("[Email Fallback] To:", user.email) print("[Email Fallback] To:", user.email)
print("[Email Fallback] Subject:", subject) print("[Email Fallback] Subject:", subject)

View File

@@ -34,7 +34,8 @@ def test_authenticated_route_requires_auth(client):
async def test_on_after_request_verify_enqueues_email(monkeypatch): async def test_on_after_request_verify_enqueues_email(monkeypatch):
calls = {} calls = {}
def fake_enqueue_email(to: str, subject: str, body: str): class FakeCeleryTask:
def delay(to: str, subject: str, body: str):
calls.setdefault("emails", []).append({ calls.setdefault("emails", []).append({
"to": to, "to": to,
"subject": subject, "subject": subject,
@@ -42,7 +43,7 @@ async def test_on_after_request_verify_enqueues_email(monkeypatch):
}) })
# Patch the enqueue_email used inside user_service # Patch the enqueue_email used inside user_service
monkeypatch.setattr(user_service, "enqueue_email", fake_enqueue_email) monkeypatch.setattr(user_service, "send_email", FakeCeleryTask)
class DummyUser: class DummyUser:
def __init__(self, email): def __init__(self, email):

View File

@@ -1,73 +1,21 @@
# React + TypeScript + Vite # React + TypeScript + Vite
This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules. This directory contains the frontend code.
Currently, two official plugins are available: ## Directory Structure
- [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Babel](https://babeljs.io/) (or [oxc](https://oxc.rs) when used in [rolldown-vite](https://vite.dev/guide/rolldown)) for Fast Refresh - `public/` - static files
- [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/) for Fast Refresh - `src/` - frontend code
- `assets/` - static assets
## React Compiler - `pages/` - React pages
- `api.ts` - API client
The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation). - `App.tsx` - React app
- `main.tsx` - entry point
## Expanding the ESLint configuration - `index.css` - global styles
- `config.ts` - configuration
If you are developing a production application, we recommend updating the configuration to enable type-aware lint rules: - `ui.css` - UI styles
- `vite.config.ts` - Vite configuration
```js - `package.json` - NPM dependencies
export default defineConfig([ - `tsconfig.json` - TypeScript configuration
globalIgnores(['dist']), - `index.html` - HTML template
{ -
files: ['**/*.{ts,tsx}'],
extends: [
// Other configs...
// Remove tseslint.configs.recommended and replace with this
tseslint.configs.recommendedTypeChecked,
// Alternatively, use this for stricter rules
tseslint.configs.strictTypeChecked,
// Optionally, add this for stylistic rules
tseslint.configs.stylisticTypeChecked,
// Other configs...
],
languageOptions: {
parserOptions: {
project: ['./tsconfig.node.json', './tsconfig.app.json'],
tsconfigRootDir: import.meta.dirname,
},
// other options...
},
},
])
```
You can also install [eslint-plugin-react-x](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-x) and [eslint-plugin-react-dom](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-dom) for React-specific lint rules:
```js
// eslint.config.js
import reactX from 'eslint-plugin-react-x'
import reactDom from 'eslint-plugin-react-dom'
export default defineConfig([
globalIgnores(['dist']),
{
files: ['**/*.{ts,tsx}'],
extends: [
// Other configs...
// Enable lint rules for React
reactX.configs['recommended-typescript'],
// Enable lint rules for React DOM
reactDom.configs.recommended,
],
languageOptions: {
parserOptions: {
project: ['./tsconfig.node.json', './tsconfig.app.json'],
tsconfigRootDir: import.meta.dirname,
},
// other options...
},
},
])
```