sortifal/MLOps

Go to file

Alexis Bruneteau 3cb1b23669 Add DVC S3 credentials to CI/CD pipeline

Configure DVC to use AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
from Gitea secrets (DVC_ID and DVC_PASSWORD) for MinIO/S3 access.

Changes:
- Added DVC credentials to all DVC operations (pull/push)
- Changed poetry install to use --no-root flag for faster installs
- Credentials applied to both test and train jobs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

2025-10-01 17:52:21 +02:00

setup dvc

2025-09-30 17:03:15 +02:00

.gitea/workflows

Add DVC S3 credentials to CI/CD pipeline

2025-10-01 17:52:21 +02:00

Add CI/CD pipeline, monitoring, and model training components for CS:GO MLOps platform

2025-09-30 16:14:56 +02:00

Add CI/CD pipeline, monitoring, and model training components for CS:GO MLOps platform

2025-09-30 16:14:56 +02:00

setup dvc

2025-09-30 17:03:15 +02:00

Add CI/CD pipeline, monitoring, and model training components for CS:GO MLOps platform

2025-09-30 16:14:56 +02:00

secrets and mlflow should now work

2025-10-01 17:35:13 +02:00

maybe maybe not

2025-10-01 15:04:13 +02:00

secrets and mlflow should now work

2025-10-01 17:35:13 +02:00

train fix

2025-09-30 17:04:43 +02:00

.coverage

test

2025-09-30 16:38:14 +02:00

.dvcignore

Initialize DVC

2025-09-30 15:48:38 +02:00

.gitignore

Initial project structure

2025-09-30 15:44:35 +02:00

docker-compose.yml

Add CI/CD pipeline, monitoring, and model training components for CS:GO MLOps platform

2025-09-30 16:14:56 +02:00

dvc.yaml

maybe maybe not

2025-10-01 15:04:13 +02:00

params.yaml

maybe maybe not

2025-10-01 15:04:13 +02:00

poetry.lock

Add dvc-s3 dependency for S3/MinIO storage support

2025-10-01 17:44:24 +02:00

pyproject.toml

Add dvc-s3 dependency for S3/MinIO storage support

2025-10-01 17:44:24 +02:00

README.md

Add Prometheus client dependency and update README with project details

2025-09-30 16:23:29 +02:00

README.md

MLOps Project

This is an MLOps project for CSGO data analysis and model training.

Features

Data pipeline with Apache Airflow
Model training with PyTorch and scikit-learn
MLflow for experiment tracking
DVC for data versioning
Monitoring with Prometheus
FastAPI for API serving

Setup

Install dependencies:
```
poetry install
```

Run the data pipeline:

airflow dags unpause csgo_data_pipeline

Project Structure

dags/: Airflow DAGs
src/: Source code
models/: Trained models
data/: Data files
notebooks/: Jupyter notebooks
tests/: Test files
config/: Configuration files
docker/: Docker files
kubernetes/: Kubernetes manifests