sortifal/MLOps
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add DVC S3 credentials to CI/CD pipeline

Browse Source 
Configure DVC to use AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
from Gitea secrets (DVC_ID and DVC_PASSWORD) for MinIO/S3 access.

Changes:
- Added DVC credentials to all DVC operations (pull/push)
- Changed poetry install to use --no-root flag for faster installs
- Credentials applied to both test and train jobs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Alexis Bruneteau 2025-10-01 17:52:21 +02:00
parent d61fad678c
commit 3cb1b23669
1 changed files with 11 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
.gitea/workflows/mlops-pipeline.yml
View File

@ -31,12 +31,15 @@ jobs:
run: pip install poetry run: pip install poetry
- name: Install dependencies - name: Install dependencies
run: poetry install --no-interaction run: poetry install --no-interaction --no-root
- name: Setup DVC - name: Setup DVC
run: | run: |
poetry run dvc remote list poetry run dvc remote list
poetry run dvc pull || echo "DVC pull failed, continuing..." poetry run dvc pull || echo "DVC pull failed, continuing..."
env:
AWS_ACCESS_KEY_ID: ${{ secrets.DVC_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.DVC_PASSWORD }}
- name: Start API server - name: Start API server
run: | run: |
@ -72,12 +75,15 @@ jobs:
run: pip install poetry run: pip install poetry
- name: Install dependencies - name: Install dependencies
run: poetry install --no-interaction run: poetry install --no-interaction --no-root
- name: Setup DVC - name: Setup DVC
run: | run: |
poetry run dvc remote list poetry run dvc remote list
poetry run dvc pull || echo "DVC pull failed, continuing..." poetry run dvc pull || echo "DVC pull failed, continuing..."
env:
AWS_ACCESS_KEY_ID: ${{ secrets.DVC_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.DVC_PASSWORD }}
- name: Run DVC pipeline - name: Run DVC pipeline
run: poetry run dvc repro run: poetry run dvc repro
@ -89,6 +95,9 @@ jobs:
- name: Push DVC changes - name: Push DVC changes
run: | run: |
poetry run dvc push || echo "DVC push failed, continuing..." poetry run dvc push || echo "DVC push failed, continuing..."
env:
AWS_ACCESS_KEY_ID: ${{ secrets.DVC_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.DVC_PASSWORD }}
deploy: deploy:
needs: train needs: train