Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ Always reference these instructions first and fallback to search or bash command
## Working Effectively

### Bootstrap, Build, and Test Process

- **Install system dependencies (REQUIRED):**
- `sudo apt-get update && sudo apt-get install -y ffmpeg` -- takes 3 minutes. NEVER CANCEL. Set timeout to 5+ minutes.
- **Setup repository:**
Expand All @@ -19,6 +20,7 @@ Always reference these instructions first and fallback to search or bash command
- `make security` -- takes 5 seconds. Runs bandit security scanner.

### Run the Application

- **Development mode (RECOMMENDED):**
```bash
# Set environment variables explicitly (required)
Expand All @@ -34,6 +36,7 @@ Always reference these instructions first and fallback to search or bash command
- Use development mode instead when Docker build fails

### Validation Scenarios

- **ALWAYS test video conversion after making changes:**
1. Start the application in development mode with proper environment variables
2. Create a test video: `ffmpeg -f lavfi -i testsrc=duration=5:size=320x240:rate=1 -f lavfi -i sine=frequency=1000:duration=5 -c:v libx264 -c:a aac -shortest test_input/sample.mp4 -y`
Expand All @@ -47,27 +50,32 @@ Always reference these instructions first and fallback to search or bash command
## Known Limitations and Workarounds

### Docker Build Issues

- **Problem:** `make build` fails with SSL certificate verification errors in sandboxed environments
- **Workaround:** Use development mode instead: `INPUT_DIR=./test_input OUTPUT_DIR=./test_output PYTHONPATH=. python main.py`
- **Note:** This is a known limitation of network-restricted environments, not a code issue

### Environment Configuration

- **Problem:** `make dev` fails because .env file is not automatically loaded
- **Workaround:** Set environment variables explicitly when running Python directly
- **Required variables:** `INPUT_DIR`, `OUTPUT_DIR`, `PYTHONPATH=.`

### Integration Tests

- **Problem:** Integration tests are skipped due to network access requirements
- **Expected:** Tests download sample files from samples.ffmpeg.org
- **Workaround:** Unit tests provide sufficient coverage for development

### Directory Permissions

- **Problem:** Permission denied errors for test directories
- **Fix:** Run `sudo chown -R $(whoami):$(id -gn) test_input test_output` to fix ownership

## Common Tasks

### Make All Tests Pass

```bash
# Install dependencies if not done
sudo apt-get update && sudo apt-get install -y ffmpeg # 3 min timeout
Expand All @@ -82,6 +90,7 @@ make security # 5 seconds
```

### Test Video Conversion Functionality

```bash
# Start application (in separate terminal/session)
INPUT_DIR=./test_input OUTPUT_DIR=./test_output PYTHONPATH=. python main.py
Expand All @@ -99,6 +108,7 @@ make health
```

### Debug Application Issues

```bash
# Run with debug logging
LOG_LEVEL=DEBUG INPUT_DIR=./test_input OUTPUT_DIR=./test_output PYTHONPATH=. python main.py
Expand All @@ -111,6 +121,7 @@ curl http://localhost:8080/metrics | python3 -m json.tool
## Repository Structure

### Key Files and Directories

```
vidaud/
├── src/ # Main application code
Expand All @@ -129,6 +140,7 @@ vidaud/
```

### Frequently Used Commands (from Makefile)

```bash
make help # Show all available commands
make setup # Initial project setup (fast)
Expand All @@ -144,6 +156,7 @@ make clean # Clean up Docker resources
## Configuration Reference

### Environment Variables (Required for Development)

- `INPUT_DIR=./test_input` -- Directory to monitor for video files
- `OUTPUT_DIR=./test_output` -- Directory for converted audio files
- `PYTHONPATH=.` -- Required for Python module imports
Expand All @@ -152,22 +165,26 @@ make clean # Clean up Docker resources
- `MAX_PARALLEL_JOBS=4` -- Optional: Concurrent conversion jobs

### Supported Video Formats

- **Input:** MKV, WebM, MP4, AVI, MOV, WMV
- **Output:** MP3 (320kbps, 44.1kHz) or FLAC (lossless)

### Health Endpoints

- `http://localhost:8080/health` -- Application health status
- `http://localhost:8080/metrics` -- Processing metrics and statistics

## CI/CD Pipeline (.github/workflows/ci.yml)

The automated pipeline runs:

1. **Tests** on Python 3.11 and 3.12 (unit + integration)
2. **Quality checks** (flake8, black, bandit security scan)
3. **Docker build** for AMD64 and ARM64 architectures
4. **Release** automation with GitHub releases

### Workflow triggers:

- Push to `main` or `feature/*` branches
- Pull requests to `main`
- Version tags (`v*`) trigger releases
- Version tags (`v*`) trigger releases
9 changes: 5 additions & 4 deletions .github/release.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
# Configuration for automatically generated release notes
# See: https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes
#
Expand Down Expand Up @@ -28,7 +29,7 @@ changelog:
- enhancement
- feat

- title: 🐛 Bug Fixes
- title: 🐛 Bug Fixes
labels:
- bug
- bugfix
Expand All @@ -43,8 +44,8 @@ changelog:
labels:
- dependencies
- deps
# This will catch Renovate PRs based on the "dependencies" label
# configured in renovate.json
# This will catch Renovate PRs based on the "dependencies" label
# configured in renovate.json

- title: 🔧 Maintenance
labels:
Expand Down Expand Up @@ -77,4 +78,4 @@ changelog:

- title: Other Changes
labels:
- "*"
- "*"
23 changes: 16 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
---
name: CI/CD Pipeline

on:
push:
branches: [main]
tags: ["v*"]
branches:
- main
tags:
- "v*"
pull_request:
branches: [main]
branches:
- main

concurrency:
group: ${{ github.head_ref || github.ref_name }}
Expand All @@ -23,7 +27,9 @@
contents: read
strategy:
matrix:
python-version: ["3.11", "3.12"]
python-version:
- "3.11"
- "3.12"

steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
Expand All @@ -34,22 +40,22 @@
python-version: ${{ matrix.python-version }}

- name: Install FFmpeg
run: |

Check warning on line 43 in .github/workflows/ci.yml

View workflow job for this annotation

GitHub Actions / lint / inline scripts

Inline script

Job 'test', step 'Install FFmpeg': multiple lines. Move the code to a script under scripts/ or .github/scripts/ and call it with one command, or opt out with '# lint: allow-inline-script <reason>'.
sudo apt-get update
sudo apt-get install -y ffmpeg

- name: Install dependencies
run: |

Check warning on line 48 in .github/workflows/ci.yml

View workflow job for this annotation

GitHub Actions / lint / inline scripts

Inline script

Job 'test', step 'Install dependencies': multiple lines. Move the code to a script under scripts/ or .github/scripts/ and call it with one command, or opt out with '# lint: allow-inline-script <reason>'.
python -m pip install --upgrade pip
pip install -r requirements.txt

- name: Run unit tests
run: |

Check warning on line 53 in .github/workflows/ci.yml

View workflow job for this annotation

GitHub Actions / lint / inline scripts

Inline script

Job 'test', step 'Run unit tests': multiple lines. Move the code to a script under scripts/ or .github/scripts/ and call it with one command, or opt out with '# lint: allow-inline-script <reason>'.
# Uses pyproject.toml [tool.pytest.ini_options] config automatically
python -m pytest tests/ -v --cov=src --cov-report=xml --cov-report=term-missing -m "not integration"

- name: Run integration tests
run: |

Check warning on line 58 in .github/workflows/ci.yml

View workflow job for this annotation

GitHub Actions / lint / inline scripts

Inline script

Job 'test', step 'Run integration tests': multiple lines. Move the code to a script under scripts/ or .github/scripts/ and call it with one command, or opt out with '# lint: allow-inline-script <reason>'.
# Integration tests that download real video samples
python -m pytest tests/ -v -m integration
continue-on-error: true # Don't fail build if network issues occur
Expand All @@ -76,17 +82,17 @@
python-version: "3.14"

- name: Install dependencies
run: |

Check warning on line 85 in .github/workflows/ci.yml

View workflow job for this annotation

GitHub Actions / lint / inline scripts

Inline script

Job 'lint-and-security-scan', step 'Install dependencies': multiple lines. Move the code to a script under scripts/ or .github/scripts/ and call it with one command, or opt out with '# lint: allow-inline-script <reason>'.
python -m pip install --upgrade pip
pip install -r requirements.txt

- name: Lint with flake8
run: |

Check warning on line 90 in .github/workflows/ci.yml

View workflow job for this annotation

GitHub Actions / lint / inline scripts

Inline script

Job 'lint-and-security-scan', step 'Lint with flake8': multiple lines. Move the code to a script under scripts/ or .github/scripts/ and call it with one command, or opt out with '# lint: allow-inline-script <reason>'.
# Uses .flake8 config file automatically
python -m flake8 src/ tests/

- name: Check formatting with black
run: |

Check warning on line 95 in .github/workflows/ci.yml

View workflow job for this annotation

GitHub Actions / lint / inline scripts

Inline script

Job 'lint-and-security-scan', step 'Check formatting with black': multiple lines. Move the code to a script under scripts/ or .github/scripts/ and call it with one command, or opt out with '# lint: allow-inline-script <reason>'.
# Uses pyproject.toml [tool.black] config automatically
python -m black --check src/ tests/

Expand All @@ -100,7 +106,9 @@

docker-build:
runs-on: ubuntu-latest
needs: [test, lint-and-security-scan]
needs:
- test
- lint-and-security-scan
permissions:
contents: read
packages: write
Expand All @@ -113,7 +121,7 @@

- name: Set lowercase image name
id: image_name
run: echo "value=$(echo ${{ env.IMAGE_NAME }} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_OUTPUT
run: echo "value=$(echo "${IMAGE_NAME}" | tr '[:upper:]' '[:lower:]')" >> "$GITHUB_OUTPUT"

Check warning on line 124 in .github/workflows/ci.yml

View workflow job for this annotation

GitHub Actions / lint / inline scripts

Inline script

Job 'docker-build', step 'Set lowercase image name': command substitution ($(...), backticks or <(...)). Move the code to a script under scripts/ or .github/scripts/ and call it with one command, or opt out with '# lint: allow-inline-script <reason>'.

- name: Log in to Container Registry
if: github.event_name != 'pull_request'
Expand Down Expand Up @@ -164,7 +172,7 @@
cache-to: type=gha,mode=max

- name: Test Docker image
run: |

Check warning on line 175 in .github/workflows/ci.yml

View workflow job for this annotation

GitHub Actions / lint / inline scripts

Inline script

Job 'docker-build', step 'Test Docker image': multiple lines, a pipe (|), a command list (&&, ||, ; or &), command substitution ($(...), backticks or <(...)), a ${{ }} expression (pass it through env: instead). Move the code to a script under scripts/ or .github/scripts/ and call it with one command, or opt out with '# lint: allow-inline-script <reason>'.
# Basic smoke test of the Docker image using the first tag
IMAGE_TAG=$(echo "${{ steps.meta.outputs.tags }}" | head -n1)
CONTAINER_NAME="test-container-${{ github.run_id }}-${{ github.run_attempt }}"
Expand All @@ -184,7 +192,8 @@

release:
if: startsWith(github.ref, 'refs/tags/v')
needs: [docker-build]
needs:
- docker-build
runs-on: ubuntu-latest
permissions:
contents: write
Expand Down
47 changes: 47 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
# Lint caller that repo-steward adds to public repos without one (rule
# baseline/required-file#lint, policy/baseline.toml [[files]]). It follows
# DevSecNinja/.github workflow-templates/lint.yml and
# docs/workflow-trigger-conventions.md. Private repos get lint-private.yml.
# Renovate keeps the pin and the mise version current. When steward opens the
# PR, main becomes the repo's default branch.
name: Lint

on:
# gitleaks scans only the PR's own commits here. The full history is
# scanned by secret-scan.yml (DevSecNinja/.github#520); this workflow's
# push run scans it too, as the reusable workflow does on every non-PR event.
pull_request:
# Public repos upload SARIF to code scanning, which needs a default-branch
# analysis as the baseline for PR alerts.
push:
branches:
- main

permissions:
contents: read

# One run per PR (or branch). A newer push to a PR cancels its running run; a
# running push run is never cancelled, a queued one is superseded.
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
lint:
uses: DevSecNinja/.github/.github/workflows/lint.yml@231c380d4f75486e159ed2e3d806bc76ff193a43 # v4.7.0
permissions:
contents: read
security-events: write
with:
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
# renovate: datasource=github-releases depName=jdx/mise
mise-version: "2026.11.9"
# The linters run from the repo's own mise config (.mise.toml): pin the
# tools it needs there. Every linter is on by default; turning off
# gitleaks, trivy, zizmor or checkov needs a repo-steward exception
# (rule security/ci-scanners). See DevSecNinja/.github
# workflow-templates/lint.yml for the other inputs.
# Spec Kit output (vendored scripts) is not linted, as the central dprint
# and yamlfmt configs exclude .specify/**. The linters stay on for the rest.
lint-shellcheck-exclude: ".specify/*"
lint-shfmt-exclude: ".specify/*"
48 changes: 48 additions & 0 deletions .github/workflows/secret-scan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
---
# Secret-scan caller that repo-steward adds to every repo without one (rule
# baseline/required-file#secret-scan, policy/baseline.toml [[files]]). It
# follows DevSecNinja/.github's own secret-scan-caller.yml: lint-include.yml,
# the thin variant of the reusable lint workflow where every linter is off
# unless set to true, with only gitleaks on. gitleaks scans the full git
# history on every event that isn't a pull request (DevSecNinja/.github#520),
# so new gitleaks
# rules re-check old commits. Findings stay private: public repos upload them
# to code scanning (category gitleaks/history) and log a count only; private
# repos list them in the job summary, which only collaborators can read.
# Pull requests are scanned by the lint caller (their own commits only).
# Renovate keeps the pin and the mise version current. When steward opens the
# PR, main becomes the repo's default branch.
name: Secret scan

on:
push:
branches:
- main
schedule:
- cron: "23 5 * * 1"
workflow_dispatch:

permissions:
contents: read

# A scan that already started finishes; the next one queues.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

jobs:
gitleaks:
uses: DevSecNinja/.github/.github/workflows/lint-include.yml@231c380d4f75486e159ed2e3d806bc76ff193a43 # v4.7.0
permissions:
contents: read
# Public repos: the history findings go to code scanning.
security-events: write
with:
# renovate: datasource=github-releases depName=jdx/mise
mise-version: "2026.11.9"
# History findings only report (warnings); this fails the run when the
# scan or the code scanning upload fails, so a silent gap can't look
# green. gitleaks must be pinned in the repo's .mise.toml.
lint-fail-on-error: true
Comment thread
DevSecNinja marked this conversation as resolved.
Resolved
# lint-include.yml runs no linter unless it is set to true.
lint-gitleaks: true
15 changes: 15 additions & 0 deletions .mise.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Tool versions for this repository, pinned and Renovate-managed (mise manager).
# These are the lint tools run by the shared lint workflow
# (DevSecNinja/.github .github/workflows/lint.yml). The workflow installs from
# this config and fails with guidance if a tool it needs is not pinned here.
[tools]
dprint = "0.64.3"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
yamlfmt = "0.21.0"
actionlint = "1.7.12"
gitleaks = "8.30.1"
shellcheck = "0.11.0"
shfmt = "3.14.1"
trivy = "0.76.1"
Comment thread
DevSecNinja marked this conversation as resolved.
Resolved
zizmor = "1.31.1"
Comment thread
DevSecNinja marked this conversation as resolved.
Resolved
"pipx:yamllint" = "1.38.0"
"pipx:checkov" = "3.4.0"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
Loading
Loading