diff --git a/.devcontainer/README.md b/.devcontainer/README.md index b243834..c2669ea 100644 --- a/.devcontainer/README.md +++ b/.devcontainer/README.md @@ -36,6 +36,7 @@ This devcontainer provides a complete development environment for the Docker Inf 3. Use Command Palette: `Remote-Containers: Reopen in Container` The initial setup will take a few minutes to: + - Pull the base Debian image - Install all required tools and dependencies - Configure the development environment @@ -111,6 +112,7 @@ The devcontainer has access to the entire repository: ### VS Code Settings The devcontainer configures: + - Format on save - LF line endings - YAML indentation (2 spaces) @@ -120,6 +122,7 @@ The devcontainer configures: ### Docker-in-Docker The container runs Docker-in-Docker to support: + - Testing Docker installations - Running Docker Compose modules - Container-based testing @@ -156,6 +159,7 @@ ls /usr/local/lib/bats/ ### Slow Container Startup First-time setup takes longer due to: + - Installing Ansible and dependencies - Downloading Ansible collections/roles - Installing Bats framework diff --git a/.devcontainer/post-create.sh b/.devcontainer/post-create.sh index 66d5df9..8c92ba8 100644 --- a/.devcontainer/post-create.sh +++ b/.devcontainer/post-create.sh @@ -47,7 +47,7 @@ echo "🚀 Setting up Docker Infrastructure development environment..." # Set Docker socket permissions echo "đŸŗ Configuring Docker permissions..." sudo groupadd -f docker --gid 780 # Note: this group ID aligns with the docker_group role -sudo groupmod -g 780 docker # In case the group already existed with a different GID +sudo groupmod -g 780 docker # In case the group already existed with a different GID sudo usermod -aG docker vscode check_docker_sock || exit 1 @@ -85,11 +85,11 @@ echo "" # Check if Docker is usable without sudo in THIS session if ! docker info >/dev/null 2>&1; then - echo "âš ī¸ Docker is not accessible in this session." - echo " You were likely added to the 'docker' group after this shell started." - echo " Run: sudo su - \$USER" - echo " (or reopen the terminal) to refresh group membership." - echo "" + echo "âš ī¸ Docker is not accessible in this session." + echo " You were likely added to the 'docker' group after this shell started." + echo " Run: sudo su - \$USER" + echo " (or reopen the terminal) to refresh group membership." + echo "" fi echo "Quick commands:" diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 0000000..245f5b7 --- /dev/null +++ b/.editorconfig @@ -0,0 +1,21 @@ +; https://editorconfig.org/ + +root = true + +[*] +indent_style = space +indent_size = 2 +end_of_line = lf +charset = utf-8 +trim_trailing_whitespace = true +insert_final_newline = true + +[*.md] +indent_size = 4 +trim_trailing_whitespace = false + +[*.sh] +indent_size = 4 + +[*.py] +indent_size = 4 diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 9d4fdad..b2c6c3f 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -18,10 +18,13 @@ This is an Ansible-based infrastructure automation repository that manages Docke ## Key Commands ### Important: Always Use Task Runner + When suggesting commands to the user, **always prefer `task` commands** from `Taskfile.yml` over raw commands. Only fall back to raw commands if no equivalent task exists. Run `task --list` to discover available tasks. ### Important: Always Use mise for Tool Management + When installing or managing tool versions (Python, Ansible, ansible-lint, yamllint, Bats, Task, etc.), **always use `mise`** instead of `pip install`, `apt install`, or other package managers. Tool versions are defined in `.mise.toml` and must stay in sync. + ```bash # Install all tools at versions defined in .mise.toml mise install @@ -34,6 +37,7 @@ mise use pipx:ansible-lint@26.5.0 ``` ### Task Runner (Recommended) + ```bash # Show all available tasks task --list @@ -58,6 +62,7 @@ task help ``` ### Testing + ```bash # Run all Bats tests (with Task) task test @@ -78,6 +83,7 @@ task test:docker ``` ### Running Ansible + ```bash # Dry-run with Task task ansible:check @@ -97,6 +103,7 @@ sudo ansible-pull \ ``` ### Local Development + ```bash # Install all dependencies (with Task) task install @@ -139,6 +146,7 @@ tests/ ## Code Conventions ### Ansible Playbooks and Roles + - Use YAML with 2-space indentation - Follow Ansible best practices for role structure - All playbooks must start with `---` @@ -149,6 +157,7 @@ tests/ - Use `handlers/main.yml` for service restarts ### YAML Style + - Use 2-space indentation (never tabs) - No trailing whitespace - End files with a single newline @@ -157,12 +166,14 @@ tests/ - Use lowercase for booleans: `true`, `false` ### Naming Conventions + - Roles: lowercase with underscores (e.g., `ansible_pull_setup`) - Variables: lowercase with underscores (e.g., `server_features`) - Host names: lowercase (e.g., `svlazext`) - Tags: lowercase (e.g., `docker`, `traefik`) ### File Organization + - Group related tasks in role subdirectories - Use `tasks/main.yml` as the entry point for roles - Split complex roles into separate task files @@ -172,6 +183,7 @@ tests/ ## Testing Requirements ### Before Making Changes + 1. **ALWAYS run tests first**: `./tests/bash/run-tests.sh` - This is a required step before committing any Ansible or YAML changes - Fix all test failures before proceeding @@ -182,6 +194,7 @@ tests/ - GitHub SSH Keys: `./tests/bash/run-tests.sh --test github-ssh-keys-test.bats` ### After Making Changes + - **All YAML files must pass yamllint** (no errors, warnings acceptable) - **All Ansible files must pass ansible-lint** (no errors, warnings acceptable) - Playbooks must pass syntax-check @@ -189,7 +202,9 @@ tests/ - **Run linting again before final commit** to ensure all issues are resolved ### CI/CD Pipeline + The repository uses the [Bats testing framework](https://github.com/bats-core/bats-core) for comprehensive testing: + - Lint checks (yamllint, ansible-lint) - Syntax validation (Ansible playbooks and shell scripts) - Docker provisioning tests @@ -199,6 +214,7 @@ The repository uses the [Bats testing framework](https://github.com/bats-core/ba All tests are located in `tests/bash/` and use the Bats test format. Workflows trigger on: + - Push to `main` or `copilot/**` branches - Pull requests - Manual workflow dispatch @@ -208,17 +224,20 @@ See [tests/bash/README.md](../tests/bash/README.md) for detailed testing documen ## Important Boundaries ### DO NOT Modify + - `.git/` directory or git history - LICENSE file (MIT License) - Production server configurations without explicit approval - Secrets or credentials (use Ansible Vault if needed) ### DO Modify Carefully + - `ansible/inventory/hosts.yml` - Server inventory (only with clear understanding) - `ansible/playbooks/main.yml` - Main playbook (ensure backward compatibility) - GitHub Actions workflows - Test thoroughly before merging ### ALWAYS + - Follow existing code structure and patterns - Test changes using provided test scripts - Maintain backward compatibility with existing servers @@ -229,6 +248,7 @@ See [tests/bash/README.md](../tests/bash/README.md) for detailed testing documen ## Example Workflows ### Adding a New Ansible Role + 1. Create role structure: ```bash cd ansible/roles @@ -253,6 +273,7 @@ See [tests/bash/README.md](../tests/bash/README.md) for detailed testing documen 5. Test with CI pipeline ### Adding a New Server + 1. Add to `ansible/inventory/hosts.yml`: ```yaml NEWSERVER: @@ -266,6 +287,7 @@ See [tests/bash/README.md](../tests/bash/README.md) for detailed testing documen 3. Test the configuration ### Modifying Docker Compose Modules + 1. Edit module definition in `ansible/roles/docker_compose_modules/vars/modules/` 2. Update role tasks if needed in `ansible/roles/docker_compose_modules/tasks/` 3. Test with: `bash ansible/scripts/tests/docker-test.sh` diff --git a/.yamlfmt.yaml b/.yamlfmt.yaml new file mode 100644 index 0000000..ce24c16 --- /dev/null +++ b/.yamlfmt.yaml @@ -0,0 +1,26 @@ +--- +doublestar: true +formatter: + type: basic + disable_alias_key_correction: true + indent: 2 + max_line_length: 120 + pad_line_comments: 1 + drop_merge_tag: true + eof_newline: true + force_array_style: block + include_document_start: true + line_ending: lf + retain_line_breaks_single: true + scan_folded_as_literal: true + trim_trailing_whitespace: true +# APM-managed / generated files use the tool's own formatting — never hand-edited. +exclude: + - apm.yml + - apm.lock.yaml + - apm_modules/** + # Spec Kit output and Python virtual environments. Listed explicitly rather + # than through gitignore_excludes: without a .gitignore, yamlfmt then lints + # nothing and still exits 0. + - .specify/** + - .venv*/** diff --git a/.yamllint.yaml b/.yamllint.yaml new file mode 100644 index 0000000..6e6f5ff --- /dev/null +++ b/.yamllint.yaml @@ -0,0 +1,22 @@ +--- +extends: default + +rules: + # Inline comment spacing varies by author style — not auto-fixable + comments: disable + # Long lines are unavoidable with image digests, Traefik labels, and URLs — not auto-fixable + line-length: disable + # Let yamlfmt handle document start markers + document-start: disable + # Allow 'on:' as a mapping key in GitHub Actions workflows + truthy: + allowed-values: + - "true" + - "false" + check-keys: false + +# APM-managed / generated files use the tool's own formatting — never hand-edited. +ignore: |- + apm.yml + apm.lock.yaml + apm_modules/ diff --git a/INSTALL.md b/INSTALL.md index 3de4a20..0ec2eaa 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -120,6 +120,7 @@ For svlazext, ensure the following: For svldev (Development/Management Server), the following steps are largely automated: **Automated Steps:** + - ✅ Ansible user creation with sudo privileges (`system_setup` role) - ✅ Docker group membership for specified users (`geerlingguy.docker` role) - ✅ SSH service configuration (`system_setup` role) diff --git a/README.md b/README.md index 96fe605..cb4257a 100644 --- a/README.md +++ b/README.md @@ -72,6 +72,7 @@ For Ansible documentation and structure, see [ansible/README.md](ansible/README. ### svlazext (Debian) Primary Docker host configured with: + - ✅ Docker Engine (geerlingguy.docker) - ✅ UFW Firewall (HTTP, HTTPS, SSH) - ✅ Chezmoi dotfiles management @@ -81,6 +82,7 @@ Primary Docker host configured with: ### svldev (Debian) Development/management server configured with: + - ✅ Automated system setup (ansible user, sudo, SSH) - ✅ Docker Engine with user group management (geerlingguy.docker) - ✅ UFW Firewall (SSH) @@ -143,18 +145,18 @@ task help ### Common Task Commands -| Command | Description | -|---------|-------------| -| `task install` | Install all dependencies (Python, Ansible, Bats) | -| `task test` | Run all tests | -| `task test:lint` | Run linting tests only | -| `task test:syntax` | Run syntax validation tests | -| `task lint` | Run yamllint and ansible-lint | -| `task ansible:check` | Dry-run ansible-pull (no changes) | -| `task ansible:pull` | Run ansible-pull (apply configuration) | -| `task ci:local` | Run full CI pipeline locally | -| `task info` | Display environment information | -| `task help` | Show detailed help and examples | +| Command | Description | +| -------------------- | ------------------------------------------------ | +| `task install` | Install all dependencies (Python, Ansible, Bats) | +| `task test` | Run all tests | +| `task test:lint` | Run linting tests only | +| `task test:syntax` | Run syntax validation tests | +| `task lint` | Run yamllint and ansible-lint | +| `task ansible:check` | Dry-run ansible-pull (no changes) | +| `task ansible:pull` | Run ansible-pull (apply configuration) | +| `task ci:local` | Run full CI pipeline locally | +| `task info` | Display environment information | +| `task help` | Show detailed help and examples | See [Taskfile.yml](Taskfile.yml) for all available tasks. @@ -165,6 +167,7 @@ The repository includes comprehensive testing using the [Bats testing framework] ### Running Tests Locally With Task (recommended): + ```bash # Run all tests task test @@ -176,6 +179,7 @@ task test:docker ``` Without Task: + ```bash # Run all tests ./tests/bash/run-tests.sh @@ -190,6 +194,7 @@ Without Task: ### Test Suite Tests include: + - **Linting**: yamllint and ansible-lint checks - **Syntax validation**: Ansible playbooks and shell scripts - **Docker provisioning**: Installation and configuration @@ -201,6 +206,7 @@ See [tests/bash/README.md](tests/bash/README.md) for detailed testing documentat ### CI/CD Pipeline Tests are automatically run via GitHub Actions on: + - Push to `main` or `copilot/**` branches - Pull requests - Manual workflow dispatch diff --git a/Taskfile.yml b/Taskfile.yml index edf5096..5b2f7d8 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -1,3 +1,4 @@ +--- version: "3" # Task runner for Docker Infrastructure repository @@ -63,8 +64,7 @@ tasks: install:ansible-collections: desc: Install required Ansible collections cmds: - - ansible-galaxy collection install -r {{.ANSIBLE_DIR}}/requirements.yml - --force + - ansible-galaxy collection install -r {{.ANSIBLE_DIR}}/requirements.yml --force sources: - "{{.ANSIBLE_DIR}}/requirements.yml" @@ -184,8 +184,7 @@ tasks: desc: Check Ansible playbook syntax dir: "{{.ANSIBLE_DIR}}" cmds: - - ansible-playbook --syntax-check {{.ANSIBLE_PLAYBOOK}} -i - {{.ANSIBLE_INVENTORY}} + - ansible-playbook --syntax-check {{.ANSIBLE_PLAYBOOK}} -i {{.ANSIBLE_INVENTORY}} syntax:scripts: desc: Check shell script syntax @@ -225,23 +224,20 @@ tasks: desc: Run main playbook locally (requires sudo) dir: "{{.ANSIBLE_DIR}}" cmds: - - sudo ansible-playbook {{.ANSIBLE_PLAYBOOK}} -i {{.ANSIBLE_INVENTORY}} - --extra-vars "target_host=$(hostname)" + - sudo ansible-playbook {{.ANSIBLE_PLAYBOOK}} -i {{.ANSIBLE_INVENTORY}} --extra-vars "target_host=$(hostname)" ansible:playbook:check: desc: Run main playbook in check mode (dry-run) dir: "{{.ANSIBLE_DIR}}" cmds: - - sudo ansible-playbook {{.ANSIBLE_PLAYBOOK}} -i {{.ANSIBLE_INVENTORY}} - --extra-vars "target_host=$(hostname)" --check + - sudo ansible-playbook {{.ANSIBLE_PLAYBOOK}} -i {{.ANSIBLE_INVENTORY}} --extra-vars "target_host=$(hostname)" --check ansible:playbook:tags: - desc: Run playbook with specific tags (usage - task ansible:playbook:tags -- - TAG=docker) + desc: Run playbook with specific tags (usage - task ansible:playbook:tags -- TAG=docker) dir: "{{.ANSIBLE_DIR}}" cmds: - - sudo ansible-playbook {{.ANSIBLE_PLAYBOOK}} -i {{.ANSIBLE_INVENTORY}} - --extra-vars "target_host=$(hostname)" --tags "{{.TAG}}" + - sudo ansible-playbook {{.ANSIBLE_PLAYBOOK}} -i {{.ANSIBLE_INVENTORY}} --extra-vars "target_host=$(hostname)" --tags + "{{.TAG}}" ansible:logs: desc: View ansible-pull logs (journalctl) @@ -277,33 +273,29 @@ tasks: desc: Run maintenance update playbook (update config & images) dir: "{{.REPO_ROOT}}" cmds: - - sudo ANSIBLE_CONFIG={{.REPO_ROOT}}/ansible.cfg ansible-playbook - {{.ANSIBLE_DIR}}/playbooks/maintenance-update.yml -i - {{.ANSIBLE_INVENTORY}} --extra-vars "target_host=$(hostname)" + - sudo ANSIBLE_CONFIG={{.REPO_ROOT}}/ansible.cfg ansible-playbook {{.ANSIBLE_DIR}}/playbooks/maintenance-update.yml + -i {{.ANSIBLE_INVENTORY}} --extra-vars "target_host=$(hostname)" maintenance:daily: desc: Run daily maintenance playbook (OS patches) dir: "{{.REPO_ROOT}}" cmds: - - sudo ANSIBLE_CONFIG={{.REPO_ROOT}}/ansible.cfg ansible-playbook - {{.ANSIBLE_DIR}}/playbooks/maintenance-daily.yml -i + - sudo ANSIBLE_CONFIG={{.REPO_ROOT}}/ansible.cfg ansible-playbook {{.ANSIBLE_DIR}}/playbooks/maintenance-daily.yml -i {{.ANSIBLE_INVENTORY}} --extra-vars "target_host=$(hostname)" maintenance:weekly: desc: Run weekly maintenance playbook (patches & reboot) dir: "{{.REPO_ROOT}}" cmds: - - sudo ANSIBLE_CONFIG={{.REPO_ROOT}}/ansible.cfg ansible-playbook - {{.ANSIBLE_DIR}}/playbooks/maintenance-weekly.yml -i - {{.ANSIBLE_INVENTORY}} --extra-vars "target_host=$(hostname)" + - sudo ANSIBLE_CONFIG={{.REPO_ROOT}}/ansible.cfg ansible-playbook {{.ANSIBLE_DIR}}/playbooks/maintenance-weekly.yml + -i {{.ANSIBLE_INVENTORY}} --extra-vars "target_host=$(hostname)" maintenance:user-tools: desc: Run user-tools maintenance playbook (chezmoi, Homebrew, mise) dir: "{{.REPO_ROOT}}" cmds: - - sudo ANSIBLE_CONFIG={{.REPO_ROOT}}/ansible.cfg ansible-playbook - {{.ANSIBLE_DIR}}/playbooks/maintenance-user-tools.yml -i - {{.ANSIBLE_INVENTORY}} --extra-vars "target_host=$(hostname)" + - sudo ANSIBLE_CONFIG={{.REPO_ROOT}}/ansible.cfg ansible-playbook {{.ANSIBLE_DIR}}/playbooks/maintenance-user-tools.yml + -i {{.ANSIBLE_INVENTORY}} --extra-vars "target_host=$(hostname)" maintenance:logs: desc: View maintenance logs (usage - task maintenance:logs [-- daily|weekly|all]) @@ -329,8 +321,7 @@ tasks: esac maintenance:logs:follow: - desc: Follow maintenance logs in real-time (usage - task maintenance:logs:follow - [-- daily|weekly]) + desc: Follow maintenance logs in real-time (usage - task maintenance:logs:follow [-- daily|weekly]) cmds: - | TYPE="{{.CLI_ARGS | default "daily"}}" @@ -396,8 +387,7 @@ tasks: # Devcontainer Prebuild devcontainer:run: - desc: Run both phases - build images (Phase 1) then prewarm containers (Phase 2) - via systemd OnSuccess chain + desc: Run both phases - build images (Phase 1) then prewarm containers (Phase 2) via systemd OnSuccess chain cmds: - | echo "Triggering devcontainer-build.service (Phase 1 + Phase 2 via OnSuccess chain)..." @@ -414,8 +404,7 @@ tasks: sudo /usr/local/bin/devcontainer-build.sh devcontainer:up: - desc: Run Phase 2 only - prewarm containers from existing images (does not - rebuild) + desc: Run Phase 2 only - prewarm containers from existing images (does not rebuild) cmds: - | echo "Triggering devcontainer-up.service (Phase 2)..." @@ -437,8 +426,7 @@ tasks: sudo systemctl status devcontainer-up.service --no-pager -l || true devcontainer:logs: - desc: View devcontainer prebuild logs (usage - task devcontainer:logs [-- - build|up|all]) + desc: View devcontainer prebuild logs (usage - task devcontainer:logs [-- build|up|all]) cmds: - | TYPE="{{.CLI_ARGS | default "all"}}" @@ -461,8 +449,7 @@ tasks: esac devcontainer:logs:follow: - desc: Follow devcontainer logs in real-time (usage - task - devcontainer:logs:follow [-- build|up]) + desc: Follow devcontainer logs in real-time (usage - task devcontainer:logs:follow [-- build|up]) cmds: - | TYPE="{{.CLI_ARGS | default "build"}}" @@ -488,8 +475,7 @@ tasks: dev:format: desc: Format code (YAML files) cmds: - - echo "YAML formatting is done via yamllint checks. Run 'task lint:yaml' - to check." + - echo "YAML formatting is done via yamllint checks. Run 'task lint:yaml' to check." dev:clean: desc: Clean temporary files and caches diff --git a/ansible/README.md b/ansible/README.md index 5659cf6..4c384d3 100644 --- a/ansible/README.md +++ b/ansible/README.md @@ -47,6 +47,7 @@ ansible/ ### svlazext (Debian) Primary Docker host with the following features: + - Docker Engine (via geerlingguy.docker role) - UFW firewall with HTTP/HTTPS/SSH access - Chezmoi dotfiles management @@ -62,27 +63,33 @@ See [INSTALL.md](INSTALL.md) for detailed installation instructions. The maintenance role provides automated server maintenance with three types of tasks: ### 1. Configuration and Docker Image Updates + **Playbook**: `maintenance-update.yml` Pulls the latest Ansible configuration and updates Docker images: + - Pulls latest changes from Git repository - Updates chezmoi dotfiles - Pulls and restarts Docker Compose services with updated images ### 2. Daily OS Patches (Non-disturbing) + **Playbook**: `maintenance-daily.yml` **Schedule**: Daily at 8 PM Applies safe OS updates without disrupting services: + - Updates packages except Docker Engine and kernel - Excludes updates requiring reboot - Safe to run on production systems ### 3. Weekly Full Patches + **Playbook**: `maintenance-weekly.yml` **Schedule**: Saturday at 8 AM Performs comprehensive system updates: + - Applies all available OS updates - Includes Docker Engine and kernel updates - Automatically reboots if required diff --git a/ansible/playbooks/main.yml b/ansible/playbooks/main.yml index f0a031f..2111d11 100644 --- a/ansible/playbooks/main.yml +++ b/ansible/playbooks/main.yml @@ -55,15 +55,21 @@ roles: - role: github_ssh_keys when: "'github_ssh_keys' in server_features" - tags: [github_ssh_keys, ssh] + tags: + - github_ssh_keys + - ssh - role: system_setup when: "'system_setup' in server_features" - tags: [system_setup, base] + tags: + - system_setup + - base - role: docker_group when: "'docker' in server_features" - tags: [docker, docker_group] + tags: + - docker + - docker_group - role: geerlingguy.docker when: "'docker' in server_features" @@ -71,15 +77,20 @@ - role: apt_hold when: ansible_os_family == "Debian" - tags: [apt_hold] + tags: + - apt_hold - role: ufw when: "'ufw' in server_features or 'firewall' in server_features" - tags: [ufw, firewall] + tags: + - ufw + - firewall - role: package_managers when: "'package_managers' in server_features" - tags: [package_managers, homebrew] + tags: + - package_managers + - homebrew - role: chezmoi when: "'chezmoi' in server_features" @@ -87,28 +98,38 @@ - role: docker_compose_modules when: "compose_modules is defined and compose_modules | length > 0" - tags: [docker_compose, services] + tags: + - docker_compose + - services - role: ansible_pull_setup when: "'ansible_pull_setup' in server_features" - tags: [ansible_pull, automation] + tags: + - ansible_pull + - automation - role: maintenance when: "'maintenance' in server_features" - tags: [maintenance, automation] + tags: + - maintenance + - automation - role: packages when: "'packages' in server_features" - tags: [packages] + tags: + - packages - role: git_repos when: "'git_repos' in server_features and git_repos | default([]) | length > 0" - tags: [git_repos] + tags: + - git_repos - role: devcontainer_prebuild when: "'devcontainer_prebuild' in server_features" - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - role: cron_jobs when: "'cron_jobs' in server_features and cron_jobs | default([]) | length > 0" - tags: [cron_jobs] + tags: + - cron_jobs diff --git a/ansible/roles/ansible_pull_setup/defaults/main.yml b/ansible/roles/ansible_pull_setup/defaults/main.yml index 1a85cd6..4d85c25 100644 --- a/ansible/roles/ansible_pull_setup/defaults/main.yml +++ b/ansible/roles/ansible_pull_setup/defaults/main.yml @@ -1,8 +1,8 @@ --- # Default variables for Ansible Pull Setup role -ansible_pull_scheduler: "systemd" # Options: cron, systemd -ansible_pull_interval: "1h" # For systemd timer (e.g., 1h, 30m) -ansible_pull_cron_schedule: "0 * * * *" # For cron (hourly) +ansible_pull_scheduler: "systemd" # Options: cron, systemd +ansible_pull_interval: "1h" # For systemd timer (e.g., 1h, 30m) +ansible_pull_cron_schedule: "0 * * * *" # For cron (hourly) # Repository settings ansible_pull_repo_url: "https://github.com/DevSecNinja/docker.git" diff --git a/ansible/roles/ansible_pull_setup/tasks/main.yml b/ansible/roles/ansible_pull_setup/tasks/main.yml index b6224d9..0aa2eb9 100644 --- a/ansible/roles/ansible_pull_setup/tasks/main.yml +++ b/ansible/roles/ansible_pull_setup/tasks/main.yml @@ -58,7 +58,7 @@ - name: Display setup information ansible.builtin.debug: - msg: > + msg: >- Ansible pull automation configured with {{ ansible_pull_scheduler }}. {% if ansible_pull_scheduler == 'systemd' %} Check status: sudo systemctl status ansible-pull.timer diff --git a/ansible/roles/apt_hold/tasks/main.yml b/ansible/roles/apt_hold/tasks/main.yml index 15cc1eb..4e10172 100644 --- a/ansible/roles/apt_hold/tasks/main.yml +++ b/ansible/roles/apt_hold/tasks/main.yml @@ -14,4 +14,5 @@ failed_when: - apt_hold_result.failed - "'Failed to find package' not in (apt_hold_result.msg | default(''))" - tags: [apt_hold] + tags: + - apt_hold diff --git a/ansible/roles/apt_hold/tasks/unhold.yml b/ansible/roles/apt_hold/tasks/unhold.yml index 677aa30..d439c29 100644 --- a/ansible/roles/apt_hold/tasks/unhold.yml +++ b/ansible/roles/apt_hold/tasks/unhold.yml @@ -14,4 +14,5 @@ failed_when: - apt_unhold_result.failed - "'Failed to find package' not in (apt_unhold_result.msg | default(''))" - tags: [apt_hold] + tags: + - apt_hold diff --git a/ansible/roles/chezmoi/tasks/main.yml b/ansible/roles/chezmoi/tasks/main.yml index 076f4f0..ff1788e 100644 --- a/ansible/roles/chezmoi/tasks/main.yml +++ b/ansible/roles/chezmoi/tasks/main.yml @@ -133,7 +133,8 @@ - name: Download Chezmoi archive (arm64) ansible.builtin.get_url: - url: "https://github.com/twpayne/chezmoi/releases/download/v{{ chezmoi_resolved_version }}/chezmoi_{{ chezmoi_resolved_version }}_linux_arm64.tar.gz" + url: "https://github.com/twpayne/chezmoi/releases/download/v{{ chezmoi_resolved_version }}/chezmoi_{{ chezmoi_resolved_version + }}_linux_arm64.tar.gz" dest: "/tmp/chezmoi_linux_arm64.tar.gz" mode: "0644" force: true diff --git a/ansible/roles/cron_jobs/tasks/main.yml b/ansible/roles/cron_jobs/tasks/main.yml index ead464e..103f204 100644 --- a/ansible/roles/cron_jobs/tasks/main.yml +++ b/ansible/roles/cron_jobs/tasks/main.yml @@ -9,7 +9,8 @@ retries: 10 delay: 30 until: cron_apt_result is succeeded - tags: [cron_jobs] + tags: + - cron_jobs - name: Configure cron jobs ansible.builtin.cron: @@ -23,4 +24,5 @@ user: "{{ item.user | default(ansible_user) }}" state: "{{ item.state | default('present') }}" loop: "{{ cron_jobs }}" - tags: [cron_jobs] + tags: + - cron_jobs diff --git a/ansible/roles/devcontainer_prebuild/tasks/main.yml b/ansible/roles/devcontainer_prebuild/tasks/main.yml index 1689f92..5df4bd3 100644 --- a/ansible/roles/devcontainer_prebuild/tasks/main.yml +++ b/ansible/roles/devcontainer_prebuild/tasks/main.yml @@ -5,17 +5,20 @@ ansible.builtin.stat: path: "{{ devcontainer_cli_install_dir }}/devcontainer" register: devcontainer_standalone_binary - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Check if devcontainer CLI is installed (Homebrew location) ansible.builtin.stat: path: "{{ homebrew_prefix }}/bin/devcontainer" register: devcontainer_brew_binary - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Install devcontainer CLI when: not devcontainer_standalone_binary.stat.exists and not devcontainer_brew_binary.stat.exists - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild block: - name: Install devcontainer CLI via Homebrew become: true @@ -54,7 +57,8 @@ path: "{{ devcontainer_prebuild_config_dir }}" state: directory mode: "0755" - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Deploy default devcontainer configuration become: true @@ -63,19 +67,22 @@ src: devcontainer-default.json.j2 dest: "{{ devcontainer_prebuild_config_dir }}/devcontainer.json" mode: "0644" - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Remove legacy devcontainer-prebuild.service unit (replaced by build+up) ansible.builtin.file: path: /etc/systemd/system/devcontainer-prebuild.service state: absent - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Remove legacy devcontainer-prebuild.sh script (replaced by build+up) ansible.builtin.file: path: /usr/local/bin/devcontainer-prebuild.sh state: absent - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Deploy devcontainer build script (Phase 1) ansible.builtin.template: @@ -84,7 +91,8 @@ mode: "0755" owner: root group: root - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Deploy devcontainer up script (Phase 2) ansible.builtin.template: @@ -93,7 +101,8 @@ mode: "0755" owner: root group: root - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Deploy devcontainer-build systemd service unit (Phase 1) ansible.builtin.template: @@ -102,7 +111,8 @@ mode: "0644" owner: root group: root - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Deploy devcontainer-up systemd service unit (Phase 2) ansible.builtin.template: @@ -111,7 +121,8 @@ mode: "0644" owner: root group: root - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Deploy devcontainer-prebuild systemd timer unit ansible.builtin.template: @@ -120,7 +131,8 @@ mode: "0644" owner: root group: root - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Enable and start devcontainer-prebuild timer ansible.builtin.systemd: @@ -128,7 +140,8 @@ enabled: true state: started daemon_reload: true - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Deploy devcontainer idle-stop script ansible.builtin.template: @@ -137,7 +150,8 @@ mode: "0755" owner: root group: root - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Deploy devcontainer-stop systemd service unit ansible.builtin.template: @@ -146,7 +160,8 @@ mode: "0644" owner: root group: root - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Deploy devcontainer-stop systemd timer unit ansible.builtin.template: @@ -155,7 +170,8 @@ mode: "0644" owner: root group: root - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild - name: Enable and start devcontainer-stop timer ansible.builtin.systemd: @@ -163,4 +179,5 @@ enabled: true state: started daemon_reload: true - tags: [devcontainer_prebuild] + tags: + - devcontainer_prebuild diff --git a/ansible/roles/git_repos/tasks/clone_shim.yml b/ansible/roles/git_repos/tasks/clone_shim.yml index fb1d994..6f7b976 100644 --- a/ansible/roles/git_repos/tasks/clone_shim.yml +++ b/ansible/roles/git_repos/tasks/clone_shim.yml @@ -20,7 +20,8 @@ ansible.builtin.stat: path: "{{ shim_dest }}/.git" register: git_repos_shim_stat - tags: [git_repos] + tags: + - git_repos - name: "Create git shim for '{{ shim_repo }}'" when: not git_repos_shim_stat.stat.exists @@ -28,12 +29,13 @@ become_user: "{{ shim_owner }}" environment: GIT_TERMINAL_PROMPT: "0" - tags: [git_repos] + tags: + - git_repos vars: git_repos_shim_version_effective: "{{ (shim_version | default('HEAD') | string | trim) }}" git_repos_shim_need_detect: "{{ (shim_version | default('HEAD') | string | trim) in ['', 'HEAD'] }}" block: - - name: "Detect remote default branch for '{{ shim_repo }}'" # noqa: command-instead-of-module + - name: "Detect remote default branch for '{{ shim_repo }}'" # noqa: command-instead-of-module ansible.builtin.shell: cmd: >- set -o pipefail; @@ -65,12 +67,12 @@ become: true become_user: root - - name: "Initialise empty repository at '{{ shim_dest }}'" # noqa: command-instead-of-module + - name: "Initialise empty repository at '{{ shim_dest }}'" # noqa: command-instead-of-module ansible.builtin.command: cmd: "git init -q {{ shim_dest | quote }}" creates: "{{ shim_dest }}/.git" - - name: "Configure remote and branch tracking for '{{ shim_dest }}'" # noqa: command-instead-of-module + - name: "Configure remote and branch tracking for '{{ shim_dest }}'" # noqa: command-instead-of-module ansible.builtin.command: cmd: "{{ item }}" chdir: "{{ shim_dest }}" @@ -101,13 +103,14 @@ become_user: "{{ shim_owner }}" environment: GIT_TERMINAL_PROMPT: "0" - tags: [git_repos] + tags: + - git_repos block: # An un-pulled shim has no remote-tracking ref, so '@{u}' does not resolve # and the repository is left as an empty shim. Once the user has run # `git pull` (populating the repo), the checked-out branch gains an upstream # and is fast-forwarded to the latest commit on subsequent runs. - - name: "Check whether '{{ shim_dest }}' tracks an upstream branch" # noqa: command-instead-of-module + - name: "Check whether '{{ shim_dest }}' tracks an upstream branch" # noqa: command-instead-of-module ansible.builtin.command: cmd: git rev-parse --abbrev-ref --symbolic-full-name @{u} chdir: "{{ shim_dest }}" @@ -115,7 +118,7 @@ changed_when: false failed_when: false - - name: "Fast-forward '{{ shim_dest }}' to the latest upstream commit" # noqa: command-instead-of-module + - name: "Fast-forward '{{ shim_dest }}' to the latest upstream commit" # noqa: command-instead-of-module ansible.builtin.command: cmd: git pull --ff-only chdir: "{{ shim_dest }}" @@ -142,4 +145,5 @@ when: - git_repos_shim_upstream.rc == 0 - git_repos_shim_pull.rc | default(0) != 0 - tags: [git_repos] + tags: + - git_repos diff --git a/ansible/roles/git_repos/tasks/github_user.yml b/ansible/roles/git_repos/tasks/github_user.yml index 8c46b23..cc82e87 100644 --- a/ansible/roles/git_repos/tasks/github_user.yml +++ b/ansible/roles/git_repos/tasks/github_user.yml @@ -17,7 +17,8 @@ {{ git_repos_entry.token | default(git_repos_github_token) }} git_repos_limit: >- {{ git_repos_entry.limit | default(0) | int }} - tags: [git_repos] + tags: + - git_repos - name: "Query GitHub API for repositories of '{{ git_repos_entry.github_user }}'" ansible.builtin.uri: @@ -37,7 +38,8 @@ check_mode: false changed_when: false no_log: "{{ git_repos_resolved_token | length > 0 }}" - tags: [git_repos] + tags: + - git_repos - name: "Build repository lists for '{{ git_repos_entry.github_user }}'" ansible.builtin.set_fact: @@ -45,7 +47,8 @@ {{ git_repos_api_pages.results | map(attribute='json') | flatten | rejectattr('private') | list }} - tags: [git_repos] + tags: + - git_repos - name: "Select archived and clone targets for '{{ git_repos_entry.github_user }}'" ansible.builtin.set_fact: @@ -55,7 +58,8 @@ {{ git_repos_public if (git_repos_include_archived and not git_repos_archive_mode) else (git_repos_public | rejectattr('archived') | list) }} - tags: [git_repos] + tags: + - git_repos - name: "Apply clone limit for '{{ git_repos_entry.github_user }}'" ansible.builtin.set_fact: @@ -63,7 +67,8 @@ {{ git_repos_clone_candidates[:git_repos_limit] if git_repos_limit > 0 else git_repos_clone_candidates }} - tags: [git_repos] + tags: + - git_repos - name: "Ensure projects directory exists for '{{ git_repos_entry.github_user }}'" ansible.builtin.file: @@ -73,7 +78,8 @@ group: "{{ git_repos_group }}" mode: "0755" become: true - tags: [git_repos] + tags: + - git_repos - name: "Create git repository shims for '{{ git_repos_entry.github_user }}'" ansible.builtin.include_tasks: clone_shim.yml @@ -87,11 +93,13 @@ shim_version: "{{ shim_item.default_branch }}" shim_owner: "{{ git_repos_owner }}" shim_group: "{{ git_repos_group }}" - tags: [git_repos] + tags: + - git_repos - name: "Relocate archived repositories for '{{ git_repos_entry.github_user }}'" when: git_repos_archive_mode - tags: [git_repos] + tags: + - git_repos block: - name: "Ensure .archive directory exists in '{{ git_repos_dest }}'" ansible.builtin.file: diff --git a/ansible/roles/git_repos/tasks/main.yml b/ansible/roles/git_repos/tasks/main.yml index c5add84..ea66b75 100644 --- a/ansible/roles/git_repos/tasks/main.yml +++ b/ansible/roles/git_repos/tasks/main.yml @@ -9,7 +9,8 @@ retries: 10 delay: 30 until: git_apt_result is succeeded - tags: [git_repos] + tags: + - git_repos - name: Create git repository shims (single-repo format) ansible.builtin.include_tasks: clone_shim.yml @@ -23,12 +24,12 @@ shim_version: "{{ shim_item.version | default('HEAD') }}" shim_owner: "{{ shim_item.user | default(ansible_user) }}" shim_group: "{{ shim_item.group | default(shim_item.user | default(ansible_user)) }}" - tags: [git_repos] + tags: + - git_repos - name: Create git repository shims (multi-repo format) ansible.builtin.include_tasks: clone_shim.yml - loop: "{{ git_repos | selectattr('repos', 'defined') | list | - subelements('repos') }}" + loop: "{{ git_repos | selectattr('repos', 'defined') | list | subelements('repos') }}" loop_control: loop_var: shim_item label: "{{ shim_item.1 }}" @@ -38,7 +39,8 @@ shim_version: "{{ shim_item.0.version | default('HEAD') }}" shim_owner: "{{ shim_item.0.user | default(ansible_user) }}" shim_group: "{{ shim_item.0.group | default(shim_item.0.user | default(ansible_user)) }}" - tags: [git_repos] + tags: + - git_repos - name: Synchronise repositories for GitHub users ansible.builtin.include_tasks: github_user.yml @@ -46,4 +48,5 @@ loop_control: loop_var: git_repos_entry label: "{{ git_repos_entry.github_user }}" - tags: [git_repos] + tags: + - git_repos diff --git a/ansible/roles/github_ssh_keys/tasks/main.yml b/ansible/roles/github_ssh_keys/tasks/main.yml index aad3275..02aae53 100644 --- a/ansible/roles/github_ssh_keys/tasks/main.yml +++ b/ansible/roles/github_ssh_keys/tasks/main.yml @@ -21,7 +21,9 @@ ansible.builtin.uri: url: "https://github.com/{{ github_ssh_keys_username }}.keys" return_content: true - status_code: [200, 404] + status_code: + - 200 + - 404 register: github_keys_response check_mode: false tags: github_ssh_keys diff --git a/ansible/roles/maintenance/tasks/docker_maintenance.yml b/ansible/roles/maintenance/tasks/docker_maintenance.yml index 43cff1f..4fd7f6f 100644 --- a/ansible/roles/maintenance/tasks/docker_maintenance.yml +++ b/ansible/roles/maintenance/tasks/docker_maintenance.yml @@ -122,6 +122,6 @@ - name: Display Docker disk usage ansible.builtin.debug: - msg: | + msg: |- Docker disk usage: {{ maintenance_docker_disk_usage.stdout }} diff --git a/ansible/roles/maintenance/tasks/main.yml b/ansible/roles/maintenance/tasks/main.yml index 95fbe72..9b4e0da 100644 --- a/ansible/roles/maintenance/tasks/main.yml +++ b/ansible/roles/maintenance/tasks/main.yml @@ -38,7 +38,7 @@ - name: Display maintenance information ansible.builtin.debug: - msg: | + msg: |- Maintenance automation configured: - Timezone: {{ maintenance_timezone | default('not set') }} - Daily patches: diff --git a/ansible/roles/package_managers/tasks/install_homebrew.yml b/ansible/roles/package_managers/tasks/install_homebrew.yml index 78bd6c6..3113a9c 100644 --- a/ansible/roles/package_managers/tasks/install_homebrew.yml +++ b/ansible/roles/package_managers/tasks/install_homebrew.yml @@ -6,11 +6,15 @@ ansible.builtin.stat: path: "{{ homebrew_prefix }}/bin/brew" register: homebrew_binary - tags: [package_managers, homebrew] + tags: + - package_managers + - homebrew - name: Install Homebrew when: not homebrew_binary.stat.exists - tags: [package_managers, homebrew] + tags: + - package_managers + - homebrew block: - name: Install Homebrew system dependencies ansible.builtin.apt: @@ -76,7 +80,9 @@ group: "{{ homebrew_user }}" recurse: true when: homebrew_binary.stat.exists or (homebrew_install_result is defined and homebrew_install_result.rc == 0) - tags: [package_managers, homebrew] + tags: + - package_managers + - homebrew - name: Verify Homebrew installation become: true @@ -85,10 +91,14 @@ register: homebrew_version_output changed_when: false when: homebrew_binary.stat.exists or (homebrew_install_result is defined and homebrew_install_result.rc == 0) - tags: [package_managers, homebrew] + tags: + - package_managers + - homebrew - name: Display Homebrew version ansible.builtin.debug: msg: "Homebrew version: {{ homebrew_version_output.stdout | default('not installed') }}" when: homebrew_version_output is defined and homebrew_version_output is not skipped - tags: [package_managers, homebrew] + tags: + - package_managers + - homebrew diff --git a/ansible/roles/package_managers/tasks/install_mise.yml b/ansible/roles/package_managers/tasks/install_mise.yml index 637b2c4..d517d1a 100644 --- a/ansible/roles/package_managers/tasks/install_mise.yml +++ b/ansible/roles/package_managers/tasks/install_mise.yml @@ -6,11 +6,15 @@ ansible.builtin.stat: path: "/home/{{ mise_user }}/.local/bin/mise" register: mise_binary - tags: [package_managers, mise] + tags: + - package_managers + - mise - name: Install mise when: not mise_binary.stat.exists - tags: [package_managers, mise] + tags: + - package_managers + - mise block: - name: Download mise install script ansible.builtin.get_url: @@ -40,10 +44,14 @@ when: > mise_binary.stat.exists or (mise_install_result is defined and mise_install_result.rc == 0) - tags: [package_managers, mise] + tags: + - package_managers + - mise - name: Display mise version ansible.builtin.debug: msg: "Mise version: {{ mise_version_output.stdout }}" when: mise_version_output is not skipped - tags: [package_managers, mise] + tags: + - package_managers + - mise diff --git a/ansible/roles/package_managers/tasks/main.yml b/ansible/roles/package_managers/tasks/main.yml index 24be86a..4c7a17a 100644 --- a/ansible/roles/package_managers/tasks/main.yml +++ b/ansible/roles/package_managers/tasks/main.yml @@ -2,9 +2,13 @@ # Tasks for Package Managers role - name: Install Homebrew ansible.builtin.include_tasks: install_homebrew.yml - tags: [package_managers, homebrew] + tags: + - package_managers + - homebrew - name: Install mise ansible.builtin.include_tasks: install_mise.yml when: install_mise | bool - tags: [package_managers, mise] + tags: + - package_managers + - mise diff --git a/ansible/roles/packages/tasks/main.yml b/ansible/roles/packages/tasks/main.yml index 724dcf8..6623fff 100644 --- a/ansible/roles/packages/tasks/main.yml +++ b/ansible/roles/packages/tasks/main.yml @@ -10,7 +10,8 @@ retries: 10 delay: 30 until: packages_apt_result is succeeded - tags: [packages] + tags: + - packages - name: Install Homebrew packages community.general.homebrew: @@ -20,4 +21,5 @@ become: true become_user: "{{ packages_brew_user }}" when: packages_brew | length > 0 - tags: [packages] + tags: + - packages diff --git a/ansible/roles/system_setup/handlers/main.yml b/ansible/roles/system_setup/handlers/main.yml index e3d1ea8..93e7a77 100644 --- a/ansible/roles/system_setup/handlers/main.yml +++ b/ansible/roles/system_setup/handlers/main.yml @@ -3,10 +3,16 @@ ansible.builtin.systemd: name: systemd-resolved state: restarted - tags: [system_setup, dns, systemd_resolved] + tags: + - system_setup + - dns + - systemd_resolved - name: Reload SSH service ansible.builtin.service: name: "{{ ssh_service_name }}" state: reloaded - tags: [system_setup, ssh, copilot] + tags: + - system_setup + - ssh + - copilot diff --git a/ansible/roles/system_setup/tasks/main.yml b/ansible/roles/system_setup/tasks/main.yml index 4afc373..e73e32f 100644 --- a/ansible/roles/system_setup/tasks/main.yml +++ b/ansible/roles/system_setup/tasks/main.yml @@ -8,7 +8,9 @@ create_home: "{{ ansible_user_create_home }}" append: true state: present - tags: [system_setup, ansible_user] + tags: + - system_setup + - ansible_user - name: Configure passwordless sudo for ansible user ansible.builtin.template: @@ -17,7 +19,10 @@ mode: "0440" validate: "visudo -cf %s" when: ansible_user_passwordless_sudo - tags: [system_setup, ansible_user, sudo] + tags: + - system_setup + - ansible_user + - sudo - name: Ensure SSH service is enabled and running ansible.builtin.systemd: @@ -25,7 +30,9 @@ enabled: true state: started when: ensure_ssh_service - tags: [system_setup, ssh] + tags: + - system_setup + - ssh - name: Ensure sshd includes its drop-in configuration directory ansible.builtin.lineinfile: @@ -40,7 +47,10 @@ - sshd_ensure_include_dropins | bool - sshd_accept_env_vars | length > 0 notify: Reload SSH service - tags: [system_setup, ssh, copilot] + tags: + - system_setup + - ssh + - copilot - name: Ensure the sshd drop-in configuration directory exists ansible.builtin.file: @@ -53,7 +63,10 @@ - ensure_ssh_service | bool - sshd_accept_env_enabled | bool - sshd_accept_env_vars | length > 0 - tags: [system_setup, ssh, copilot] + tags: + - system_setup + - ssh + - copilot - name: Accept client-forwarded environment variables (e.g. Copilot CLI token) ansible.builtin.template: @@ -68,7 +81,10 @@ - sshd_accept_env_enabled | bool - sshd_accept_env_vars | length > 0 notify: Reload SSH service - tags: [system_setup, ssh, copilot] + tags: + - system_setup + - ssh + - copilot - name: Remove the accepted-env drop-in when disabled ansible.builtin.file: @@ -78,7 +94,10 @@ - ensure_ssh_service | bool - not (sshd_accept_env_enabled | bool) or (sshd_accept_env_vars | length == 0) notify: Reload SSH service - tags: [system_setup, ssh, copilot] + tags: + - system_setup + - ssh + - copilot - name: Point /etc/resolv.conf to upstream resolver (non-stub) ansible.builtin.file: @@ -87,7 +106,10 @@ state: link force: true when: disable_systemd_resolved_stub - tags: [system_setup, dns, systemd_resolved] + tags: + - system_setup + - dns + - systemd_resolved - name: Disable systemd-resolved DNS stub listener ansible.builtin.lineinfile: @@ -97,11 +119,15 @@ create: false when: disable_systemd_resolved_stub notify: Restart systemd-resolved - tags: [system_setup, dns, systemd_resolved] + tags: + - system_setup + - dns + - systemd_resolved - name: Display system setup information ansible.builtin.debug: msg: > System setup completed. Ansible user: {{ ansible_user_name }} - tags: [system_setup] + tags: + - system_setup diff --git a/ansible/roles/ufw/defaults/main.yml b/ansible/roles/ufw/defaults/main.yml index a2638bb..a1243af 100644 --- a/ansible/roles/ufw/defaults/main.yml +++ b/ansible/roles/ufw/defaults/main.yml @@ -12,8 +12,8 @@ ufw_ssh_port: 22 # Common service ports ufw_allowed_ports: - - 80 # HTTP - - 443 # HTTPS + - 80 # HTTP + - 443 # HTTPS # Custom rules (list of dicts with port, protocol, rule) ufw_rules: [] diff --git a/ansible/scripts/ansible-pull.sh b/ansible/scripts/ansible-pull.sh index a1ead2b..3712180 100755 --- a/ansible/scripts/ansible-pull.sh +++ b/ansible/scripts/ansible-pull.sh @@ -31,7 +31,7 @@ log() { # repairs a broken package state. Non-lock failures are logged verbatim rather # than silently reported as lock contention. wait_for_apt_locks() { - if ! command -v apt-get &> /dev/null; then + if ! command -v apt-get &>/dev/null; then return 0 fi @@ -42,7 +42,7 @@ wait_for_apt_locks() { if [ "$rc" -eq 0 ]; then log "apt/dpkg locks are available" - elif grep -qiE 'could not get lock|unable to acquire|frontend lock|temporarily unavailable' <<< "$output"; then + elif grep -qiE 'could not get lock|unable to acquire|frontend lock|temporarily unavailable' <<<"$output"; then log "WARNING: apt/dpkg still locked after ${APT_LOCK_TIMEOUT}s, continuing anyway" else log "WARNING: 'apt-get check' failed for a non-lock reason (rc=$rc), continuing anyway:" @@ -53,7 +53,7 @@ wait_for_apt_locks() { log "Starting ansible-pull run" # Install Ansible if not present -if ! command -v ansible-pull &> /dev/null; then +if ! command -v ansible-pull &>/dev/null; then log "Ansible not found, installing..." # Check if running as root if [ "$EUID" -ne 0 ]; then diff --git a/docs/tools/MISE.md b/docs/tools/MISE.md index 4796dee..62ed5ac 100644 --- a/docs/tools/MISE.md +++ b/docs/tools/MISE.md @@ -5,6 +5,7 @@ This project uses [mise](https://mise.jdx.dev/) (formerly rtx) for managing tool ## What is mise? mise is a polyglot tool version manager that: + - Manages versions of Python, Node.js, and other tools - Automatically activates the correct versions when entering the project directory - Loads environment variables from `.env` diff --git a/dprint.json b/dprint.json new file mode 100644 index 0000000..f5e8c80 --- /dev/null +++ b/dprint.json @@ -0,0 +1,26 @@ +{ + "$schema": "https://dprint.dev/schemas/v0.json", + "includes": ["**/*.md"], + "excludes": [ + "**/node_modules", + "**/CHANGELOG.md", + "**/release-notes.md", + "apm_modules/**", + ".github/prompts/**", + ".github/instructions/**", + ".github/agents/**", + ".github/chatmodes/**", + ".claude/**", + ".cursor/**", + ".opencode/**", + ".gemini/**", + ".windsurf/**", + ".kiro/**", + ".agents/**", + ".github/skills/**", + ".specify/**" + ], + "plugins": [ + "https://plugins.dprint.dev/markdown-0.21.1.wasm" + ] +} diff --git a/tests/bash/README.md b/tests/bash/README.md index 394f60f..34ee324 100644 --- a/tests/bash/README.md +++ b/tests/bash/README.md @@ -5,6 +5,7 @@ This directory contains the test suite for the Ansible Docker infrastructure usi ## Overview The test suite validates: + - Ansible configuration syntax and linting - Ansible playbook execution - Docker provisioning @@ -61,6 +62,7 @@ This installs dependencies automatically and generates JUnit XML output: - Bats (will be auto-installed if not present) The test runner will automatically: + - Install Bats if not found (via npm, homebrew, or from source in CI mode) - Install required Python packages (ansible, ansible-lint, yamllint) - Install required Ansible collections @@ -68,6 +70,7 @@ The test runner will automatically: ### CI/CD In CI mode (`--ci` flag), the script will: + - Install Bats from GitHub releases - Generate JUnit XML test results (`test-results.xml`) - Exit with non-zero status on failures @@ -97,6 +100,7 @@ setup() { ### Linting Tests (`lint-test.bats`) Validates code quality and style: + - yamllint checks on YAML files - ansible-lint checks on Ansible roles - Excludes external roles (e.g., geerlingguy.docker) @@ -104,6 +108,7 @@ Validates code quality and style: ### Syntax Tests (`syntax-test.bats`) Validates syntax without execution: + - Ansible playbook syntax - Shell script syntax (bash and sh) - Verifies required files exist @@ -111,6 +116,7 @@ Validates syntax without execution: ### Docker Tests (`docker-test.bats`) Tests Docker provisioning: + - Ansible can install required collections - Test inventory can be created - Playbook passes in check mode @@ -120,6 +126,7 @@ Tests Docker provisioning: ### Ansible Pull Tests (`ansible-pull-test.bats`) Tests ansible-pull functionality: + - ansible-pull script exists and is valid - ansible-pull can run from local repository - Inventory files are in correct locations @@ -127,6 +134,7 @@ Tests ansible-pull functionality: ### Git Repos Tests (`git-repos-test.bats`) Tests the `git_repos` role `github_user` enumeration format: + - Role task and defaults files exist (including `clone_shim.yml`) - `github_user.yml` and `clone_shim.yml` task files are valid YAML - `main.yml` wires in the `github_user` sub-tasks @@ -143,11 +151,13 @@ Tests the `git_repos` role `github_user` enumeration format: ### GitHub Actions Workflow The CI workflow (`.github/workflows/ci.yml`) runs the test suite on: + - Push to `main` or `copilot/**` branches - Pull requests - Manual workflow dispatch Features: + - Automatic dependency installation - Test result publishing - Test artifacts upload @@ -156,6 +166,7 @@ Features: ### Test Results Test results are published using the `EnricoMi/publish-unit-test-result-action` which provides: + - Test summary in PR checks - Failed test details - Test trends over time @@ -222,6 +233,7 @@ The test runner will attempt to install Bats automatically. If this fails: ### Tests Fail Locally But Pass in CI This usually indicates environment differences. Common causes: + - Missing system packages - Different Python/Ansible versions - File permissions issues diff --git a/tests/bash/run-tests.sh b/tests/bash/run-tests.sh index 1b39123..da8dcf5 100755 --- a/tests/bash/run-tests.sh +++ b/tests/bash/run-tests.sh @@ -19,39 +19,39 @@ SPECIFIC_TESTS=() # Parse arguments while [[ $# -gt 0 ]]; do - case $1 in - --ci) - CI_MODE=true - OUTPUT_FORMAT="junit" - OUTPUT_FILE="test-results.xml" - shift - ;; - --output) - OUTPUT_FILE="$2" - shift 2 - ;; - --test) - SPECIFIC_TESTS+=("$2") - shift 2 - ;; - -h | --help) - echo "Usage: $0 [OPTIONS]" - echo "Run Bats tests for Ansible and Docker infrastructure" - echo "" - echo "Options:" - echo " --ci Run in CI mode (installs dependencies, exits on failure)" - echo " Automatically uses JUnit XML format for test results" - echo " --output FILE Output file for test results" - echo " (default: test-results.tap, or test-results.xml in CI mode)" - echo " --test FILE Run specific test file (can be used multiple times)" - echo " -h, --help Show this help message" - exit 0 - ;; - *) - echo "Unknown option: $1" - exit 1 - ;; - esac + case $1 in + --ci) + CI_MODE=true + OUTPUT_FORMAT="junit" + OUTPUT_FILE="test-results.xml" + shift + ;; + --output) + OUTPUT_FILE="$2" + shift 2 + ;; + --test) + SPECIFIC_TESTS+=("$2") + shift 2 + ;; + -h | --help) + echo "Usage: $0 [OPTIONS]" + echo "Run Bats tests for Ansible and Docker infrastructure" + echo "" + echo "Options:" + echo " --ci Run in CI mode (installs dependencies, exits on failure)" + echo " Automatically uses JUnit XML format for test results" + echo " --output FILE Output file for test results" + echo " (default: test-results.tap, or test-results.xml in CI mode)" + echo " --test FILE Run specific test file (can be used multiple times)" + echo " -h, --help Show this help message" + exit 0 + ;; + *) + echo "Unknown option: $1" + exit 1 + ;; + esac done echo -e "${BLUE}╔════════════════════════════════════════╗${NC}" @@ -61,60 +61,60 @@ echo "" # Verify tool versions match .mise.toml if command -v mise >/dev/null 2>&1; then - echo -e "${BLUE}🔍 Checking tool versions via mise...${NC}" - MISE_OUTDATED=$(mise ls --missing 2>/dev/null || true) - if [ -n "$MISE_OUTDATED" ]; then - echo -e "${YELLOW}âš ī¸ Some mise tools are missing or outdated:${NC}" - echo "$MISE_OUTDATED" - echo -e "${YELLOW} Running 'mise install' to sync...${NC}" - mise install 2>&1 - fi - echo -e "${GREEN}✅ Tool versions match .mise.toml${NC}" - mise ls 2>/dev/null - echo "" + echo -e "${BLUE}🔍 Checking tool versions via mise...${NC}" + MISE_OUTDATED=$(mise ls --missing 2>/dev/null || true) + if [ -n "$MISE_OUTDATED" ]; then + echo -e "${YELLOW}âš ī¸ Some mise tools are missing or outdated:${NC}" + echo "$MISE_OUTDATED" + echo -e "${YELLOW} Running 'mise install' to sync...${NC}" + mise install 2>&1 + fi + echo -e "${GREEN}✅ Tool versions match .mise.toml${NC}" + mise ls 2>/dev/null + echo "" else - echo -e "${YELLOW}âš ī¸ mise not found — tool versions not verified against .mise.toml${NC}" - echo -e "${YELLOW} Install mise: https://mise.jdx.dev/${NC}" - echo "" + echo -e "${YELLOW}âš ī¸ mise not found — tool versions not verified against .mise.toml${NC}" + echo -e "${YELLOW} Install mise: https://mise.jdx.dev/${NC}" + echo "" fi # Check if bats is installed if ! command -v bats >/dev/null 2>&1; then - echo -e "${YELLOW}đŸ“Ļ Bats not found, installing...${NC}" - - if [ "$CI_MODE" = true ]; then - # Install in CI mode - echo "Installing Bats from GitHub..." - BATS_VERSION="1.14.0" - git clone --depth 1 --branch "v${BATS_VERSION}" https://github.com/bats-core/bats-core.git /tmp/bats-core - cd /tmp/bats-core - sudo ./install.sh /usr/local - cd - - rm -rf /tmp/bats-core - else - # Try to install locally - if command -v npm >/dev/null 2>&1; then - echo "Installing Bats via npm..." - npm install -g bats - elif command -v brew >/dev/null 2>&1; then - echo "Installing Bats via Homebrew..." - brew install bats-core - else - echo -e "${RED}❌ Cannot install Bats automatically${NC}" - echo "Please install Bats manually:" - echo " - macOS: brew install bats-core" - echo " - Ubuntu/Debian: sudo apt-get install bats" - echo " - npm: npm install -g bats" - echo " - Manual: https://github.com/bats-core/bats-core#installation" - exit 1 - fi - fi - - # Verify installation - if ! command -v bats >/dev/null 2>&1; then - echo -e "${RED}❌ Bats installation failed${NC}" - exit 1 - fi + echo -e "${YELLOW}đŸ“Ļ Bats not found, installing...${NC}" + + if [ "$CI_MODE" = true ]; then + # Install in CI mode + echo "Installing Bats from GitHub..." + BATS_VERSION="1.14.0" + git clone --depth 1 --branch "v${BATS_VERSION}" https://github.com/bats-core/bats-core.git /tmp/bats-core + cd /tmp/bats-core + sudo ./install.sh /usr/local + cd - + rm -rf /tmp/bats-core + else + # Try to install locally + if command -v npm >/dev/null 2>&1; then + echo "Installing Bats via npm..." + npm install -g bats + elif command -v brew >/dev/null 2>&1; then + echo "Installing Bats via Homebrew..." + brew install bats-core + else + echo -e "${RED}❌ Cannot install Bats automatically${NC}" + echo "Please install Bats manually:" + echo " - macOS: brew install bats-core" + echo " - Ubuntu/Debian: sudo apt-get install bats" + echo " - npm: npm install -g bats" + echo " - Manual: https://github.com/bats-core/bats-core#installation" + exit 1 + fi + fi + + # Verify installation + if ! command -v bats >/dev/null 2>&1; then + echo -e "${RED}❌ Bats installation failed${NC}" + exit 1 + fi fi # Display Bats version @@ -128,7 +128,7 @@ TESTS_DIR="${SCRIPT_DIR}" # Make OUTPUT_FILE path absolute if it's relative if [[ ! "$OUTPUT_FILE" = /* ]]; then - OUTPUT_FILE="${SCRIPT_DIR}/${OUTPUT_FILE}" + OUTPUT_FILE="${SCRIPT_DIR}/${OUTPUT_FILE}" fi # Find all .bats test files @@ -136,30 +136,30 @@ echo -e "${BLUE}🔍 Discovering test files...${NC}" TEST_FILES=() if [ ${#SPECIFIC_TESTS[@]} -gt 0 ]; then - # Run specific tests - for test in "${SPECIFIC_TESTS[@]}"; do - if [ -f "${TESTS_DIR}/${test}" ]; then - TEST_FILES+=("${TESTS_DIR}/${test}") - echo " 📝 Found: $(basename "$test")" - elif [ -f "$test" ]; then - TEST_FILES+=("$test") - echo " 📝 Found: $(basename "$test")" - else - echo " ❌ Test not found: $test" - exit 1 - fi - done + # Run specific tests + for test in "${SPECIFIC_TESTS[@]}"; do + if [ -f "${TESTS_DIR}/${test}" ]; then + TEST_FILES+=("${TESTS_DIR}/${test}") + echo " 📝 Found: $(basename "$test")" + elif [ -f "$test" ]; then + TEST_FILES+=("$test") + echo " 📝 Found: $(basename "$test")" + else + echo " ❌ Test not found: $test" + exit 1 + fi + done else - # Run all tests - while IFS= read -r -d '' file; do - TEST_FILES+=("$file") - echo " 📝 Found: $(basename "$file")" - done < <(find "$TESTS_DIR" -name "*.bats" -print0) + # Run all tests + while IFS= read -r -d '' file; do + TEST_FILES+=("$file") + echo " 📝 Found: $(basename "$file")" + done < <(find "$TESTS_DIR" -name "*.bats" -print0) fi if [ ${#TEST_FILES[@]} -eq 0 ]; then - echo -e "${YELLOW}âš ī¸ No test files found in $TESTS_DIR${NC}" - exit 0 + echo -e "${YELLOW}âš ī¸ No test files found in $TESTS_DIR${NC}" + exit 0 fi echo "" @@ -168,72 +168,72 @@ echo "" # Run tests if [ "$CI_MODE" = true ]; then - # In CI mode, produce human-readable output on terminal AND JUnit XML for reporting - if [ "$OUTPUT_FORMAT" = "junit" ]; then - # Run with pretty terminal output and a separate JUnit report file - # --report-formatter writes JUnit XML to --output directory - REPORT_DIR="$(dirname "$OUTPUT_FILE")" - bats --formatter tap --report-formatter junit --output "$REPORT_DIR" "${TEST_FILES[@]}" - EXIT_CODE=$? - # Rename the report file to the expected output filename - if [ -f "${REPORT_DIR}/report.xml" ]; then - mv "${REPORT_DIR}/report.xml" "$OUTPUT_FILE" - fi - else - # Use TAP format - bats --tap "${TEST_FILES[@]}" 2>&1 | tee "$OUTPUT_FILE" - EXIT_CODE=${PIPESTATUS[0]} - fi + # In CI mode, produce human-readable output on terminal AND JUnit XML for reporting + if [ "$OUTPUT_FORMAT" = "junit" ]; then + # Run with pretty terminal output and a separate JUnit report file + # --report-formatter writes JUnit XML to --output directory + REPORT_DIR="$(dirname "$OUTPUT_FILE")" + bats --formatter tap --report-formatter junit --output "$REPORT_DIR" "${TEST_FILES[@]}" + EXIT_CODE=$? + # Rename the report file to the expected output filename + if [ -f "${REPORT_DIR}/report.xml" ]; then + mv "${REPORT_DIR}/report.xml" "$OUTPUT_FILE" + fi + else + # Use TAP format + bats --tap "${TEST_FILES[@]}" 2>&1 | tee "$OUTPUT_FILE" + EXIT_CODE=${PIPESTATUS[0]} + fi else - # In interactive mode, show output directly (no file needed) - bats "${TEST_FILES[@]}" - EXIT_CODE=$? + # In interactive mode, show output directly (no file needed) + bats "${TEST_FILES[@]}" + EXIT_CODE=$? fi # Display results based on exit code echo "" -if [ $EXIT_CODE -eq 0 ]; then - echo -e "${GREEN}╔════════════════════════════════════════╗${NC}" - echo -e "${GREEN}║ ✅ All Tests Passed! ║${NC}" - echo -e "${GREEN}╚════════════════════════════════════════╝${NC}" +if [ "$EXIT_CODE" -eq 0 ]; then + echo -e "${GREEN}╔════════════════════════════════════════╗${NC}" + echo -e "${GREEN}║ ✅ All Tests Passed! ║${NC}" + echo -e "${GREEN}╚════════════════════════════════════════╝${NC}" else - echo -e "${RED}╔════════════════════════════════════════╗${NC}" - echo -e "${RED}║ ❌ Some Tests Failed ║${NC}" - echo -e "${RED}╚════════════════════════════════════════╝${NC}" + echo -e "${RED}╔════════════════════════════════════════╗${NC}" + echo -e "${RED}║ ❌ Some Tests Failed ║${NC}" + echo -e "${RED}╚════════════════════════════════════════╝${NC}" fi # In CI mode, display full output and summary if [ "$CI_MODE" = true ]; then - echo "" - echo -e "${BLUE}📊 Test Results Summary:${NC}" - if [ -f "$OUTPUT_FILE" ]; then - if [ "$OUTPUT_FORMAT" = "junit" ]; then - echo " JUnit XML report generated: $OUTPUT_FILE" - # Display a simple summary from JUnit XML - if command -v xmllint >/dev/null 2>&1; then - # Validate XML first - if xmllint --noout "$OUTPUT_FILE" 2>/dev/null; then - TOTAL=$(xmllint --xpath "count(//testcase)" "$OUTPUT_FILE" 2>/dev/null || echo "0") - FAILURES=$(xmllint --xpath "count(//testcase/failure)" "$OUTPUT_FILE" 2>/dev/null || echo "0") - ERRORS=$(xmllint --xpath "count(//testcase/error)" "$OUTPUT_FILE" 2>/dev/null || echo "0") - echo " Total: $TOTAL, Failures: $FAILURES, Errors: $ERRORS" - else - echo " âš ī¸ Warning: XML report is malformed or empty. Check test execution for errors." - fi - else - echo " â„šī¸ xmllint not available for detailed summary" - fi - else - # TAP format summary - grep -E "^(ok|not ok)" "$OUTPUT_FILE" | sort | uniq -c || echo " No test results found" - fi - fi - - if [ "$OUTPUT_FORMAT" = "tap" ]; then - echo "" - echo -e "${BLUE}📋 Full Test Output:${NC}" - cat "$OUTPUT_FILE" - fi + echo "" + echo -e "${BLUE}📊 Test Results Summary:${NC}" + if [ -f "$OUTPUT_FILE" ]; then + if [ "$OUTPUT_FORMAT" = "junit" ]; then + echo " JUnit XML report generated: $OUTPUT_FILE" + # Display a simple summary from JUnit XML + if command -v xmllint >/dev/null 2>&1; then + # Validate XML first + if xmllint --noout "$OUTPUT_FILE" 2>/dev/null; then + TOTAL=$(xmllint --xpath "count(//testcase)" "$OUTPUT_FILE" 2>/dev/null || echo "0") + FAILURES=$(xmllint --xpath "count(//testcase/failure)" "$OUTPUT_FILE" 2>/dev/null || echo "0") + ERRORS=$(xmllint --xpath "count(//testcase/error)" "$OUTPUT_FILE" 2>/dev/null || echo "0") + echo " Total: $TOTAL, Failures: $FAILURES, Errors: $ERRORS" + else + echo " âš ī¸ Warning: XML report is malformed or empty. Check test execution for errors." + fi + else + echo " â„šī¸ xmllint not available for detailed summary" + fi + else + # TAP format summary + grep -E "^(ok|not ok)" "$OUTPUT_FILE" | sort | uniq -c || echo " No test results found" + fi + fi + + if [ "$OUTPUT_FORMAT" = "tap" ]; then + echo "" + echo -e "${BLUE}📋 Full Test Output:${NC}" + cat "$OUTPUT_FILE" + fi fi -exit $EXIT_CODE +exit "$EXIT_CODE"