feat: read configuration from XDG base directory and new install instructions, etc

.gitattributes

@@ -0,0 +1,13 @@
+# Shell and helper scripts must use LF line endings on every platform.
+# On Windows, a CRLF line ending on the shebang line (e.g. "#!/usr/bin/env
+# bash\r") makes the interpreter lookup fail, so force LF regardless of the
+# user's core.autocrlf setting.
+*.sh        text eol=lf
+*.perl      text eol=lf
+
+# The launcher and its real implementation are extension-less / .sh scripts.
+ell         text eol=lf
+ell.sh      text eol=lf
+
+# Templates and configuration examples are line-oriented text; keep them LF.
+*.json      text eol=lf

README.md

@@ -20,32 +20,52 @@ A command-line interface for LLMs written in Bash.
 To use ell, you need the following:
 
 - bash-4.1 or later and coreutils / OS X utilities
-- jq (For parsing JSON)
 - curl (For sending HTTPS requests)
 - perl (Not necessary if you don't use record mode. For PCRE. POSIX bash doesn't support look-ahead and look-behind regular expressions)
 - util-linux (Not necessary if you don't use record mode. For `script` command to record terminal input and output)
 
 ## Install
 
-```
-git clone --depth 1 https://github.com/simonmysun/ell.git ~/.ellrc.d
-echo 'export PATH="${HOME}/.ellrc.d:${PATH}"' >> ~/.bashrc
+```bash
+git clone --depth 1 https://github.com/simonmysun/ell.git \
+  "${XDG_DATA_HOME:-$HOME/.local/share}/ell"
+echo 'export PATH="${XDG_DATA_HOME:-$HOME/.local/share}/ell:$PATH"' >> ~/.bashrc
 ```
 
 or
 
+```bash
+git clone --depth 1 git@github.com:simonmysun/ell.git \
+  "${XDG_DATA_HOME:-$HOME/.local/share}/ell"
+echo 'export PATH="${XDG_DATA_HOME:-$HOME/.local/share}/ell:$PATH"' >> ~/.bashrc
 ```
-git clone --depth 1 git@github.com:simonmysun/ell.git ~/.ellrc.d
-echo 'export PATH="${HOME}/.ellrc.d:${PATH}"' >> ~/.bashrc
-```
 
-This will clone the repository into `.ellrc.d` in your home directory and add it to your PATH. 
+This clones the repository into `${XDG_DATA_HOME:-$HOME/.local/share}/ell`
+(following the [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html))
+and adds it to your `PATH`. You may clone it anywhere you like; only the
+directory on your `PATH` matters.
+
+> **Upgrading from an older install?** The previous layout
+> (`git clone ... ~/.ellrc.d` with configuration in `~/.ellrc`) still works:
+> `~/.ellrc` is still read, and templates and plugins under `~/.ellrc.d` are
+> still picked up. No migration is required.
+
+### Windows
+
+ell is a Bash program, so run it from a Bash environment such as **Git Bash**,
+**MSYS2**, **Cygwin** or **WSL**.
+
+The `ell` command is a plain wrapper script (not a symbolic link), so it works
+even when git does not create symlinks on checkout — the default on Windows
+unless Developer Mode or administrator rights are available. No extra setup is
+required; clone the repository and add its directory to your `PATH` as shown
+above. You invoke it the same way, e.g. `ell "your prompt"`.
 
 ## Configuration
 
 See [Configuration](docs/Configuration.md).
 
-Here's an example configuration to use `gemini-1.5-flash` from Google. You need to set these variables in your `~/.ellrc`:
+Here's an example configuration to use `gemini-1.5-flash` from Google. You need to set these variables in your config file, `${XDG_CONFIG_HOME:-$HOME/.config}/ell/config` (the legacy `~/.ellrc` also still works):
 
 ```ini
 ELL_API_STYLE=gemini
@@ -173,6 +193,23 @@ See [Risks Consideration](docs/Risk_Consideration.md).
 - https://github.com/sigoden/aichat A CLI tool talks to various LLM providers, written in Rust.
 - https://github.com/npiv/chatblade A CLI Swiss Army Knife for ChatGPT, written in python
 
+## Testing
+
+The test suite is meant to be run inside Docker. The tests hard-code the
+container path `/ell` (the repository is mounted there read-only) and rely on
+a clean, predictable environment, so running them directly on your host is not
+supported and will fail (e.g. templates are looked up under `/ell/templates/`).
+
+Run the full suite against the supported Bash versions with:
+
+```bash
+bash tests/docker.sh
+```
+
+This mounts the repository into `bash:4.1` and `bash:5.2` containers and runs
+`tests/entry.sh`, which installs the runtime dependencies and executes every
+test (`logging`, `piping`, `templating`, `parse_output` and `redaction`).
+
 ## Contributing
 
 Contributions are welcome! If you have any ideas, suggestions, or bug reports, please open an issue or submit a pull request.

docs/Configuration.md

@@ -8,9 +8,11 @@ ell can be configured in three ways (in order of precedence, from lowest to high
 - environment variables
 - command line arguments
 
-The configuration files are read and applied in the following order:
+The configuration files are read and applied in the following order (later
+files override earlier ones):
 
-- `~/.ellrc`
+- `${XDG_CONFIG_HOME:-$HOME/.config}/ell/config` (the main config, following the [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html))
+- `~/.ellrc` (legacy location, still read for backward compatibility)
 - `.ellrc` in the current directory
 - `$ELL_CONFIG` specified in the environment variables or command line arguments.
 
@@ -23,11 +25,11 @@ If you are running ell in a relatively hostile environment, it is recommended to
 The following variables can be set in the configuration files, environment variables:
 
 - `ELL_LOG_LEVEL`: The log level of the logger. The default is `2`. A log level of `0` will log everything. A log level of `3` will log token usage.
-- `ELL_CONFIG`: The configuration file to use. The default is `~/.ellrc`.
+- `ELL_CONFIG`: An extra configuration file to load, applied last (highest precedence among files). Unset by default. The standard config files (`${XDG_CONFIG_HOME:-$HOME/.config}/ell/config`, `~/.ellrc`, `./.ellrc`) are always read regardless of this variable.
 - `ELL_LLM_MODEL`: The model to use. Default is `gpt-4o-mini`.
 - `ELL_LLM_TEMPERATURE`: The temperature of the model. The default is `0.6`.
 - `ELL_LLM_MAX_TOKENS`: The maximum number of tokens to generate. The default is `4096`.
-- `ELL_TEMPLATE_PATH`: The path to the templates. The default is `~/.ellrc.d/templates`.
+- `ELL_TEMPLATE_PATH`: Force templates to be loaded from this single directory (note the trailing slash, e.g. `/path/to/templates/`). When unset (the default), templates are searched, in order, under `${XDG_CONFIG_HOME:-$HOME/.config}/ell/templates/`, `${XDG_DATA_HOME:-$HOME/.local/share}/ell/templates/`, `~/.ellrc.d/templates/` (legacy) and the `templates/` directory bundled with ell. This lets you override a bundled template by placing a file with the same name under your XDG config directory.
 - `ELL_TEMPLATE`: The template to use. The default is `default`. The file extension is not needed.
 - `ELL_INPUT_FILE`: The input file to use. If specified, it will override the prompt given in command line arguments. Setting this to `-` will let ell always read from stdin. 
 - `ELL_RECORD`: This is used for controlling whether record mode is on. It should be set to `false` unless you want to disable recording.  

docs/Plugins.md

@@ -24,11 +24,22 @@ Ell supports plugins to extend its functionality through a hook system. Currentl
 - `post_llm`: Called after the response is received and decoded from the language model.
 - `pre_output`: Called before the output is sent to the user.
 
-Plugins should be placed in the `./plugins` directory related to the ell script, typically located at `~/.ellrc.d/plugins` if you follow the installation instructions in the readme. 
+Plugins are discovered from the `plugins/` directory under each of the
+following roots, searched in this priority order:
 
-Each plugin should be a folder containing executable shell scripts. The file name should follow the format `XX_${HOOK_NAME}.sh`, where `XX` is a number that determines the execution order among other plugins. For example, the paginator plugin is placed in `~/.ellrc.d/plugins/paginator/90_pre_output.sh`.
+- `${XDG_CONFIG_HOME:-$HOME/.config}/ell/plugins/` (your own plugins)
+- `${XDG_DATA_HOME:-$HOME/.local/share}/ell/plugins/`
+- `~/.ellrc.d/plugins/` (legacy location, still supported)
+- the `plugins/` directory bundled with ell (built-in plugins)
 
-Plugin scripts are executed in ascending numerical order and piped to each other.
+If the same plugin hook (identified by `<plugin-dir>/<hook-file>`) exists in
+more than one root, only the highest-priority copy runs, so you can drop a
+plugin with the same name under your XDG config directory to override a
+built-in one.
+
+Each plugin should be a folder containing executable shell scripts. The file name should follow the format `XX_${HOOK_NAME}.sh`, where `XX` is a number that determines the execution order among other plugins. For example, the built-in paginator plugin is placed in `plugins/paginator/90_pre_output.sh` (so a user copy would live at `${XDG_CONFIG_HOME:-$HOME/.config}/ell/plugins/paginator/90_pre_output.sh`).
+
+Plugin scripts are executed in ascending numerical order (across all roots, ordered by `<plugin-dir>/<hook-file>`) and piped to each other.
 
 It is recommended to write plugins in a streaming manner.
 

ell

@@ -0,0 +1,28 @@
+#!/usr/bin/env bash
+
+# Thin launcher for ell.
+#
+# This file used to be a symbolic link to ell.sh, but git symlinks are not
+# checked out as links on Windows (unless core.symlinks is enabled, which
+# usually requires Developer Mode or administrator rights). There they become
+# plain text files containing the link target, which cannot be executed.
+#
+# A real wrapper script works identically on Linux, macOS and Windows (Git
+# Bash / MSYS2 / Cygwin / WSL), so `ell` stays a valid, extension-less command
+# everywhere. It simply resolves its own directory and execs the real script.
+
+# Resolve the directory this launcher lives in, following any symlinks that
+# may still exist (e.g. a link created in ~/.local/bin on Unix).
+SOURCE="${BASH_SOURCE[0]:-${0}}";
+while [ -h "${SOURCE}" ]; do
+  DIR="$(cd -P "$(dirname "${SOURCE}")" >/dev/null 2>&1 && pwd)";
+  SOURCE="$(readlink "${SOURCE}")";
+  # If the link target is relative, resolve it against the link's directory.
+  case "${SOURCE}" in
+    /*) ;;
+    *) SOURCE="${DIR}/${SOURCE}"; ;;
+  esac
+done
+ELL_DIR="$(cd -P "$(dirname "${SOURCE}")" >/dev/null 2>&1 && pwd)";
+
+exec "${ELL_DIR}/ell.sh" "${@}";

ell.sh

@@ -22,6 +22,8 @@ BASE_DIR=$(dirname "${0}");
 . "${BASE_DIR}/helpers/parse_arguments.sh";
 . "${BASE_DIR}/helpers/load_config.sh";
 . "${BASE_DIR}/helpers/piping.sh";
+. "${BASE_DIR}/helpers/json.sh";
+. "${BASE_DIR}/helpers/resolve_paths.sh";
 
 logging_debug "Starting ${0}";
 
@@ -37,7 +39,9 @@ load_config;
 : "${ELL_LLM_MODEL:=gpt-4o-mini}";
 : "${ELL_LLM_TEMPERATURE:=0.6}";
 : "${ELL_LLM_MAX_TOKENS:=4096}";
-: "${ELL_TEMPLATE_PATH:="${HOME}/.ellrc.d/templates/"}";
+# ELL_TEMPLATE_PATH is intentionally left unset by default: templates are
+# resolved through resolve_template() across the XDG and bundled search roots.
+# Setting it explicitly (e.g. via -T) forces that single directory instead.
 : "${ELL_TEMPLATE:=default-openai}";
 : "${ELL_INPUT_FILE:=""}";
 : "${ELL_RECORD:="false"}";
@@ -83,9 +87,9 @@ export COLUMNS;
 # Logging_debug "Decorating the generate_completion to apply hooks before and after";
 eval "$(printf "orig_"; command -V generate_completion | tail -n +2)";
 generate_completion() {
-  pre_llm_hooks=$(ls ${BASE_DIR}/plugins/*/*_pre_llm.sh 2>/dev/null | sort -k3 -t/);
+  pre_llm_hooks=$(list_plugin_hooks _pre_llm.sh);
   logging_debug "Pre LLM hooks: ${pre_llm_hooks}";
-  post_llm_hooks=$(ls ${BASE_DIR}/plugins/*/*_post_llm.sh 2>/dev/null | sort -k3 -t/);
+  post_llm_hooks=$(list_plugin_hooks _post_llm.sh);
   logging_debug "Post LLM hooks: ${post_llm_hooks}";
   piping "${pre_llm_hooks[@]}" \
   | orig_generate_completion \
@@ -116,11 +120,17 @@ if [ "x${ELL_RECORD}" = "xtrue" ] || [ "x${ELL_INTERACTIVE}" = "xtrue" ] && [ "x
   exit 0;
 fi
 
-# Logging_debug "Checking if the template is available";
-if [ ! -f "${ELL_TEMPLATE_PATH}${ELL_TEMPLATE}.json" ]; then
-  logging_fatal "Template not found: ${ELL_TEMPLATE_PATH}${ELL_TEMPLATE}.json";
+# Logging_debug "Resolving the template across the search roots";
+ELL_TEMPLATE_FILE="$(resolve_template "${ELL_TEMPLATE}")";
+if [ -z "${ELL_TEMPLATE_FILE}" ]; then
+  if [ -n "${ELL_TEMPLATE_PATH}" ]; then
+    logging_fatal "Template not found: ${ELL_TEMPLATE_PATH}${ELL_TEMPLATE}.json";
+  else
+    logging_fatal "Template not found: ${ELL_TEMPLATE}.json (searched XDG config/data, ~/.ellrc.d and ${BASE_DIR})";
+  fi
   exit 1;
 fi
+logging_debug "Using template: ${ELL_TEMPLATE_FILE}";
 
 # Logging_debug "Checking if we are going to read from a file";
 if [ -n "${ELL_INPUT_FILE}" ]; then
@@ -142,9 +152,9 @@ else
 fi
 
 # Logging_debug "Loading the post_input and pre_output hooks";
-post_input_hooks=$(ls ${BASE_DIR}/plugins/*/*_post_input.sh 2>/dev/null | sort -k3 -t/);
+post_input_hooks=$(list_plugin_hooks _post_input.sh);
 logging_debug "Post input hooks: ${post_input_hooks}";
-pre_output_hooks=$(ls ${BASE_DIR}/plugins/*/*_pre_output.sh 2>/dev/null | sort -k3 -t/);
+pre_output_hooks=$(list_plugin_hooks _pre_output.sh);
 logging_debug "Pre output hooks: ${pre_output_hooks}";
 
 # Logging_debug "Checking if we are going to enter interactive mode";
@@ -161,7 +171,7 @@ if [ "x${ELL_INTERACTIVE}" = "xtrue" ]; then
       export SHELL_CONTEXT="$(tail -c 3000 "${ELL_TMP_SHELL_LOG}" | "${BASE_DIR}/helpers/render_to_text.perl" | sed  -e 's/\\/\\\\/g' -e 's/"/\\"/g'| awk '{printf "%s\\n", $0}')";
     fi
     PAYLOAD="$(eval "cat <<EOF
-$(cat "${ELL_TEMPLATE_PATH}${ELL_TEMPLATE}.json")
+$(cat "${ELL_TEMPLATE_FILE}")
 EOF")";
     printf "%s" "${ELL_PS2}";
     echo "${PAYLOAD}" | generate_completion | piping "${pre_output_hooks[@]}";
@@ -171,7 +181,7 @@ else
   USER_PROMPT=$(echo "${USER_PROMPT}" | piping "${post_input_hooks[@]}");
 
   PAYLOAD="$(eval "cat <<EOF
-$(cat "${ELL_TEMPLATE_PATH}${ELL_TEMPLATE}.json")
+$(cat "${ELL_TEMPLATE_FILE}")
 EOF")";
 
   echo "${PAYLOAD}" | generate_completion | piping "${pre_output_hooks[@]}";