From 37ca06ba3bd1510978a24e45e24c576139476008 Mon Sep 17 00:00:00 2001 From: Ani Date: Sun, 16 Aug 2026 22:54:00 +0000 Subject: [PATCH 1/2] Canon-only RAG: single-pass retrieval, canon DB, port config Pin the vector DB to the canon-filtered release and simplify the RAG path to a single-pass retrieve-rerank-generate flow, plus fix container port handling so the chat client and API agree. - Dockerfile: pin MEMORYALPHA_DB_RELEASE to v0.5.2 (Tier 2 strict canon-only DB; non-canon real-world pages dropped upstream). - rag.py: single-pass no-tools ask()/ask_stream() (retrieve -> rerank -> stuff -> generate) with real token counts; legacy tool loop kept behind ask_with_tools(). - ask.py: GET/POST /memoryalpha/rag/ask and /memoryalpha/rag/stream. - .env/docker-compose.yml/chat.sh: split container-internal APP_PORT (default 8000) from host-published API_PORT (default 18000) so the published mapping and uvicorn listen port cannot drift; chat.sh targets APP_PORT with a RAG_API_URL override. - wait-for-ollama.sh: fix empty-model handling when pulling DEFAULT_MODEL. - README: document endpoints, ports, and canon filtering. --- .env | 8 ++ Dockerfile | 2 +- README.md | 34 +++++---- api/memoryalpha/ask.py | 44 +++++++++-- api/memoryalpha/rag.py | 161 +++++++++++++++++++++++++++++++++-------- chat.sh | 12 ++- docker-compose.yml | 4 +- wait-for-ollama.sh | 13 +++- 8 files changed, 219 insertions(+), 59 deletions(-) diff --git a/.env b/.env index 0b97733..06c8307 100644 --- a/.env +++ b/.env @@ -1,5 +1,13 @@ DEFAULT_MODEL="qwen3:0.6b-q4_K_M" +# Port the API listens on *inside* the container. Clients running inside the +# container (e.g. chat.sh) should target this port. +APP_PORT="8000" + +# Host port to publish the REST API on. Mapped to APP_PORT in docker-compose. +# Overridable to avoid conflicts with other services on the host. +API_PORT="18000" + OLLAMA_URL="http://ollama:11434" DB_PATH="/data/enmemoryalpha_db" TEXT_COLLECTION_NAME="memoryalpha_text" \ No newline at end of file diff --git a/Dockerfile b/Dockerfile index f9353b9..ee1901a 100644 --- a/Dockerfile +++ b/Dockerfile @@ -11,7 +11,7 @@ RUN pip install --no-cache-dir -r /tmp/pip-tmp/requirements.txt \ WORKDIR /data -ARG MEMORYALPHA_DB_RELEASE=v0.5.0 +ARG MEMORYALPHA_DB_RELEASE=v0.5.2 RUN wget https://github.com/aniongithub/memoryalpha-vectordb/releases/download/${MEMORYALPHA_DB_RELEASE}/enmemoryalpha_db.tar.gz &&\ tar -xzf enmemoryalpha_db.tar.gz &&\ rm enmemoryalpha_db.tar.gz &&\ diff --git a/README.md b/README.md index 59b1793..8c784e5 100644 --- a/README.md +++ b/README.md @@ -11,10 +11,10 @@ This project provides a REST API that enables natural language queries over the The system implements: - **Retrieval-Augmented Generation (RAG)** for context-aware responses +- **Single-pass retrieval** (retrieve → cross-encoder rerank → stuff → generate) that works well with small local models — no tool-calling required - **Streaming responses** for real-time interaction - **Cross-encoder reranking** for improved document relevance - **Conversation history** for multi-turn dialogues -- **Thinking modes** (disabled/quiet/verbose) for different interaction styles ## Quick Start @@ -54,19 +54,25 @@ The system implements: ### API Endpoints - **Health Check:** `GET /memoryalpha/health` -- **Streaming Chat:** `GET /memoryalpha/rag/stream` +- **Ask (full response):** `GET` or `POST /memoryalpha/rag/ask` — returns the complete answer as JSON +- **Streaming Chat:** `GET` or `POST /memoryalpha/rag/stream` — streams the answer as `text/plain` chunks + +> The host port defaults to `8000` but is configurable via `API_PORT` in `.env` +> (the container always listens on `8000`). Adjust the URLs below to match your `API_PORT`. #### Example API Usage -* Streaming API +* Synchronous API (full JSON response) +```bash +curl "http://localhost:8000/memoryalpha/rag/ask?question=What%20is%20a%20Transporter?&max_tokens=512&top_k=10&top_p=0.8&temperature=0.3" +``` +* Streaming API (plain-text chunks) ```bash -curl -N -H "Accept: text/event-stream" \ - "http://localhost:8000/memoryalpha/rag/stream?question=What%20is%20the%20Enterprise?&thinkingmode=DISABLED&max_tokens=512&top_k=5" +curl -N "http://localhost:8000/memoryalpha/rag/stream?question=What%20is%20the%20Enterprise?&max_tokens=512&top_k=10" ``` -* Synchronous API +* Legacy tool-calling path (opt-in; unreliable with very small models) ```bash -curl -N -H "Accept: text/event-stream" \ - "http://localhost:8000/memoryalpha/rag/ask?question=What%20is%20a%20Transporter?&thinkingmode=VERBOSE&max_tokens=512&top_k=5&top_p=0.8&temperature=0.3" +curl "http://localhost:8000/memoryalpha/rag/ask?question=What%20is%20a%20Transporter?&use_tools=true" ``` ## Configuration @@ -78,14 +84,14 @@ The system uses the following environment variables (set in `.env`): ```env # Ollama Configuration OLLAMA_URL=http://ollama:11434 -DEFAULT_MODEL=qwen3:0.5b +DEFAULT_MODEL=qwen3:0.6b-q4_K_M -# Database Configuration +# Database Configuration DB_PATH=/data/enmemoryalpha_db -COLLECTION_NAME=memoryalpha +TEXT_COLLECTION_NAME=memoryalpha_text # API Configuration -THINKING_MODE=DISABLED +API_PORT=8000 MAX_TOKENS=2048 TOP_K=10 ``` @@ -93,11 +99,11 @@ TOP_K=10 ### Query Parameters - `question`: Your Star Trek question -- `thinkingmode`: `DISABLED`, `QUIET`, or `VERBOSE` - `max_tokens`: Maximum response length (default: 2048) - `top_k`: Number of documents to retrieve (default: 10) - `top_p`: Sampling parameter (default: 0.8) - `temperature`: Response creativity (default: 0.3) +- `use_tools`: Use the legacy tool-calling agent loop instead of single-pass RAG (default: `false`; `/ask` only) ## Development @@ -137,7 +143,7 @@ If you prefer local development without containers: 3. **Set up Ollama locally:** ```bash # Install Ollama (see https://ollama.ai) - ollama pull qwen3:0.5b + ollama pull qwen3:0.6b-q4_K_M ``` 4. **Download the MemoryAlpha database:** ```bash diff --git a/api/memoryalpha/ask.py b/api/memoryalpha/ask.py index 518927c..79a75c7 100644 --- a/api/memoryalpha/ask.py +++ b/api/memoryalpha/ask.py @@ -1,5 +1,5 @@ from fastapi import APIRouter, Query, Body -from fastapi.responses import JSONResponse +from fastapi.responses import JSONResponse, StreamingResponse from pydantic import BaseModel from typing import Optional @@ -16,6 +16,7 @@ class AskRequest(BaseModel): top_k: Optional[int] = 10 top_p: Optional[float] = 0.8 temperature: Optional[float] = 0.3 + use_tools: Optional[bool] = False @router.post("/memoryalpha/rag/ask") def ask_endpoint_post(request: AskRequest): @@ -29,7 +30,8 @@ def ask_endpoint_post(request: AskRequest): max_tokens=request.max_tokens, top_k=request.top_k, top_p=request.top_p, - temperature=request.temperature + temperature=request.temperature, + use_tools=request.use_tools, ) return JSONResponse(content=result) except Exception as e: @@ -41,11 +43,12 @@ def ask_endpoint( max_tokens: int = Query(2048, description="Maximum tokens to generate"), top_k: int = Query(10, description="Number of documents to retrieve"), top_p: float = Query(0.8, description="Sampling parameter"), - temperature: float = Query(0.3, description="Randomness/creativity of output") + temperature: float = Query(0.3, description="Randomness/creativity of output"), + use_tools: bool = Query(False, description="Use the legacy tool-calling agent loop instead of single-pass RAG"), ): """ Query the RAG pipeline and return the full response. - Now uses advanced tool-enabled RAG by default for better results. + Uses single-pass (no tool-calling) RAG by default; set use_tools=true for the legacy loop. """ try: result = rag_instance.ask( @@ -53,8 +56,39 @@ def ask_endpoint( max_tokens=max_tokens, top_k=top_k, top_p=top_p, - temperature=temperature + temperature=temperature, + use_tools=use_tools, ) return JSONResponse(content=result) except Exception as e: return JSONResponse(status_code=500, content={"error": str(e)}) + +@router.post("/memoryalpha/rag/stream") +def stream_endpoint_post(request: AskRequest): + """Stream the answer as it is generated (single-pass RAG). Returns text/plain chunks.""" + generator = rag_instance.ask_stream( + request.question, + max_tokens=request.max_tokens, + top_k=request.top_k, + top_p=request.top_p, + temperature=request.temperature, + ) + return StreamingResponse(generator, media_type="text/plain; charset=utf-8") + +@router.get("/memoryalpha/rag/stream") +def stream_endpoint( + question: str = Query(..., description="The user question"), + max_tokens: int = Query(2048, description="Maximum tokens to generate"), + top_k: int = Query(10, description="Number of documents to retrieve"), + top_p: float = Query(0.8, description="Sampling parameter"), + temperature: float = Query(0.3, description="Randomness/creativity of output"), +): + """Stream the answer as it is generated (single-pass RAG). Returns text/plain chunks.""" + generator = rag_instance.ask_stream( + question, + max_tokens=max_tokens, + top_k=top_k, + top_p=top_p, + temperature=temperature, + ) + return StreamingResponse(generator, media_type="text/plain; charset=utf-8") diff --git a/api/memoryalpha/rag.py b/api/memoryalpha/rag.py index 7d52beb..546d935 100644 --- a/api/memoryalpha/rag.py +++ b/api/memoryalpha/rag.py @@ -33,7 +33,7 @@ def get_user_prompt(context_text: str, query: str) -> str: INQUIRY: {query} -Accessing Starfleet database records. Provide analysis using ONLY the information in the records above. If the records don't contain the information needed to answer this inquiry, state that the information is not available in current records.""" +Using the records above, answer the inquiry in a single concise paragraph.""" class MemoryAlphaRAG: def __init__(self, @@ -115,10 +115,10 @@ def search(self, query: str, top_k: int = 10) -> List[Dict[str, Any]]: """Search the Memory Alpha database for relevant documents.""" try: - # Perform semantic search + # Perform semantic search, retrieving extra candidates for reranking. results = self.text_collection.query( query_texts=[query], - n_results=min(top_k * 2, 50) # Get more results for reranking + n_results=min(top_k * 2, 50) ) if not results["documents"] or not results["documents"][0]: @@ -133,30 +133,24 @@ def search(self, query: str, top_k: int = 10) -> List[Dict[str, Any]]: "distance": dist }) - # Re-rank using cross-encoder if available + # Re-rank the leading candidates with the cross-encoder if available. if self.cross_encoder and len(docs) > top_k: logger.info("Re-ranking results with cross-encoder") - # Limit to top candidates for re-ranking to avoid performance issues - rerank_candidates = docs[:min(len(docs), top_k + 5)] # Only re-rank top candidates - + rerank_candidates = docs[:min(len(docs), top_k + 5)] + # Prepare pairs for cross-encoder with truncated content - pairs = [] - for doc in rerank_candidates: - content = doc['content'] - if len(content) > 512: # Truncate long content for cross-encoder - content = content[:512] - pairs.append([query, content]) - + pairs = [[query, doc['content'][:512]] for doc in rerank_candidates] + try: scores = self.cross_encoder.predict(pairs) - + # Sort by cross-encoder scores (higher is better) ranked_docs = sorted(zip(rerank_candidates, scores), key=lambda x: x[1], reverse=True) reranked = [doc for doc, score in ranked_docs] - - # Replace original docs with re-ranked ones + + # Replace the reranked head, keep the remaining tail order. docs = reranked + docs[len(rerank_candidates):] - logger.info(f"Cross-encoder re-ranking completed, top score: {scores[0]:.4f}") + logger.info(f"Cross-encoder re-ranking completed, top score: {ranked_docs[0][1]:.4f}") except Exception as e: logger.warning(f"Cross-encoder re-ranking failed: {e}, using original ranking") # Continue with original docs if re-ranking fails @@ -169,18 +163,15 @@ def search(self, query: str, top_k: int = 10) -> List[Dict[str, Any]]: def build_prompt(self, query: str, docs: List[Dict[str, Any]]) -> tuple[str, str]: """Build the prompt with retrieved documents.""" - system_prompt = """You are an LCARS computer system with access to Star Trek Memory Alpha records. - -CRITICAL INSTRUCTIONS: -- You MUST answer ONLY using information from the provided records -- If the records don't contain relevant information, say "I don't have information about that in my records" -- DO NOT make up information, invent characters, or hallucinate details -- DO NOT use external knowledge about Star Trek - only use the provided records -- AVOID mirror universe references unless specifically asked about it -- If asked about something not in the records, be honest about the limitation -- Stay in character as an LCARS computer system at all times + system_prompt = """You are the LCARS computer of a Federation starship. Answer the inquiry using the Star Trek Memory Alpha records provided in the message. -Answer directly in a single paragraph.""" +- Treat the records below as your source of truth and summarize what they say to answer the inquiry. +- Only if the records are truly unrelated to the inquiry, reply: "I don't have information about that in my records." +- Do not add facts that are not supported by the records, and do not speculate. +- Report in-universe facts only. Never mention actors, episodes, writers, production, or that Star Trek is fictional. +- Ignore mirror-universe, alternate-timeline, novel, or comic material unless the inquiry is specifically about it. +- Do not mention "documents", "records", "context", or the search process in your answer. +- Answer in a single concise paragraph, in-character as the LCARS computer.""" if not docs: context_text = "" @@ -223,12 +214,120 @@ def search_tool(self, query: str, top_k: int = 5) -> str: logger.info(f"Formatted search result length: {len(formatted_result)}") return formatted_result + def _clean_answer(self, text: str) -> str: + """Strip ANSI codes and stray LCARS prefixes from a model response.""" + text = re.sub(r"\033\[[0-9;]*m", "", text) + return text.replace("LCARS: ", "").strip() + def ask(self, query: str, max_tokens: int = 2048, top_k: int = 10, top_p: float = 0.8, temperature: float = 0.3, - model: str = os.getenv("DEFAULT_MODEL")) -> Dict[str, Any]: + model: str = os.getenv("DEFAULT_MODEL"), use_tools: bool = False) -> Dict[str, Any]: """ - Ask a question using the advanced Memory Alpha RAG system with tool use. + Answer a question using single-pass RAG: retrieve -> rerank -> stuff -> generate. + + This is the default path and works well with small local models because it never + relies on the model to emit tool calls. Set use_tools=True to fall back to the + legacy tool-calling agent loop (ask_with_tools). Returns a dictionary with answer and token usage information. """ + if not model: + raise ValueError("model must be provided or set in DEFAULT_MODEL environment variable.") + + if use_tools: + return self.ask_with_tools(query, max_tokens=max_tokens, top_k=top_k, + top_p=top_p, temperature=temperature, model=model) + + logger.info(f"Starting single-pass RAG for query: {query}") + + # Retrieve + rerank using the raw user question (no model-invented keywords). + docs = self.search(query, top_k=top_k) + system_prompt, user_prompt = self.build_prompt(query, docs) + messages = [ + {"role": "system", "content": system_prompt}, + {"role": "user", "content": user_prompt}, + ] + + try: + result = self.ollama_client.chat( + model=model, + messages=messages, + stream=False, + think=False, + options={"temperature": temperature, "top_p": top_p, "num_predict": max_tokens}, + ) + except Exception as e: + logger.error(f"Chat failed: {e}") + return { + "answer": f"Error processing query: {str(e)}", + "token_usage": {"input_tokens": 0, "output_tokens": 0, "total_tokens": 0}, + } + + answer = self._clean_answer((result.get("message") or {}).get("content", "") or "") + if not answer: + answer = "I apologize, but I was unable to generate a response." + + # Real token counts reported by Ollama (not character estimates). + input_tokens = int(result.get("prompt_eval_count", 0) or 0) + output_tokens = int(result.get("eval_count", 0) or 0) + + self._update_history(query, answer) + return { + "answer": answer, + "token_usage": { + "input_tokens": input_tokens, + "output_tokens": output_tokens, + "total_tokens": input_tokens + output_tokens, + }, + } + + def ask_stream(self, query: str, max_tokens: int = 2048, top_k: int = 10, top_p: float = 0.8, + temperature: float = 0.3, model: str = os.getenv("DEFAULT_MODEL")): + """ + Single-pass RAG that streams the answer as it is generated. + + Yields plain-text chunks. Retrieval and prompt construction are identical to + ask(); only the generation step is streamed. + """ + if not model: + raise ValueError("model must be provided or set in DEFAULT_MODEL environment variable.") + + logger.info(f"Starting streaming single-pass RAG for query: {query}") + + docs = self.search(query, top_k=top_k) + system_prompt, user_prompt = self.build_prompt(query, docs) + messages = [ + {"role": "system", "content": system_prompt}, + {"role": "user", "content": user_prompt}, + ] + + collected: List[str] = [] + try: + for chunk in self.ollama_client.chat( + model=model, + messages=messages, + stream=True, + think=False, + options={"temperature": temperature, "top_p": top_p, "num_predict": max_tokens}, + ): + piece = (chunk.get("message") or {}).get("content", "") + if piece: + collected.append(piece) + yield piece + except Exception as e: + logger.error(f"Streaming chat failed: {e}") + yield f"\n[Error processing query: {str(e)}]" + return + + answer = self._clean_answer("".join(collected)) + if answer: + self._update_history(query, answer) + + def ask_with_tools(self, query: str, max_tokens: int = 2048, top_k: int = 10, top_p: float = 0.8, temperature: float = 0.3, + model: str = os.getenv("DEFAULT_MODEL")) -> Dict[str, Any]: + """ + Legacy: answer a question using the tool-calling agent loop. + Kept for comparison; unreliable with very small local models. Returns a dict + with answer and (estimated) token usage information. + """ if not model: raise ValueError("model must be provided or set in DEFAULT_MODEL environment variable.") diff --git a/chat.sh b/chat.sh index ce76fd7..f9670bf 100755 --- a/chat.sh +++ b/chat.sh @@ -1,10 +1,14 @@ #!/bin/bash # Interactive chat script for MemoryAlpha RAG API -BASE_URL="http://localhost:8000" -THINKING_MODE="VERBOSE" +# Load vars from .env. chat.sh runs *inside* the container, so it targets the +# container-internal APP_PORT (not the host-published API_PORT). +if [ -f .env ]; then set -a; . ./.env; set +a; fi +# Container listens on APP_PORT (default 8000). Set RAG_API_URL to override +# entirely (e.g. http://localhost:${API_PORT} when running from the host). +BASE_URL="${RAG_API_URL:-http://localhost:${APP_PORT:-8000}}" MAX_TOKENS=2048 -TOP_K=5 +TOP_K=10 TOP_P=0.8 TEMPERATURE=0.3 @@ -21,7 +25,7 @@ ask_question() { echo "----------------------------------------" local response response=$(curl -s \ - "${BASE_URL}/memoryalpha/rag/ask?question=${encoded_question}&thinkingmode=${THINKING_MODE}&max_tokens=${MAX_TOKENS}&top_k=${TOP_K}&top_p=${TOP_P}&temperature=${TEMPERATURE}") + "${BASE_URL}/memoryalpha/rag/ask?question=${encoded_question}&max_tokens=${MAX_TOKENS}&top_k=${TOP_K}&top_p=${TOP_P}&temperature=${TEMPERATURE}") # Check if response is valid JSON if ! echo "$response" | jq . >/dev/null 2>&1; then diff --git a/docker-compose.yml b/docker-compose.yml index 4ec8b99..5b57d57 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -22,12 +22,12 @@ services: - .:/workspace/memoryalpha-rag-api - model_cache:/root/.cache ports: - - "8000:8000" # for REST API + - "${API_PORT:-8000}:${APP_PORT:-8000}" # host API_PORT -> container APP_PORT networks: - odn env_file: - .env - entrypoint: ["./wait-for-ollama.sh", "uvicorn", "api.main:app", "--host", "0.0.0.0", "--port", "8000", "--log-level", "info"] + entrypoint: ["./wait-for-ollama.sh", "uvicorn", "api.main:app", "--host", "0.0.0.0", "--port", "${APP_PORT:-8000}", "--log-level", "info"] volumes: ollama_data: diff --git a/wait-for-ollama.sh b/wait-for-ollama.sh index b561af7..ae6bf37 100755 --- a/wait-for-ollama.sh +++ b/wait-for-ollama.sh @@ -9,18 +9,27 @@ echo "✅ Ollama is ready." pull_model() { local model_name="$1" + if [ -z "$model_name" ]; then + echo "⏭️ No model name provided, skipping." + return 0 + fi echo "🔍 Checking if model '$model_name' is available..." if curl -s "$OLLAMA_URL/api/tags" | grep -q "\"name\":\"$model_name\""; then echo "✅ Model '$model_name' is already available." else echo "📥 Model '$model_name' not found. Pulling it now..." - curl -X POST "$OLLAMA_URL/api/pull" -H "Content-Type: application/json" -d "{\"name\":\"$model_name\"}" + local response + response=$(curl -s -X POST "$OLLAMA_URL/api/pull" -H "Content-Type: application/json" -d "{\"name\":\"$model_name\"}") echo "" + if echo "$response" | grep -q '"error"'; then + echo "❌ Failed to pull model '$model_name': $response" + return 1 + fi echo "✅ Model '$model_name' has been pulled successfully." fi } -# Pull the default models +# Pull the default models (DEFAULT_IMAGE_MODEL is optional and skipped if unset) pull_model "$DEFAULT_MODEL" pull_model "$DEFAULT_IMAGE_MODEL" From 72ae0dd12c73f998cc6c90752597cd7f621bcc93 Mon Sep 17 00:00:00 2001 From: Ani Date: Sun, 16 Aug 2026 23:18:10 +0000 Subject: [PATCH 2/2] CI: honor configurable API_PORT in health/ask/openapi checks The committed .env now sets API_PORT=18000, and docker compose reads it for the host port mapping, so the published port is no longer always 8000. Source .env and target ${API_PORT:-8000} in the workflow curls (health readiness, ask endpoint, OpenAPI spec) across pr-check, ci-build, and release so CI tests the actually-published port. --- .github/workflows/ci-build.yml | 26 +++++++++++++++++--------- .github/workflows/pr-check.yml | 24 ++++++++++++++++-------- .github/workflows/release.yml | 9 ++++++--- 3 files changed, 39 insertions(+), 20 deletions(-) diff --git a/.github/workflows/ci-build.yml b/.github/workflows/ci-build.yml index 70c62e3..1f021b0 100644 --- a/.github/workflows/ci-build.yml +++ b/.github/workflows/ci-build.yml @@ -29,19 +29,25 @@ jobs: run: | # Start services in background docker compose up -d - + + # Host publishes on API_PORT (see .env / docker-compose.yml) + set -a; [ -f .env ] && . ./.env; set +a + PORT="${API_PORT:-8000}" + # Wait for services to be ready (max 5 minutes) - timeout 300 bash -c 'until curl -f http://localhost:8000/memoryalpha/health > /dev/null 2>&1; do sleep 5; echo "Waiting for API..."; done' - + timeout 300 bash -c "until curl -f http://localhost:${PORT}/memoryalpha/health > /dev/null 2>&1; do sleep 5; echo 'Waiting for API...'; done" + # Verify health endpoint - curl -f http://localhost:8000/memoryalpha/health - + curl -f "http://localhost:${PORT}/memoryalpha/health" + echo "✅ Health check passed" - + - name: Test ask endpoint run: | + set -a; [ -f .env ] && . ./.env; set +a + PORT="${API_PORT:-8000}" # Test the ask endpoint with a simple query - response=$(curl -X POST "http://localhost:8000/memoryalpha/rag/ask" -H "Content-Type: application/json" -d '{ + response=$(curl -X POST "http://localhost:${PORT}/memoryalpha/rag/ask" -H "Content-Type: application/json" -d '{ "question": "What is the color of Vulcan blood?" }') # Check if response contains expected content @@ -52,11 +58,13 @@ jobs: echo "Response: $response" exit 1 fi - + - name: Generate OpenAPI spec run: | + set -a; [ -f .env ] && . ./.env; set +a + PORT="${API_PORT:-8000}" # Download OpenAPI spec - curl -s http://localhost:8000/openapi.json -o memoryalpha-rag-api-spec.json + curl -s "http://localhost:${PORT}/openapi.json" -o memoryalpha-rag-api-spec.json cat memoryalpha-rag-api-spec.json - name: Cleanup diff --git a/.github/workflows/pr-check.yml b/.github/workflows/pr-check.yml index 318b214..7d5414e 100644 --- a/.github/workflows/pr-check.yml +++ b/.github/workflows/pr-check.yml @@ -28,19 +28,25 @@ jobs: run: | # Start services in background docker compose up -d - + + # Host publishes on API_PORT (see .env / docker-compose.yml) + set -a; [ -f .env ] && . ./.env; set +a + PORT="${API_PORT:-8000}" + # Wait for services to be ready (max 5 minutes) - timeout 300 bash -c 'until curl -f http://localhost:8000/memoryalpha/health > /dev/null 2>&1; do sleep 5; echo "Waiting for API..."; done' - + timeout 300 bash -c "until curl -f http://localhost:${PORT}/memoryalpha/health > /dev/null 2>&1; do sleep 5; echo 'Waiting for API...'; done" + # Verify health endpoint - curl -f http://localhost:8000/memoryalpha/health - + curl -f "http://localhost:${PORT}/memoryalpha/health" + echo "✅ Health check passed" - + - name: Test ask endpoint run: | + set -a; [ -f .env ] && . ./.env; set +a + PORT="${API_PORT:-8000}" # Test the ask endpoint with a simple query - response=$(curl -X POST "http://localhost:8000/memoryalpha/rag/ask" -H "Content-Type: application/json" -d '{ + response=$(curl -X POST "http://localhost:${PORT}/memoryalpha/rag/ask" -H "Content-Type: application/json" -d '{ "question": "What was the name of human who discovered warp drive?" }') # Check if response contains expected content @@ -54,8 +60,10 @@ jobs: - name: Generate OpenAPI spec run: | + set -a; [ -f .env ] && . ./.env; set +a + PORT="${API_PORT:-8000}" # Download OpenAPI spec - curl -s http://localhost:8000/openapi.json -o memoryalpha-rag-api-spec.json + curl -s "http://localhost:${PORT}/openapi.json" -o memoryalpha-rag-api-spec.json cat memoryalpha-rag-api-spec.json - name: Cleanup diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index e68586a..6e6d32e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -56,10 +56,13 @@ jobs: - name: Generate OpenAPI spec run: | docker compose up -d lcars + # Host publishes on API_PORT (see .env / docker-compose.yml) + set -a; [ -f .env ] && . ./.env; set +a + PORT="${API_PORT:-8000}" # Wait for API to be ready - timeout 120 bash -c 'until curl -f http://localhost:8000/memoryalpha/health > /dev/null 2>&1; do sleep 5; echo "Waiting for API..."; done' + timeout 120 bash -c "until curl -f http://localhost:${PORT}/memoryalpha/health > /dev/null 2>&1; do sleep 5; echo 'Waiting for API...'; done" # Download OpenAPI spec - curl -s http://localhost:8000/openapi.json -o memoryalpha-rag-api-spec.json + curl -s "http://localhost:${PORT}/openapi.json" -o memoryalpha-rag-api-spec.json docker compose down -v - name: Upload OpenAPI spec to release @@ -67,4 +70,4 @@ jobs: with: files: memoryalpha-rag-api-spec.json env: - GITHUB_TOKEN: ${{ secrets.MEMORYALPHA_RAG_API_MODIFY_RELEASE_TOKEN }} \ No newline at end of file + GITHUB_TOKEN: ${{ secrets.MEMORYALPHA_RAG_API_MODIFY_RELEASE_TOKEN }}