Skip to content
Merged
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@
[![Pydantic v2](https://img.shields.io/badge/Contract-Pydantic%20v2-e92063.svg)](https://docs.pydantic.dev/)
[![Database](https://img.shields.io/badge/Database-SQLite%20%2B%20PostgreSQL-4169E1.svg)](https://www.sqlite.org/)
[![Pytest](https://img.shields.io/badge/Backend%20Tests-100%20Passed-brightgreen.svg)](https://pytest.org)
[![XCTest](https://img.shields.io/badge/iOS%20Tests-22%20Passed-brightgreen.svg)](https://developer.apple.com/xcode/)
[![XCTest](https://img.shields.io/badge/iOS%20Tests-28%20Passed-brightgreen.svg)](https://developer.apple.com/xcode/)
[![Figma Prototype](https://img.shields.io/badge/Figma-100%25%20Prototype-pink.svg)](https://www.figma.com/proto/fable-ios-prototype-midterm)

**Fable** is a native iOS literary reading and community publishing platform with a companion FastAPI REST backend, engineered as a distraction-free sanctuary for **open-source classic books, world folklore, timeless fables, and original community manuscripts**. Fable couples a refined editorial aesthetic with an offline-first, contract-first system architecture: the live catalog returns provider metadata and official covers, while the reader fetches chapter text when a book is opened and caches it for offline reading.
**Fable** is a native iOS literary reading and community publishing platform with a companion FastAPI REST backend, engineered as a distraction-free sanctuary for **federated open-source classic books (Standard Ebooks & Project Gutenberg), world folklore, timeless fables, and original community manuscripts**. Fable couples a refined editorial aesthetic with an offline-first, contract-first system architecture: the live catalog returns federated provider metadata and official covers without downloading every book's text, while the reader fetches chapter text on demand when a book is opened and caches it for offline reading.

> [!NOTE]
> **Open-Source Literature Focus (`StoryStore.isMangaEnabled = false`):** Manga remains disabled by default on iOS while chapter continuity and rights are unverified (see [`ADR 0002`](docs/adr/0002-ios-open-source-literature-focus-and-manga-feature-flag.md) and [`docs/DATA_SOURCES_AND_API_STRATEGY.md`](docs/DATA_SOURCES_AND_API_STRATEGY.md)).
Expand All @@ -22,10 +22,11 @@

| Document | Purpose |
|---|---|
| **[`docs/AGENT_HANDOFF_CONTEXT.md`](docs/AGENT_HANDOFF_CONTEXT.md)** | Architecture notes, current implementation boundaries, and verification guidance |
| **[`docs/AGENT_HANDOFF_CONTEXT.md`](docs/AGENT_HANDOFF_CONTEXT.md)** | Architecture notes, active branch milestones (`feat/federated-book-sources`), and verification guidance |
| **[`docs/DATA_SOURCES_AND_API_STRATEGY.md`](docs/DATA_SOURCES_AND_API_STRATEGY.md)** | Federated provider metadata, Standard Ebooks & Gutenberg endpoints, on-demand chapter loading, and edition identity |
| **[`docs/simulator_captures/README.md`](docs/simulator_captures/README.md)** | **29 Retina iOS Simulator Screenshots** + **Full Walkthrough Video (`fable_ios_full_walkthrough.mp4`)** + Federated Endpoint Index |
| **[`docs/presentation/08_FEATURE_CATALOG_AND_USER_JOURNEYS.md`](docs/presentation/08_FEATURE_CATALOG_AND_USER_JOURNEYS.md)** | Complete functional feature catalog, user journey state machines, and federated discovery flows |
| **[`docs/plans/06_SECURITY_VULNERABILITIES_AND_ARCHITECTURE_REMEDIATION.md`](docs/plans/06_SECURITY_VULNERABILITIES_AND_ARCHITECTURE_REMEDIATION.md)** | **Plan 06:** 4-Phase Security & Architecture Hardening Plan (`SEC-01`–`SEC-04`, `PERF-01`–`PERF-03`, `DOM-01`–`DOM-02`, `IOS-01`–`IOS-02`) |
| **[`docs/simulator_captures/README.md`](docs/simulator_captures/README.md)** | **29 Retina iOS Simulator Screenshots** across every screen/filter/modal + **Full Walkthrough Video (`fable_ios_full_walkthrough.mp4`)** |
| **[`docs/DATA_SOURCES_AND_API_STRATEGY.md`](docs/DATA_SOURCES_AND_API_STRATEGY.md)** | Live provider metadata, official covers, on-demand book chapters, and upstream source limits |
| **[`docs/RENDER_NEON_DEPLOYMENT.md`](docs/RENDER_NEON_DEPLOYMENT.md)** | Free-tier Render + Neon deployment setup, limits, secrets, and operator steps |
| **[`docs/adr/`](docs/adr/)** | Architecture Decision Records: [`ADR 0001`](docs/adr/0001-password-hash-versioning.md) (Password Hashing), [`ADR 0002`](docs/adr/0002-ios-open-source-literature-focus-and-manga-feature-flag.md) (Manga Feature Flag), [`ADR 0003`](docs/adr/0003-comprehensive-catalog-seeding-and-chapter-1-continuity.md) (Catalog Seeding), [`ADR 0004`](docs/adr/0004-render-neon-postgres-deployment.md) (Render + Neon) |
| **[`docs/final_milestone/MASTER_PROGRESS_LOG.md`](docs/final_milestone/MASTER_PROGRESS_LOG.md)** | Chronological engineering progress log across all phases and PRs |
Expand Down
14 changes: 11 additions & 3 deletions backend/api/v1/endpoints/standard_ebooks.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,24 @@

@router.get("/public/standard-ebooks", response_model=list[StoryDTO])
async def get_standard_ebooks_stories(
topic: Optional[str] = Query(default=None, description="Filter Standard Ebooks by subject (e.g. fiction, horror)"),
search: Optional[str] = Query(default=None, description="Search keyword for title or author"),
topic: Optional[str] = Query(default=None, max_length=50, description="Filter Standard Ebooks by subject (e.g. fiction, horror)"),
search: Optional[str] = Query(default=None, max_length=200, description="Search keyword for title or author"),
):
"""Retrieve curated public-domain classics from Standard Ebooks."""
return await standard_ebooks_service.get_standard_ebooks_stories(topic=topic, search=search)


@router.get("/public/standard-ebooks/story", response_model=StoryDTO)
async def get_standard_ebooks_story(
book_id: str = Query(..., max_length=120, description="Standard Ebooks slug path (e.g. bram-stoker/dracula)"),
):
"""Retrieve metadata for a specific Standard Ebooks work."""
return await standard_ebooks_service.get_standard_ebooks_story_by_id(book_id=book_id)


@router.get("/public/standard-ebooks/chapters", response_model=list[ChapterDTO])
async def get_standard_ebooks_chapters(
book_id: str = Query(..., description="Standard Ebooks slug path (e.g. bram-stoker/dracula)"),
book_id: str = Query(..., max_length=120, description="Standard Ebooks slug path (e.g. bram-stoker/dracula)"),
):
"""Retrieve structured semantic XHTML chapters for a Standard Ebooks work."""
return await standard_ebooks_service.get_standard_ebooks_chapters(book_id=book_id)
13 changes: 11 additions & 2 deletions backend/services/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,12 @@
)
from .shelf_sync import sync_shelf, get_shelf
from .gutenberg import ingest_gutenberg_book, get_gutenberg_stories
from . import auth_service, reading_stats, manga
from .standard_ebooks import (
get_standard_ebooks_stories,
get_standard_ebooks_story_by_id,
get_standard_ebooks_chapters,
)
from . import auth_service, reading_stats, manga, standard_ebooks

__all__ = [
"extract_chapters_from_text",
Expand All @@ -29,8 +34,12 @@
"get_shelf",
"ingest_gutenberg_book",
"get_gutenberg_stories",
"get_standard_ebooks_stories",
"get_standard_ebooks_story_by_id",
"get_standard_ebooks_chapters",
"auth_service",
"reading_stats",
"manga"
"manga",
"standard_ebooks",
]

111 changes: 107 additions & 4 deletions backend/services/standard_ebooks.py
Original file line number Diff line number Diff line change
Expand Up @@ -457,20 +457,24 @@ async def get_standard_ebooks_stories(
if normalized_search:
query = urlencode({"query": normalized_search, "page": 1, "per-page": 12})
response = await client.get(f"{STANDARD_EBOOKS_BASE_URL}/ebooks?{query}")
if response.status_code == 200:
entries = parse_listing_xhtml(response.content)
if response.status_code != 200:
raise HTTPException(status_code=502, detail="Standard Ebooks catalog service is unavailable")
entries = parse_listing_xhtml(response.content)
elif normalized_topic:
slug_topic = quote(normalized_topic.replace(" ", "-"), safe="-")
response = await client.get(
f"{STANDARD_EBOOKS_BASE_URL}/subjects/{slug_topic}?sort=popularity&page=1&per-page=12"
)
if response.status_code == 200:
entries = parse_listing_xhtml(response.content)
elif response.status_code != 404:
raise HTTPException(status_code=502, detail="Standard Ebooks catalog service is unavailable")
if not entries:
query = urlencode({"query": normalized_topic, "sort": "popularity", "page": 1, "per-page": 12})
fallback_resp = await client.get(f"{STANDARD_EBOOKS_BASE_URL}/ebooks?{query}")
if fallback_resp.status_code == 200:
entries = parse_listing_xhtml(fallback_resp.content)
if fallback_resp.status_code != 200:
raise HTTPException(status_code=502, detail="Standard Ebooks catalog service is unavailable")
entries = parse_listing_xhtml(fallback_resp.content)
else:
response = await client.get(f"{STANDARD_EBOOKS_BASE_URL}/ebooks?sort=popularity&page=1&per-page=12")
if response.status_code == 200:
Expand Down Expand Up @@ -530,6 +534,105 @@ async def get_standard_ebooks_stories(
return list(stories)


def _humanize_slug_part(value: str) -> str:
return " ".join(part.capitalize() for part in value.split("-") if part)


def _parse_book_page_story(xml_bytes: bytes, slug: str) -> StoryDTO:
try:
root = ET.fromstring(xml_bytes)
except ET.ParseError as exc:
raise HTTPException(status_code=502, detail="Invalid book metadata from Standard Ebooks") from exc

slug_parts = slug.split("/")
fallback_author = _humanize_slug_part(slug_parts[0]) if slug_parts else "Standard Ebooks"
fallback_title = _humanize_slug_part(slug_parts[1]) if len(slug_parts) > 1 else slug

title = ""
author_names: list[str] = []
cover_url: Optional[str] = None

for elem in root.iter():
prop = elem.attrib.get("property", "")
tag = _local_tag(elem.tag)
if not title and prop == "schema:name" and tag in ("h1", "span", "p", "a"):
candidate = _collapse_whitespace("".join(elem.itertext()))
if candidate:
title = candidate
if prop == "schema:author":
name_el = elem.find('.//*[@property="schema:name"]')
author_text = _collapse_whitespace(
"".join((name_el if name_el is not None else elem).itertext())
)
if author_text and author_text not in author_names:
author_names.append(author_text)
if cover_url is None and tag == "img":
src = (elem.attrib.get("src") or "").strip()
if "cover" in src:
if src.startswith("/"):
cover_url = f"{STANDARD_EBOOKS_BASE_URL}{src}"
elif src.startswith("https://"):
cover_url = src

resolved_title = (title or fallback_title)[:120]
resolved_author = (", ".join(author_names) if author_names else fallback_author)[:80]
synopsis, genre, read_minutes = parse_book_detail_xhtml(xml_bytes, fallback_title=resolved_title)
now = datetime.now(timezone.utc)
return StoryDTO(
id=uuid5(STANDARD_EBOOKS_NAMESPACE, f"book:{slug}"),
title=resolved_title,
author=resolved_author,
genre=genre,
chapter="",
synopsis=synopsis,
content="",
read_time_minutes=read_minutes,
is_bookmarked=False,
is_completed=False,
created_at_utc=now,
updated_at_utc=now,
cover_image_url=cover_url,
total_pages=0,
current_page=1,
progress_percent=0,
saves_count="0",
reads_count="0",
badge_text=None,
total_chapters=0,
content_format="PROSE",
source_provider="STANDARD_EBOOKS",
provider_id=slug,
provider_download_count=None,
chapters=None,
)


async def get_standard_ebooks_story_by_id(book_id: str) -> StoryDTO:
"""Fetches metadata for a single Standard Ebooks work by its slug path."""
slug = _normalize_slug(book_id)
now = time.monotonic()
for cached_at, cached_stories in _STANDARD_EBOOKS_CATALOG_CACHE.values():
if (now - cached_at) < STANDARD_EBOOKS_CATALOG_TTL_SECONDS:
for story in cached_stories:
if story.provider_id == slug:
return story

url = f"{STANDARD_EBOOKS_BASE_URL}/ebooks/{slug}"
try:
async with _standard_ebooks_client() as client:
response = await client.get(url)
except httpx.HTTPError as exc:
logger.warning("Standard Ebooks detail fetch failed for %s: %s", slug, exc)
raise HTTPException(status_code=502, detail="Standard Ebooks catalog service is unavailable") from exc

if response.status_code == 404:
raise HTTPException(status_code=404, detail="Standard Ebooks title not found")
if response.status_code != 200:
raise HTTPException(status_code=502, detail="Standard Ebooks catalog service is unavailable")

return _parse_book_page_story(response.content, slug)


async def get_standard_ebooks_chapters(book_id: str) -> list[ChapterDTO]:
"""Fetches and parses the semantic XHTML chapter tree for a Standard Ebooks work."""
slug = _normalize_slug(book_id)
Expand Down
123 changes: 123 additions & 0 deletions backend/test_provider_http.py
Original file line number Diff line number Diff line change
Expand Up @@ -405,3 +405,126 @@ async def mock_stream(self, method, url, headers=None, **kwargs):
params={"book_id": "../etc/passwd"},
)
assert bad_resp.status_code == 400


def test_standard_ebooks_story_detail_http_endpoint_returns_metadata_and_handles_missing(monkeypatch):
from uuid import uuid5
import httpx
from services import standard_ebooks

monkeypatch.setattr(standard_ebooks, "_STANDARD_EBOOKS_CATALOG_CACHE", {})
monkeypatch.setattr(standard_ebooks, "_STANDARD_EBOOKS_DETAIL_CACHE", {})

book_xml = b"""<?xml version="1.0" encoding="utf-8"?>
<html xmlns="http://www.w3.org/1999/xhtml">
<body>
<article typeof="schema:Book">
<h1 property="schema:name">Dracula</h1>
<p property="schema:author"><span property="schema:name">Bram Stoker</span></p>
<img src="/images/covers/bram-stoker_dracula/hash/cover@2x.jpg" alt="Cover"/>
<section property="schema:description"><p>Jonathan Harker visits Count Dracula.</p></section>
<p>161,105 words (9 hours 46 minutes) with a reading ease of 77.57 (fairly easy).</p>
<ul class="tags">
<li><a href="/subjects/fiction">Fiction</a></li>
<li><a href="/subjects/horror">Horror</a></li>
</ul>
</article>
</body>
</html>"""

async def mock_get(self, url, params=None, headers=None, **kwargs):
url_str = str(url)
if url_str.endswith("/ebooks/bram-stoker/dracula"):
return httpx.Response(200, content=book_xml)
if url_str.endswith("/ebooks/missing-author/missing-book"):
return httpx.Response(404, content=b"")
return httpx.Response(503, content=b"")

monkeypatch.setattr(httpx.AsyncClient, "get", mock_get)

detail_resp = client.get(
"/api/v1/public/standard-ebooks/story",
params={"book_id": "bram-stoker/dracula"},
)
assert detail_resp.status_code == 200
story = detail_resp.json()
assert story["id"] == str(uuid5(standard_ebooks.STANDARD_EBOOKS_NAMESPACE, "book:bram-stoker/dracula"))
assert story["title"] == "Dracula"
assert story["author"] == "Bram Stoker"
assert story["genre"] == "Horror"
assert story["sourceProvider"] == "STANDARD_EBOOKS"
assert story["providerId"] == "bram-stoker/dracula"
assert story["contentFormat"] == "PROSE"
assert story["coverImageUrl"] == "https://standardebooks.org/images/covers/bram-stoker_dracula/hash/cover@2x.jpg"
assert "Jonathan Harker" in story["synopsis"]
assert story["readTimeMinutes"] > 500

missing_resp = client.get(
"/api/v1/public/standard-ebooks/story",
params={"book_id": "missing-author/missing-book"},
)
assert missing_resp.status_code == 404

invalid_resp = client.get(
"/api/v1/public/standard-ebooks/story",
params={"book_id": "../bad"},
)
assert invalid_resp.status_code == 400


def test_federated_prose_providers_preserve_distinct_edition_identities_and_isolate_partial_failures(monkeypatch):
import httpx
from services import gutenberg, standard_ebooks

monkeypatch.setattr(standard_ebooks, "_STANDARD_EBOOKS_CATALOG_CACHE", {})
monkeypatch.setattr(standard_ebooks, "_STANDARD_EBOOKS_DETAIL_CACHE", {})

async def gutenberg_search(_client, params):
assert params == {"search": "Dracula"}
return [_gutenberg_http_book(345, "Dracula", "Stoker, Bram")]

async def se_unavailable(self, url, params=None, headers=None, **kwargs):
return httpx.Response(503, content=b"upstream maintenance")

monkeypatch.setattr(gutenberg, "_fetch_gutenberg_catalog_books", gutenberg_search)
monkeypatch.setattr(httpx.AsyncClient, "get", se_unavailable)

gutenberg_resp = client.get("/api/v1/public/gutenberg", params={"search": "Dracula"})
se_failed_resp = client.get("/api/v1/public/standard-ebooks", params={"search": "Dracula"})

assert gutenberg_resp.status_code == 200
assert se_failed_resp.status_code == 502
assert se_failed_resp.json()["detail"] == "Standard Ebooks catalog service is unavailable"

gutenberg_dracula = gutenberg_resp.json()[0]
assert gutenberg_dracula["sourceProvider"] == "GUTENBERG"
assert gutenberg_dracula["providerId"] == "345"
assert gutenberg_dracula["id"] == "00000000-0000-0000-0000-000000000159"

# Now recover Standard Ebooks and verify same-title editions remain distinct.
listing_xml = b"""<?xml version="1.0" encoding="utf-8"?>
<html xmlns="http://www.w3.org/1999/xhtml">
<body><main><ol>
<li typeof="schema:Book" about="/ebooks/bram-stoker/dracula">
<a href="/ebooks/bram-stoker/dracula" property="schema:name">Dracula</a>
<p property="schema:author"><span property="schema:name">Bram Stoker</span></p>
<img src="/images/covers/bram-stoker_dracula/hash/cover@2x.jpg" alt="Cover"/>
</li>
</ol></main></body>
</html>"""

async def se_recovered(self, url, params=None, headers=None, **kwargs):
url_str = str(url)
if "/ebooks?query=Dracula" in url_str:
return httpx.Response(200, content=listing_xml)
return httpx.Response(200, content=b"<html xmlns='http://www.w3.org/1999/xhtml'><body></body></html>")

monkeypatch.setattr(httpx.AsyncClient, "get", se_recovered)
se_ok_resp = client.get("/api/v1/public/standard-ebooks", params={"search": "Dracula"})
assert se_ok_resp.status_code == 200
se_dracula = se_ok_resp.json()[0]
assert se_dracula["sourceProvider"] == "STANDARD_EBOOKS"
assert se_dracula["providerId"] == "bram-stoker/dracula"
assert se_dracula["id"] != gutenberg_dracula["id"]
assert se_dracula["title"] == gutenberg_dracula["title"]
assert se_dracula["author"] == gutenberg_dracula["author"]
Loading
Loading