Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .jules/bolt.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,7 @@
## 2026-06-11 - [General] Optimized AWS Service Clients with botocore.config.Config
**Learning:** Configuring Boto3 clients with `tcp_keepalive=True` and `retries={"max_attempts": 3, "mode": "standard"}` in the `botocore.config.Config` significantly improves connection resilience and reduces latency in AWS Lambda. TCP keep-alive ensures that connections in the pool remain active, avoiding the overhead of re-establishing TCP/TLS handshakes, while the 'standard' retry mode provides more robust exponential backoff.
**Action:** Always use a centralized `botocore.config.Config` when instantiating Boto3 resources or clients in Lambda templates to optimize performance and reliability.

## 2026-06-12 - [Stream] Bypassing Intermediate Models in High-Throughput Paths
**Learning:** In high-throughput event processing (like DynamoDB Streams), validating an intermediate 'Source' model before transforming it into a 'Destination' model adds unnecessary overhead. Since `aws-lambda-powertools` already unmarshals the DynamoDB JSON into standard Python dictionaries in `record.dynamodb.new_image` and `.keys`, we can validate the `DestinationItem` directly from these dictionaries, saving one full Pydantic instantiation and validation cycle (~20% faster per record).
**Action:** Avoid intermediate model validation in data transformation pipelines. Validate the final model directly from the unmarshalled event dictionary whenever possible.
3 changes: 3 additions & 0 deletions .template/docs/template.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,6 @@ Deploy the stack using:
```bash
mise run deploy ${name}
```


[← Back to Overview](../README.md)
3 changes: 3 additions & 0 deletions docs/template/agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,6 @@ Variable | Description | Required | Default
`SERVICE_NAME` | Powertools service name | No | `bedrock-agent`
`METRICS_NAMESPACE` | Powertools metrics namespace | No | `BedrockAgent`
`LOG_LEVEL` | Log level for the Lambda Logger | No | `INFO`


[← Back to Overview](../README.md)
3 changes: 3 additions & 0 deletions docs/template/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,6 @@ Variable | Description | Required | Default
`SERVICE_NAME` | Powertools service name | No | `rest-api`
`METRICS_NAMESPACE` | Powertools metrics namespace | No | `RestApi`
`LOG_LEVEL` | Log level for the Lambda Logger | No | `INFO`


[← Back to Overview](../README.md)
3 changes: 3 additions & 0 deletions docs/template/eventbridge.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,6 @@ Variable | Description | Required | Default
`SERVICE_NAME` | Powertools service name | No | `eventbridge`
`METRICS_NAMESPACE` | CloudWatch metrics namespace | No | `EventBridge`
`LOG_LEVEL` | Log level for the Lambda Logger | No | `INFO`


[← Back to Overview](../README.md)
3 changes: 3 additions & 0 deletions docs/template/graphql.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,3 +90,6 @@ Variable | Description | Required | Default
`SERVICE_NAME` | Powertools service name | No | `graphql-api`
`METRICS_NAMESPACE` | Powertools metrics namespace | No | `GraphQLApi`
`LOG_LEVEL` | Log level for the Lambda Logger | No | `INFO`


[← Back to Overview](../README.md)
3 changes: 3 additions & 0 deletions docs/template/s3.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,6 @@ Variable | Description | Required | Default
`SERVICE_NAME` | Powertools service name | No | `s3-processor`
`METRICS_NAMESPACE` | CloudWatch Metrics namespace | No | `S3Processor`
`LOG_LEVEL` | Log level for the Lambda Logger | No | `INFO`


[← Back to Overview](../README.md)
3 changes: 3 additions & 0 deletions docs/template/sqs.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,6 @@ Variable | Description | Required | Default
`SERVICE_NAME` | Powertools service name | No | `sqs-processor`
`METRICS_NAMESPACE` | Powertools metrics namespace | No | `SqsProcessor`
`LOG_LEVEL` | Log level for the Lambda Logger | No | `INFO`


[← Back to Overview](../README.md)
3 changes: 3 additions & 0 deletions docs/template/stream.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,6 @@ Variable | Description | Required | Default
`SERVICE_NAME` | Powertools service name | No | `dynamodb-stream`
`METRICS_NAMESPACE` | Powertools metrics namespace | No | `DynamoDBStream`
`LOG_LEVEL` | Log level for the Lambda Logger | No | `INFO`


[← Back to Overview](../README.md)
14 changes: 7 additions & 7 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ requires-python = ">=3.14"
dependencies = [
"pydantic>=2.13.4",
"pydantic-settings>=2.14.2",
"aws-lambda-powertools>=3.31.0",
"boto3>=1.43.45",
"aws-lambda-powertools>=3.31.1",
"boto3>=1.43.50",
"requests>=2.34.2",
]

Expand All @@ -21,22 +21,22 @@ rename = "scripts.rename:main"
[dependency-groups]
infra = [
"aws-cdk-lib>=2.261.0",
"constructs>=10.6.0",
"constructs>=10.7.0",
]
dev = [
"pytest>=9.1.1",
"ruff>=0.15.21",
"coverage>=7.15.0",
"ruff>=0.15.22",
"coverage>=7.15.2",
"pre-commit>=4.0.1",
"pyright>=1.1.411",
"pytest-mock>=3.15.1",
"moto>=5.2.1",
"hypothesis>=6.156.4",
"hypothesis>=6.156.6",
"click>=8.4.2",
]
docs = [
"mkdocs>=1.6.1",
"mkdocstrings[python]>=1.0.4",
"mkdocstrings[python]>=1.0.6",
"mkdocs-material>=9.6.20",
]

Expand Down
14 changes: 14 additions & 0 deletions templates/agent/handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
from aws_lambda_powertools.event_handler import BedrockAgentFunctionResolver
from aws_lambda_powertools.utilities.data_classes import BedrockAgentEvent
from aws_lambda_powertools.utilities.typing import LambdaContext
from pydantic import ValidationError

from templates.agent.models import Item
from templates.agent.settings import Settings
from templates.models import Entity
from templates.repository import Repository

settings = Settings() # type: ignore
Expand All @@ -29,11 +31,20 @@ def get_item(item_id: str) -> dict:
The item details or an error message.
"""
logger.info("Retrieving item", extra={"itemId": item_id})
try:
Entity(id=item_id) # Validate ID format before querying repository
except ValidationError:
logger.warning("Invalid item ID provided", extra={"itemId": item_id})
return {"error": "Invalid item ID format"}

try:
item = repository.get_item(item_id)
if not item:
return {"error": f"Item {item_id} not found"}
return Item.model_validate(item).dump()
except ValidationError as error:
logger.error("Item validation failed", extra={"itemId": item_id}, exc_info=error)
return {"error": "Internal server error"}
except Exception as error:
logger.error("Failed to get item", extra={"itemId": item_id}, exc_info=error)
return {"error": f"Failed to get item with ID '{item_id}'"}
Expand All @@ -57,6 +68,9 @@ def create_item(item_id: str, name: str, description: str | None = None) -> dict
item = Item(id=item_id, name=name, description=description).dump()
repository.put_item(item)
return item
except ValidationError as error:
logger.warning("Invalid item data provided", extra={"itemId": item_id}, exc_info=error)
return {"error": "Invalid item data"}
except Exception as error:
logger.error("Failed to create item", extra={"itemId": item_id}, exc_info=error)
return {"error": f"Failed to create item with ID '{item_id}'"}
Expand Down
4 changes: 2 additions & 2 deletions templates/api/handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@

from templates.api.models import Item
from templates.api.response import JsonResponse
from templates.models import Entity
from templates.api.settings import Settings
from templates.models import Entity
from templates.repository import Repository

settings = Settings()
Expand All @@ -33,7 +33,7 @@ def get_item(id: str) -> Response:
try:
Entity(id=id)
except ValidationError:
return JsonResponse({"message": "Item ID must be between 1 and 50 characters"}, status_code=400)
return JsonResponse({"message": "Invalid item ID"}, status_code=400)

try:
if (item := repository.get_item(id)) is None:
Expand Down
18 changes: 15 additions & 3 deletions templates/graphql/handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

from templates.graphql.models import Item
from templates.graphql.settings import Settings
from templates.models import Entity
from templates.repository import Repository

settings = Settings() # type: ignore
Expand All @@ -29,13 +30,21 @@ def get_item(id: str) -> dict | None:
Returns:
The item if found, or None.
"""
try:
Entity(id=id) # Validate ID format before querying repository
except ValidationError:
logger.warning("Invalid item ID provided", extra={"itemId": id})
raise RuntimeError("Invalid item ID format") from None

try:
if (item := repository.get_item(id)) is None:
return None
return Item.model_validate(item).dump()
except Exception as error:
logger.error("Failed to get item", extra={"itemId": id}, exc_info=error)
raise RuntimeError(f"Failed to get item with ID '{id}'") from None
is_val_error = isinstance(error, ValidationError)
message = "Item validation failed" if is_val_error else f"Failed to get item with ID '{id}'"
logger.error(message, extra={"itemId": id}, exc_info=error)
raise RuntimeError(message) from None


@app.resolver(type_name="Query", field_name="listItems")
Expand Down Expand Up @@ -68,7 +77,10 @@ def create_item(name: str) -> dict:
item = Item(name=name).dump()
repository.put_item(item)
return item
except (ValidationError, Exception) as error:
except ValidationError as error:
logger.warning("Invalid item data provided", extra={"itemName": name}, exc_info=error)
raise RuntimeError("Invalid item data") from None
except Exception as error:
logger.error("Failed to create item", extra={"itemName": name}, exc_info=error)
raise RuntimeError(f"Failed to create item with name '{name}'") from None

Expand Down
1 change: 1 addition & 0 deletions templates/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,5 @@ class Entity(Object):
default_factory=lambda: str(uuid4()),
min_length=1,
max_length=50,
pattern=r"^[a-zA-Z0-9-_]+$",
)
25 changes: 16 additions & 9 deletions templates/stream/handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from pydantic import ValidationError

from templates.repository import Repository
from templates.stream.models import DestinationItem, SourceItem
from templates.stream.models import DestinationItem
from templates.stream.settings import Settings

settings = Settings()
Expand All @@ -30,18 +30,18 @@ def __init__(self, repository: Repository) -> None:
self._repository = repository

@tracer.capture_method
def _process(self, item: SourceItem) -> DestinationItem | None:
def _process(self, item: dict) -> DestinationItem | None:
"""Transform a source item into a destination item.

Args:
item: The source item to process.
item: The raw source item dictionary to process.

Returns:
A `DestinationItem` on success, or `None` if validation fails.
"""
try:
# TODO: process here
return DestinationItem.model_validate(item, from_attributes=True)
# Validate and transform directly from the raw dictionary to save an intermediate model instantiation
return DestinationItem.model_validate(item)
except ValidationError as exc:
logger.error("DestinationItem validation failed", exc_info=exc)
return None
Expand All @@ -63,13 +63,20 @@ def handle_record(self, record: DynamoDBRecord) -> None:
event_name = record.event_name

if event_name and event_name.name in ("INSERT", "MODIFY"):
item = self._process(SourceItem.model_validate(record.dynamodb.new_image))
# Bypass SourceItem validation for performance in high-throughput stream processing
item = self._process(record.dynamodb.new_image)
if item is None:
raise ValueError("Failed to process record into DestinationItem")
self._repository.put_item(item.model_dump())
self._repository.put_item(item.dump())
elif event_name and event_name.name == "REMOVE":
plain_keys = SourceItem.model_validate(record.dynamodb.keys)
self._repository.delete_item(plain_keys.id)
# Proactively validate the ID before deletion to provide defense-in-depth while avoiding full model overhead
try:
item_id = record.dynamodb.keys.get("id")
DestinationItem(id=item_id)
self._repository.delete_item(item_id)
except ValidationError as exc:
logger.error("Invalid item ID in REMOVE event", exc_info=exc)
raise ValueError("Failed to process REMOVE record: invalid ID") from exc


handler = Handler(repository)
Expand Down
9 changes: 9 additions & 0 deletions tests/agent/test_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,15 @@ def test_handler_get_item_not_found():
assert "not found" in result["error"]


def test_handler_get_item_invalid_id():
from templates.agent.handler import get_item

result = get_item("invalid!")

assert "error" in result
assert "Invalid item ID" in result["error"]


def test_handler_create_item(repository):
from templates.agent.handler import create_item

Expand Down
14 changes: 13 additions & 1 deletion tests/api/test_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,19 @@ def test_get_item_id_too_long(mock_repo, lambda_context):

assert response["statusCode"] == 400
body = loads(response["body"])
assert body["message"] == "Item ID must be between 1 and 50 characters"
assert body["message"] == "Invalid item ID"


def test_get_item_id_invalid_pattern(mock_repo, lambda_context):
"""GET /items/{id} returns 400 when the ID contains invalid characters."""
import templates.api.handler as handler_module

event = _apigw_event("GET", "/items/invalid!", path_params={"id": "invalid!"})
response = handler_module.main(event, lambda_context)

assert response["statusCode"] == 400
body = loads(response["body"])
assert body["message"] == "Invalid item ID"


def test_post_item_invalid_body(mock_repo, lambda_context):
Expand Down
11 changes: 11 additions & 0 deletions tests/graphql/test_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,17 @@ def test_sensitive_data_exposure(repository, lambda_context):
assert "internal_secret" not in result_list[0]


def test_get_item_invalid_id(lambda_context):
import pytest

from templates.graphql.handler import main

event = {"info": {"parentTypeName": "Query", "fieldName": "getItem"}, "arguments": {"id": "invalid!"}}
with pytest.raises(RuntimeError) as excinfo:
main(event, lambda_context)
assert "Invalid item ID" in str(excinfo.value)


def test_error_message_information_leakage(lambda_context, mocker):
"""Verify that internal error details are NOT leaked to the client."""
from templates.graphql import handler
Expand Down
Loading