Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions scripts/instance.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -889,3 +889,5 @@ WEKO_RECORDS_UI_S3_TRANSFER_USE_THREADS = True
WEKO_RECORDS_UI_S3_TRANSFER_MAX_CONCURRENCY = 10
"""Number of threads for multipart upload/download for S3 compatible service transfer. Default is 10."""

WEKO_SEARCH_FIX_ACCESSRIGHTS = True

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Harvest fails on timestamp dates

When a file’s dateValue is present but is not exactly YYYY-MM-DD, such as a valid date-time value, Record.updated passes the file’s dateValue directly to datetime.strptime(..., "%Y-%m-%d"), raising ValueError; OAI-PMH response generation then fails when it reads record.updated.

Validate or safely parse dateValue before calling strptime, and skip values that cannot be parsed as dates.

Prompt for AI agents
In `scripts/instance.cfg` at line 892:

**Harvest fails on timestamp dates**

When a file’s `dateValue` is present but is not exactly `YYYY-MM-DD`, such as a valid date-time value, `Record.updated` passes the file’s `dateValue` directly to `datetime.strptime(..., "%Y-%m-%d")`, raising `ValueError`; OAI-PMH response generation then fails when it reads `record.updated`.

Validate or safely parse `dateValue` before calling `strptime`, and skip values that cannot be parsed as dates.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Indexing adds per-record database work

When a bulk indexing or reindexing job processes many records, indexing reads record.updated in _prepare_record; with this setting enabled, that property queries record metadata and resolves its item-type mapping for each record. Bulk indexing therefore adds per-record database and mapping work, slowing reindexing and other indexing workloads.

Avoid per-record metadata queries during indexing, for example by batching the required metadata or reusing data already loaded for the record.

Prompt for AI agents
In `scripts/instance.cfg` at line 892:

**Indexing adds per-record database work**

When a bulk indexing or reindexing job processes many records, indexing reads `record.updated` in `_prepare_record`; with this setting enabled, that property queries record metadata and resolves its item-type mapping for each record. Bulk indexing therefore adds per-record database and mapping work, slowing reindexing and other indexing workloads.

Avoid per-record metadata queries during indexing, for example by batching the required metadata or reusing data already loaded for the record.

"""Fix access rights for search indexes. True if fixing access rights."""
Loading