Repository navigation
feat: improve gifts indexing queue priority, task timeouts, and refresh UX feedback - #259
Merged
Merged
Conversation
…n Celery - Prefixes gift queues with 0-, 1-, and 2- for deterministic alphabetical sorting - Updates Celery queue_order_strategy to 'sorted' to avoid Kombu priority_cycle set-ordering bug - Enables task_acks_late to ensure workers fetch tasks just-in-time
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR addresses queue priority ordering for Telegram gift indexing and improves the user experience during manual gift refreshes:
priority_cycleiterating over a Python
setwith randomized hash seeds in RedisBRPOP. We resolved this by prefixing the queue names alphabetically and using Celery's"queue_order_strategy": "sorted", combined withtask_acks_late: Trueandworker_prefetch_multiplier: 1for deterministic priority consumption./system/async-tasks/{task_id}polling endpoint previously waited indefinitely in awhile not task_result.ready()loop. It now enforces adefault timeout (
DEFAULT_ASYNC_TASK_WAIT_TIMEOUT_SECONDS = 9.5) and returnsHTTP 408 Request Timeout("Task took longer than expected") if the task is still executing.ApiService(ASYNC_TASK_TIMEOUT_MS = 10000) for task status checks. With Ky's built-in 1-retryon status 408, the client waits ~19–20 seconds total before giving up. Added visual feedback toasts in
ChatConditions:"Gifts refreshed successfully")."Your request to refresh gifts was submitted, but took longer than expected. Please, wait for 5 minutes or try again"), while refreshing chat state.Fixes: # (if applicable)
Checklist
Before submitting your pull request, please ensure the following:
Changes
core/constants.py,indexer_gifts/celery_app.py,docker-compose.yml):0-gift-user-priority,1-gift-collections-sync, and2-gift-user-batch."queue_order_strategy": "sorted","task_acks_late": True, and"worker_prefetch_multiplier": 1.core/constants.py,core/utils/task.py,api/routes/system.py):DEFAULT_ASYNC_TASK_WAIT_TIMEOUT_SECONDS = 9.5.timeoutparameter towait_for_taskraisingTimeoutErrorwhen exceeded.get_task_status, catchingTimeoutErrorand returningHTTP 408 Request Timeout.common/utils/constants.ts,services/ApiService.ts,store/user/api.ts):ASYNC_TASK_TIMEOUT_MS = 10000.timeoutoverride inApiOptionsand mapped KyTimeoutErrorto HTTP 408.timeoutoverride specifically incompleteChatTasksAPI.store/user/user.ts,ChatConditions.tsx):TaskTimeoutErrorfor HTTP 408 responses.handleRefreshGifts.How Has This Been Tested?
make testinside the Docker test environment (all 111 tests passed in 9.97s).pre-commit(ruff,ruff-format) on all modified backend files with zero violations.npm run build(built cleanly with 0 errors).Screenshots (if applicable)
N/A
Additional Notes
succeed without blocking the UI indefinitely.