Repository navigation
[WIP][GRDM-63591] Python 3.13・依存ライブラリの更新と unoserver コンテナの分離(第1段階) - #9
Draft
miyamoto83 wants to merge 220 commits into
Draft
miyamoto83 wants to merge 220 commits into
miyamoto83 wants to merge 220 commits into
Conversation
* Add four new switches/envvars to manage keen logging:
* `KEEN_ENABLE_LOGGING` will turn on/off *all* keen logging.
* `KEEN_PRIVATE_LOG_VIEWS` will set logging for views to the
private collection.
* `KEEN_PRIVATE_LOG_ERRORS` will set logging for errors to the
private collection.
* `KEEN_PUBLIC_LOG_VIEWS` will set logging for views to the public
collection.
* The python module `certifi` was pinned to a very old version to work around issues with Rackspace. Not only is it no longer relevant, it can no longer successfully validate modern TLS certs
* conversion based off @aaxelb's work in @CenterForOpenScience/SHARE * update CI badge urls in README.md * add explicit upload to coveralls * replace python-coveralls with coveralls in dev-requirements.txt * remove old travis config files [ENG-3265]
* Allow the parent window of the MFR iframe to pass its own url when
starting the hypothes.is annotator. Parents using `postMessage`
must now send an object with two keys: `type: 'startHypothesis'`
and `parentUrl: <parent window url>`. Parents using pym must now
call `.sendMessage('setParentUrl', <parent window url>)`. The
included pym code has been updated to do this.
MFR overrides the annotator to pass the parent window's url rather
than its own, b/c the parent window url is more stable and
descriptive. It was previously doing this by calling
`document.referrer` from within the iframe. Due to privacy
concerns, browsers have started allowing the iframe to only see the
host and not the path of the parent url. Parent frames must now
pass in their own url to get around this.
* Upgrade Dockerfile image base to python v3.6. * Install gosu via apt-get. gosu v1.10 started being shipped with debian as of the `stretch` release. Install from apt-get instead of `curl`-ing it from the github release page. * Pin pip version to 18.1 to avoid dep conflicts. * Based off the following WB commits: * CenterForOpenScience/waterbutler@d03a84c * CenterForOpenScience/waterbutler@b179eb8
* Bump reportlab dep to v3.6.5 to unbreak `invoke install` on macOS.
* add boa to supported extensions * bump pygments dep to 2.12 to support boa * add debugging logs to codepygments extension
…/hotfix/oci-cicd [Hotfix][No Ticket] Enable OCI autodeployments
Hotfix to add build_spec.yaml for OCI CI/CD 26.1.1
…en witho…" This reverts commit 7f101d1.
…/revert-411-fix/ENG-11824 Revert "fix(cors): always send Cache-Control: no-store even without an Origin…"
[ENG-12505]
… the Debian package instead of fetching it with keyserver verification, and exclude the report script from flake8 because it requires Python 3.6+
…/vuln-remediation, keeping the RDM CJK CMap files, the allow-modals iframe sandbox and the README lines unrelated to the Python 3.13 update
… runtime stages, move LibreOffice into a separate unoserver image (Dockerfile.unoserver) built on Ubuntu 26.04 because Debian 13 links LibreOffice against libxml2 2.9.14 whose open CVEs cannot be fixed there (the fixed 2.15 changed ABI) and without the unused pebble binary the Ubuntu base image ships, apply security updates, copy the nbconvert templates into the runtime image and verify the dependency tree with pip check
…age image and scan both the mfr and unoserver images with Trivy
…dt/ods with Japanese text) and run them in CI against the unoserver image
…cies: Pillow 13.0.0, mistune, Mako, nbconvert 7.17.2 (CVE-2026-114140), psd-tools, Jinja2, Markdown, Pygments and ipython to versions with security fixes, propcache 0.5.4 (0.4.0 was yanked for a reference leak) and the unpinned transitive libraries (urllib3, pyasn1, soupsieve, bleach, idna, multidict) to their current releases; move black/isort/ruff to the dev group, port PSD export to PSDImage.open().composite() and drop the stray scikit-image test import
… of the original URL and keep the response open for streaming; add provider download tests
…Keen analytics when no project id is configured; add HTTP provider tests
…ar) and close it on shutdown instead of leaking a session per request
…s with asyncio.sleep so the event loop keeps serving requests, and give the worker its own control exchange so it does not receive pickle control messages from other applications sharing the broker
…e in the rst renderer)
…andlers only answer GET, but tornado parses form bodies before the method check
miyamoto83
force-pushed
the
feature/vuln-remediation-cos-wb
branch
from
October 29, 2026 08:54
8f8e292 to
7b6a7be
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket
GRDM-63591
Purpose
Python と依存ライブラリを更新し、既知の脆弱性を解消する。第 1 段階として CenterForOpenScience/modular-file-renderer 26.2.1 と waterbutler 26.6.1 の依存構成に合わせる。RDM-waterbutler の更新に合わせて、waterbutler に関わる依存の扱いを見直す第 2 段階を別 PR で行う。
Changes
Dockerfile.unoserver)に分離し、Office 系ファイルの変換は Celery ワーカー経由で unoserver に依頼する方式に統一した。pydocx による docx レンダラは削除した。control_exchangeを分離し、同一 RabbitMQ を共有する他アプリケーションの制御メッセージを受け取らないようにした。Side effects
QA Notes
feature/mfr-python313(c82d605)で、本ブランチと miya-biz/RDM-waterbutlerfeature/vuln-remediationからビルドしたイメージを指定して実行。実行 40085395598 は 17 ジョブ中 14 成功。ファイルプレビュー(txt、png、jpg、pdf、docx、pptx、csv、xlsx、webm)を含む User Tests は成功。失敗 3 件(Metadata、Wiki、Migration 20250906)は MFR を経由しない。Deployment Notes
UNOSERVER_EXTENSION_CONFIG_HOST・UNOSERVER_EXTENSION_CONFIG_PORT(既定 127.0.0.1:2003)を設定する。python3 -m invoke celery)を起動する。ブローカーはTASKS_CONFIG_BROKER_URL。control_exchangeの既定値(mfr)のままでよい。vhost を分けている場合は不要。mfr_requirementsは Python 3.5 のパスでinvoke installを実行しているため、site-packages と/usr/local/binをボリュームへコピーする方式に変更し、mfr_worker と unoserver のサービスを追加する必要がある。E2E ではsetup_rdm.shの上書きで対応している。