Repository navigation
Conversation
ndnhat1
changed the title
…i_grdm_202603_step4/2.2_quota_calculation_when_move_file
+ Update email template handle move/copy file quota and max size error + Add script migrate missing file info and update quota used
…or Institution Storage
+ Fix institution default max quota issues (owner lookup, zero quota, quota retrieval, transaction handling) + Refactor quota handling logic + Update migration (use OneToOneField and remove db_index) + Update default value in "Institutional Storage Quota Control" screen
+ Conflict resolve + Mail template render + Issue data.size is string type
…logic in Institution Storage
+ Fix user quota update rollback behavior + Align related_name between model and migration + Remove unused user parameter from file_modified + Update UT after change logic
… quota management
… in quota management based on review comments
- Upsert UserQuota on storage type change and skip institution quota for NII storage region - Handle NII quota type change on storage credential save and UT
… size limit when move/copy and update UT
yacchin1205
approved these changes
Sep 11, 2026
yacchin1205
left a comment
yacchin1205
left a comment
There was a problem hiding this comment.
OKかと思います。一点、対応するRDM-waterbutlerへのリンク(これを必ずセットでデプロイすること)を、明記いただけると良いかと思います。
…i_grdm_202603_step9/2.3_initial_quota_setting
上記について、明記いたしました。 |
Replace the boto 2 client with boto3/botocore so that the S3 addon can talk
to regions that only accept Signature Version 4 (ap-northeast-1 and later).
- utils.py / views.py / models.py build the client through boto3 with a
region-specific endpoint, and surface botocore exceptions as HTTPError
- folder_id becomes "bucket:/prefix/" so that a subfolder can be selected;
the legacy "bucket" form is still accepted
- the node/user config JS, the Rubeus config and the log action lists follow
the new folder shape
- requirements.txt gains boto3/botocore; addons/s3/requirements.txt is kept
as an empty file because the Jenkins Dockerfile still does
COPY ./addons/s3/requirements.txt and would fail the image build without it
- MAX_UPLOAD_SIZE stays at 50GB: multipart upload handles up to 5TB, so the
lower limit the first draft introduced was unnecessary
- the ${ _("...") } gettext wrappers are kept in all three mako templates
(20 translatable strings); only the aria-label/alt attributes and the th
class change are new
Co-Authored-By: An Qiuyu <qiuyu.an@hotmail.com>
…lready registered
Registering the same access key twice has to end up on the existing
ExternalAccount instead of creating a second row. The duplicate is caught by
the unique constraint on (provider, provider_id) — provider_id holds
"{aws_account_id}\t{access_key}" (GRDM-53044) — so the save raises
IntegrityError, not the ValidationError the handler was catching, and the
failed INSERT aborted the surrounding transaction.
Wrap the save in transaction.atomic() so the failure only rolls back to a
savepoint, and catch IntegrityError alongside ValidationError before falling
back to the account that is already stored.
…igV4 folder shape - test_view.py still used assert_equals/assert_in and a missing InstitutionFactory import left over from the nose era; replace them with plain pytest asserts against rv.text - _mock_folder_result still returned the pre-SigV4 get_folders payload, so the node settings tests asserted against a shape the view no longer produces
Extend addons/s3/tests/test_view.py so the SigV4 flow is exercised end to end:
- both folder_id forms ("bucket" and "bucket:/prefix/") and a non-existent
bucket raising InvalidFolderError
- the provider_id format "{aws_account_id}\t{access_key}" and the dedup path
when the same key is registered twice (GRDM-53044)
- the error branches: a missing key field, get_user_info returning None, an
invalid bucket name, an empty bucket_exists result, can_list returning None,
and the ClientError / InvalidAuthError conversions
Three test-harness fixes come with it:
- query ExternalAccount by provider_id, a plain CharField. oauth_key is an
EncryptedTextField (JWE), so its ciphertext differs on every save and
filtering on it never matches; the plaintext is checked by attribute access
instead
- use post_json/put_json instead of the json= keyword, which the webtest_plus
version used in CI does not accept, and pass expect_errors=True wherever a
4xx/401/403 response is expected
- patch get_user_info with mock.patch(..., return_value=mock.Mock(...)); the
.return_value attribute on the patcher object never reaches the started mock,
so user_info.id / user_info.display_name were missing in the view
…display name - stop lower-casing the bucket name in the settings check. S3 bucket names are case-sensitive in the legacy us-east-1 naming rules, so folding the case can point the addon at a different bucket; validate_bucket_name already rejects the names that are actually invalid - fall back gracefully when the owner returned by GetUserInfo has no display name, instead of raising while building the account label - keep the Owner entry of the admin storage-location response in its original shape so that the existing consumers keep working
…rametrize pytest.mark.parametrize has no effect on a method of a unittest.TestCase: the case is collected once and the extra arguments are never supplied, so test_serialize_settings_supports_legacy_and_prefixed_folder_ids errored on a missing fixture instead of checking the three folder_id forms. Iterate over the cases inside the test body instead.
get_bucket_location() returns a null LocationConstraint for us-east-1, but BUCKET_LOCATIONS keys that region as '', so the null fell through to set_folder and labelled the bucket '(None)' in the node settings UI.
The ListObjects call was unguarded, so expired or revoked keys and a missing bucket escaped as a raw botocore exception and reached the user as a 500 with no actionable message. Map NoCredentialsError to 403 and reuse get_status_for_error so a ClientError keeps the status S3 returned.
create_bucket only handled NoCredentialsError and ClientError, so the non-response failures - bad parameters, an unreachable endpoint - left the user with a 500. ClientError is not a BotoCoreError subclass and NoCredentialsError is matched earlier, so the existing clauses keep their behaviour.
Register R2026b at the head of MATLAB_RELEASES and in MATLAB_PRODUCTNAMES_MAP, and add its mpm input file. It makes the release available in the BinderHub addon. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
mathworks-ref-arch/matlab-dockerfile@f419020 added a commented-out supportPackageDestinationFolder section to these files. No product lines changed, so the generated product name lists are unaffected. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
hide24
approved these changes
Oct 16, 2026
ndnhat1
marked this pull request as ready for review
October 19, 2026 02:37
Feature/s3 sigv4 clean
…_initial_quota_setting Feature/nii grdm 202603 step9/2.3 initial quota setting
…auth Feature/y websocket jwt auth
…api-permission Feature/fix wiki import api permission
…3-#63370 Feature/binderhub 20261013 #63370
…_quota_calculation_when_move_file
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Changes
QA Notes
Documentation
Side Effects
Deployment Notes
Please ensure the following branches are deployed to their respective servers:
Ticket
GRDM-57690