Add SOP0013 for FDP submission metadata validation - #72
Conversation
|
Similar to the other PR @hcvdwerf - change the target branch to |
…eps-for-dcat-and-beacon
|
|
||
| ACTIONS NEEDED [Within 5 working days]: | ||
| 1. Please follow through with the dataset withdrawal request as per [Step 5](https://github.com/GenomicDataInfrastructure/standard-operating-procedures/blob/main/sops/european-level/GDI-SOP0009_dataset-withdrawal.md#85-assign-withdrawal-response-team). | ||
| 1. Please follow through with the dataset withdrawal request as per [Step 5](#85-assign-withdrawal-response-team). |
There was a problem hiding this comment.
Similar to my comments on the other PR about these URLs.
There was a problem hiding this comment.
Noted. I left SOP0009 unchanged in this review pass and kept this PR focused on SOP0013. The current branch state remains the lint-safe one already pushed.
There was a problem hiding this comment.
If the scope of the SOP is exclusively FDP metadata validation, this should be inferred by the title. Others like Beacon, HealthDCAT-AP, GDI HDM could be confused with this one.
There was a problem hiding this comment.
Applied. I renamed the SOP to GDI-SOP0013_fdp-submission-metadata-validation.md and changed the H1 so the FDP-only scope is explicit.
| | Role | Full name | GDI/node role | Organisation | | ||
| | -- | -- | -- | -- | | ||
| | Author | Hans-Christian van der Werf | SOP author and FDP metadata contributor | Health Research Infrastructure | | ||
| | Reviewer | Marcos Casado Barbero | Task 4.3 lead | European Molecular Biology Laboratory | |
There was a problem hiding this comment.
| | Reviewer | Marcos Casado Barbero | Task 4.3 lead | European Molecular Biology Laboratory | | |
| | Reviewer | Marcos Casado Barbero | Task 4.3 member | European Molecular Biology Laboratory | |
| | Approver | To be assigned | Approver according to GDI SOP governance | To be assigned | | ||
| | Authorizer | To be assigned | Authorizer according to GDI SOP governance | To be assigned | |
There was a problem hiding this comment.
Similar comments to the other PR
There was a problem hiding this comment.
Applied in line with the PR #71 governance feedback: the reviewer is now Task 4.3 member, the approver is Gabi Rinck, and the authorizer is GDI Management Board.
M-casado
left a comment
M-casado
left a comment
There was a problem hiding this comment.
Good draft again, @hcvdwerf. See comments in line and below:
- Similarly to the other PR, I would recommend format changes to ease the reading and be explicit with the "what step comes after this one".
- Even as a node-specific template, it seems a bit too generic operationally. For example, what FDP instance is covered, references to documentation, like the SHACLs (even as an example in a node), more details on how to review the outcome of the validation, where the outcomes may be logged (or why).
- The current rule is basically "if validation fails, correct and save again until it passes". There is no escalation path, no ownership for persistent failures, no distinction between content errors and validator/configuration errors, and no stopping rule for cases that cannot be resolved immediately (or if there's an error in the SHACL shapes, for example).
- Similarly, to be more explicit, there should be clear references to not just GH repositories, but tags/releases if available. For example, when referencing the general GDI metadata repository.
- I'm missing references, if any, to how this SOP may interact with the other "future" SOPs on validation against GDI HDM and Beacon. Even as placeholders.
- I would also add a comment in the original GH issue https://github.com/GenomicDataInfrastructure/GDI-Pillar-2/issues/68 regarding what is covered by this SOP.
- I would add in the introduction a clear distinction between what the FDP metadata model (the nodes' SHACL shapes) and the GDI HDM. These should not be confused, and I'm sure they are based on the original request. It should explicitly say that being FDP-compliant doesn't mean overall GDI HDM-compliant.
- Are we relying on individual FDP validation at each GDI node? When this information is harvested by the central portal, how will that validation happen? If GDI nodes have different SHACL shapes, how will they be harmonised? I think @brunopacheco1 originally was requesting catalogue-level validation as well.
- The title of the SOP must be changed to its specific scope.
|
|
||
| As the metadata submitter or designated metadata curator, collect the dataset description, related distribution metadata, and contact information needed for the FDP record. Enter or update the metadata directly in the FDP according to the deployed shapes and referenced specifications. | ||
|
|
||
| Use the governing GDI SHACL shapes and referenced standards as the source of truth while preparing the record instead of maintaining a separate local checklist. |
There was a problem hiding this comment.
Add links to these, if available (?)
There was a problem hiding this comment.
Added. The references now point to the gdi-metadata 1.2.0 release, HealthDCAT-AP release 6, DCAT 3, the public GDI FDP documentation, and FAIRDataPoint client create/edit references for save-time validation behavior.
| |:--|:--|:--| | ||
| | 1 | When dataset metadata is ready to be entered or updated in the FDP. | Metadata submitter or designated metadata curator | | ||
|
|
||
| As the metadata submitter or designated metadata curator, collect the dataset description, related distribution metadata, and contact information needed for the FDP record. Enter or update the metadata directly in the FDP according to the deployed shapes and referenced specifications. |
There was a problem hiding this comment.
Can "submitters" directly interact with the FDP? I naively thought the FDPs would only be touched by the nodes themselves, after they process submissions from submitters.
There was a problem hiding this comment.
Agreed. I changed the direct actor in the procedure to node metadata curator or FDP maintainer and removed submitter as the direct FDP actor.
| |:--|:--|:--| | ||
| | 2 | After step 1, when the metadata record is ready to be saved in the FDP. | Metadata submitter or FDP maintainer | | ||
|
|
||
| Save the metadata record in the FDP. The save action triggers SHACL validation in the FDP. |
There was a problem hiding this comment.
Do we have more details on how to save it? Something like an endpoint, a URL for documentation, a screenshot, etc.
There was a problem hiding this comment.
Also, it would be great to have a URL to sustain the statement that the save action triggers the SHACL validation. This would be the FDP specification/documentation.
There was a problem hiding this comment.
Added. The procedure now points to the node's own FDP operating instructions through placeholders such as <Node FDP operational documentation>, plus the public GDI FDP documentation reference.
There was a problem hiding this comment.
Added. I anchored the save-time validation wording with the GDI FDP documentation and the FAIRDataPoint client create/edit implementation references now listed in references [7] to [9].
|
|
||
| Use the governing GDI SHACL shapes and referenced standards as the source of truth while preparing the record instead of maintaining a separate local checklist. | ||
|
|
||
| #### 8.2. Save the record and review SHACL validation results |
There was a problem hiding this comment.
In my experience, if @context or @type of an entity are not properly set, the SHACLs of DCAT (and its derivatives) may pass validation when they shouldn't.
In other words, if I think I have a dcat:dataset, and I'm trying to validate it against the SHACL shapes, but I added dcat wrong in my context, or I provided the @type wrong, the shapes won't pick the entities to validate, and the result seems to be "correct" when it's not.
Are there any safeguards for this?
There was a problem hiding this comment.
I added an explicit safeguard for that case. The introduction now states that wrong resource targeting, shape deployment, or metadata model configuration must be treated as a local FDP configuration issue rather than a successful validation outcome, and step 8.5 covers the escalation path.
|
|
||
| Review the validation result shown by the FDP and treat the applied SHACL constraints as the authoritative pass or fail criteria. Mandatory metadata fields are those marked as required by the applicable SHACL shapes enforced by the FAIR Data Point at save time. | ||
|
|
||
| #### 8.3. Correct failures and finalize the valid record |
There was a problem hiding this comment.
I would argue that we're mixing multiple steps in one. This 8.3 includes the review, correction, redirection to the validation again (the loop), and then logging the audit trail.
As a rule of thumb, if there are multiple steps in the diagram, it's likely that they are multiple steps in the document too.
There was a problem hiding this comment.
Agreed. I split the flow into separate steps for preparation, FDP entry, save-and-review, correction, local escalation, and harvestable state, and updated the mermaid diagram to match.
|
|
||
| If validation fails, update the metadata in the FAIR Data Point and save again. Repeat the correction cycle until the SHACL validation passes. | ||
|
|
||
| When validation succeeds, the data can be harvested. Record the outcome in the local log or audit trail if applicable. |
There was a problem hiding this comment.
This end step is missing:
- How to trigger the harvesting of data by the central portal, or a reference (even as a placeholder?) on where that would exist. If there's already a central portal specification to reference, that would be best.
- Details on how to record the outcome.
There was a problem hiding this comment.
Added. The SOP now ends with an explicit harvestable by the central node step, includes placeholder future references for central harvesting / GDI HDM / Beacon follow-up work, and makes local outcome recording explicit where the node keeps a log, ticket, or audit trail.
|
|
||
| | Step identifier | When | Who | | ||
| |:--|:--|:--| | ||
| | 3 | After the FDP has returned SHACL validation feedback following save. | Metadata submitter, FDP maintainer, and designated approver as applicable | |
There was a problem hiding this comment.
I don't understand what the designated approver as applicable may be. The SOP sections above explicitly said that ones "valid", it's ready for release/harvest. This sounds like there's a gatekeeping step/role.
There was a problem hiding this comment.
Agreed. I removed designated approver as applicable from the operational flow. The procedure now ends when the record is harvestable by the central node.
|
Addressed the current review feedback in Main updates:
I also added a scope clarification comment to GDI-Pillar-2 issue 68. |
|
Added Oriol as well, as his SOP will have to do with metadata validation as well. |
costero-e
left a comment
costero-e
left a comment
There was a problem hiding this comment.
Generally, the PR makes total sense to me on the different steps to follow for a metadata validation. 👍 The only thing I see is the difference between a new submission or a submission for an update, which is slightly different for beacon and I was wondering if FDP handles it exactly the same. At least, for the logs/audit, it should be different.
| | :-------------- | :----------------------------------------------------------------------------------- | :-------------------------------------- | | ||
| | `2` | After step 1, when the metadata record is ready to be entered or updated in the FDP. | Node metadata curator or FDP maintainer | | ||
|
|
||
| Enter or update the metadata directly in the node's FDP. Follow the node's FDP operating instructions, for example `<Node FDP operational documentation>`, and ensure the record is created under the intended FDP resource type and deployed shape set. |
There was a problem hiding this comment.
I think it's important to distinguish between enter and update. And for update, is relevant to know if it's a PATCH or a PUT that will be performed. At least, for beacon, depending on the partial or complete update of information, the operation might be slightly different. I don't know how FDP works on that regard, but I guess that if the information is stored in a database, the kind of operation to perform will also be relevant.
There was a problem hiding this comment.
Good point. For the FDP side, the reference client distinguishes the two operations: create uses POST, while update uses PUT for the full entity update. I kept the SOP at the workflow level rather than documenting HTTP semantics in the step text, but for FDP specifically the update path is PUT, not PATCH. The Beacon-specific distinction is out of scope for this SOP.
| | :-------------- | :----------------------------------------------------------------------------------------------------------------------- | :-------------------------------------- | | ||
| | `6` | After step 3, when the FDP validation output passes without unresolved local validator, schema, or configuration issues. | FDP maintainer or node metadata curator | | ||
|
|
||
| When validation succeeds, the record can be treated as harvestable by the central node. This SOP ends at that point; the downstream central harvesting workflow remains outside its scope and should be defined in a future GDI harvesting specification or SOP. |
There was a problem hiding this comment.
Linked to the previous comment about enter/update, maybe it is relevant here to know the type of operation to perform in the database.
There was a problem hiding this comment.
Yes, for FDP that is the relevant distinction here. The reference implementation persists updates through a PUT on the entity, so the SOP should be read as covering create via POST and update via PUT in the FDP workflow.
|
After my initial review and comments I'll leave it here without doing a second round. It'll be in the hands of the approvers whether the SOP is ready be merged or not. |
|
Just to confirm @hcvdwerf, would you say the SOP can be considered reviewed (i.e., feedback is addressed) and can go into the next step (approval by OC/SDPC)? |
yes please! |
|
OC & SDPC have been notified |
|
As a reviewer from the OC I need to clarify the scope and relationship to other SOPs before approving. My concern here is that this is just in effect describing the process entering data in to an FDP (ie. reference implementation), but not specifically the validation where a node chooses to generate their own file against a SHACL validator, and also the validation of the metadata itself beyond conformance with the SHACL. The SOP mentions that it does not deal with validation against the HDM - maybe this should be the scope of this SOP with the SHACL validation part of the process (this is implied by the title of FDP submission metadata validation)? If not I suggest the title of the SOP should be changed to something like 'SHACL validation of FDP metadata'. |
|
As a reviewer from the SDPC, I would also like to clarify if different nodes can have different SHACL shapes and that there is not standard SHACL shape required central for the harvesting to work? I would also reiterate @jdylan request for clarification of scope and possible title change for clarity. |
|
@jdylan @waakanni I have clarified this in On scope: this SOP should be read narrowly as the node-level FDP workflow for entering or updating dataset metadata in the FDP, reviewing the save-time SHACL validation returned there, correcting metadata content issues, and handling local escalation when the validator or configuration is at fault. It is not intended to cover full HDM validation, broader metadata quality assessment beyond SHACL conformance, or the downstream central harvesting validation logic. On SHACL harmonization: nodes do not necessarily need identical SHACL files. What matters is that their deployed node-level shapes conform to the central GDI SHACL constraints expected for interoperable harvesting. I have now made that distinction explicit in the SOP so local node variation is separated from the central interoperability requirements. I have kept the current title for now and tightened the Purpose, Scope, and Introduction text accordingly. |
waakanni
left a comment
waakanni
left a comment
There was a problem hiding this comment.
As an approval from SDPC, I am satisfied with the current version.
|
Current status: pending review from the OC |
costero-e
left a comment
costero-e
left a comment
There was a problem hiding this comment.
I'm not going to lie, I'm not a user for FDP (can't help with the specific things), but I think the SOP makes sense overall so I approve it. Only thing I didn't see, is what the SHACL source of validation is. Are the definitions needed to be extracted from this repo? For me is not that obvious, maybe just clarifying it would be good, but I'm just a newbie external reader (maybe the users that read this SOP already have this knowledge and is not needed for the purpose of the SOP).
|
@jdylan, you requested changes to this SOP back in May. Could you please check that you are now happy with the new version & approve for the OC if it's ok? |
6 participants
Summary
GDI-SOP0013_submission-metadata-validation.md..docxdraft into repository-compliant Markdown and align it with the SOP template, style guide, and glossary rules.Types of changes
Motivation and Context
This branch adds a node-specific SOP template for FAIR Data Point submission metadata validation. The content comes from the reviewed draft
20260305-GDI-SOP_draft-Submission Metadata Validation.docx, converted into the repository Markdown structure and normalized into a reusable template.The current SOP scope is intentionally FDP-only. It documents save-triggered SHACL validation in the FAIR Data Point as the operational validation step and keeps the required-field definition anchored in the deployed SHACL shapes and referenced specifications.
References
Changes Introduced
sops/node-specific/GDI-SOP0013_submission-metadata-validation.mdsops/README.mdwith the SOP0013 index entrydocs/GDI-SOP_charter.mdglossary withDCAT,FAIR,FDP, andSHACLCHANGELOG.mdunderUnreleasedReview
The SOP content is based on the reviewed
.docxdraft and the later FDP-only clarification for the procedure scope. Additional review from Task 4.3 / SOP maintainers is still welcome before merge.Additional Notes
scripts/sop_linter.pyis not worktree-safe because it looks specifically for a.gitdirectory; I therefore ran the linter against a temporary full-copy validation tree with the same file contents. The linter completed with no errors and only the expected warnings for emptyGDI NodeandInstance versionfields in a node-specific template.Checklist:
General Compliance:
Only applicable if the PR includes new, or changes to, GDI SOPs (i.e., documents at
sops/):