File service

[abdellah257]

Description

This Merge contains changes that covers multiple aspects / features / improvements, including:

• file endpoints support, to download/stream a single file in raw format.
• Option to provide an hdf viewing service, that can be served with file/open endpoint.
• Deprecated the use of directory in POST requests, to make sure the directory can be resolved directly from the dataset. This closes the big issue of given access to a directory that is unrelated to a given dataset.
• Make sure all file accesses are done under the configured allowedDataDirectory.
• Deprecated the use of upload endpoint, currently still unsafe to use.
• More details about the configuration and how the directory is resolved can be found in the README.md

Changes:

new endpoints:

• POST /file
• POST|HEAD /file/download
• POST|HEAD /file/stream
• POST /file/open

new configuration example:

{
  "zipDir": "/tmpZip",
  "zipRetentionMillis": 3600000,
  "fileLinkRetentionMillis": 3600000,
  "jwtSecret": "secret123",
  "sessionSecret": "fj9832mnsaf3j9adsa",
  "facility": "ILL",
  "allowedDataDirectory": "/data",
  "directoryPathPattern": "/data/{sourceFolder}/{type}",
  "requiredKeywords": ["sourceFolder", "type"],
  "hdfViewServiceUrl": "https://hdf-viewer.example.org",
  "routeBasePath": "",
  "publicOrigin": "https://zip-service.example.org",
  "scicatApiBasePath": "https://scicat.example.org",
  "scicatApiAccessToken": "<scicatServiceAccountToken>",
  "testData": {
    "jwt": "<jwtToken>",
    "files": ["<testFileName1>", "<testFileName2>", "<testFileName3>"]
  }
}

Tests included/Docs Updated?

• Included for each change/fix?
• Passing? (Merge will not be approved unless this is checked)
• Docs updated?
• New packages used/requires npm install?
• Toggle added for new features?