Tools should be deployed directly to the back end

[jjcarver]

Currently, the workflow deployment scripts deploy both workflow XML files and tools together, directly to one of the ProteoSAFe front end web servers, by taking advantage of the /data/cluster/ mount that is present on all such servers. This is not an appropriate approach since tools are back end artifacts and should not be deployed directly a to a front end server.

Ideally the workflow deployment scripts should deploy tools directly to the ProteoSAFe back end (i.e. ccms-login-temp.ucsd.edu). This is the where all back end mounts are officially accessed, and is the only place where we can safely rely on any cross-server symlink relationships being safely resolvable.

[jjcarver]

Currently, the workflow deployment scripts tightly couple workflow XML files (which, from ProteoSAFe's perspective, are front end web server artifacts) and their associated tools (which are back end cluster artifacts). It is sensible to associate these logically (i.e. by keeping them together in the same repository), but from a deployment perspective any back end artifact (e.g. tools) should ideally be deployed directly to the back end, not the front end.

Right now we deploy both workflow XMLs and tools together by taking advantage of a file system mounted on all production ProteoSAFe web servers - /data/cluster/. This file system is mounted as a convenience on all web servers since it points to that web server's back end "delegate" file system. However, this is considered a convenience since this file system contains back end artifacts and thus its presence is strictly unnecessary on any front end server, and consequently it should not be relied upon for any production services.

Normally this would just be a philosophical nitpick that we could safely ignore, since we are going to keep the /data/cluster/ mounts on all ProteoSAFe web servers and therefore they can be relied upon in practice. However, a recent attempt to deploy a workflow to proteomics3 illustrated a more practical problem - proteomics2 and proteomics3 are supposed to have the exact same tools mount, which we accomplish via symlinks, but these only work on the cluster since proteomics2's back end is not mounted on proteomics3. Thus any attempt to deploy tools directly to proteomics3 will fail.

We could work around this with the current deployment scripts by either never deploying workflows directly to proteomics3, or by mounting proteomics2's back end on proteomics3 so that the symlinks work there. We will not do the latter since this defeats the whole purpose of ProteoSAFe's distributed design - then server front ends and back ends would be tightly coupled via some ugly interrelation existing only to make an imperfect solution work. For now we will just have to go with the former.

In the long term, however, these kinds of special server relationships can be cleanly supported as long as we make sure to deploy all workflow artifacts where they are supposed to go - XMLs to the front end, tools to the back end.