SSH MAC algorithm update required

[DagSonsteboSB]

ISSUE TYPE

• Improvement Request

COMPONENT NAME

• ACS

CLOUDSTACK VERSION

• All current versions (4.11, 4.12)

CONFIGURATION

• All

OS / ENVIRONMENT

• All

SUMMARY

• CloudStack is using the trilead-ssh2 library for ssh communications
• This does not have support for the latest MAC algorithms
• Change to different library should be considered

STEPS TO REPRODUCE

• N/A

EXPECTED RESULTS

• Use of recent SSH MAC algorithms

ACTUAL RESULTS

• N/A

[miklosbarabas]

STEPS TO REPRODUCE

On executing an addHost one could receive the error message below in the CS Management server:

2020-05-26 18:38:24,314 WARN  [o.a.c.a.c.a.h.AddHostCmd] (qtp858242339-258:ctx-301b148f ctx-4ba21dfe ctx-eb317e92) (logid:41e20b6e) Exception:
com.cloud.exception.DiscoveryException: Unable to add the host
        at com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:825)
        at com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:611)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
        at com.sun.proxy.$Proxy180.discoverHosts(Unknown Source)
        at org.apache.cloudstack.api.command.admin.host.AddHostCmd.execute(AddHostCmd.java:142)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
        at com.cloud.api.ApiServer.queueCommand(ApiServer.java:758)
        at com.cloud.api.ApiServer.handleRequest(ApiServer.java:582)
        at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:310)
        at com.cloud.api.ApiServlet$1.run(ApiServlet.java:130)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:127)
        at com.cloud.api.ApiServlet.doGet(ApiServlet.java:89)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:686)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:791)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:852)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
        at org.eclipse.jetty.server.Server.handle(Server.java:530)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)

On the client:

Error: (HTTP 530, error code 9999) Unable to add the host

Which is not really telling. Nothing in the agent logs. But in /var/log/auth.log this reads:

May 26 18:38:24 <sanitized> sshd[28624]: Unable to negotiate with <sanitized> port 39184: no matching MAC found. Their offer: hmac-sha1-96,hmac-sha1,hmac-md5-96,hmac-md5 [preauth]

EXPECTED RESULTS

• Use of recent SSH MAC algorithms

• Maybe more descriptive error message on failure to add a host? Could spare some time for users.

ACTUAL RESULTS

• N/A

• Unable to add host

Workaround is to add one of the above MACs that the client offers to the sshd config of the hosts, which is not great from security point of view.

[yadvr]

@miklosbarabas I got the same issue and used the following workaround in the sshd_config:

PubkeyAcceptedKeyTypes=+ssh-dss
HostKeyAlgorithms=+ssh-dss
KexAlgorithms=+diffie-hellman-group1-sha1

[yadvr]

Doc PR with workaround proposed for 4.15 release apache/cloudstack-documentation#146

[DaanHoogland]

@rhtyd that doc pr is merged do we close this or keep it open for follow up work?

[yadvr]

@DaanHoogland I've at least added this in the 4.15 release/docs, see - https://github.com/apache/cloudstack-documentation/pull/146/files

[yadvr]

@DaanHoogland ...cont -I think we can keep this open for now, I've also written an email to trilead developers to see if they can find time to address this. Longer term we probably want to revisit the whole SshHelper class and move to a new/modern library.