LSAgent Cloud Relay error

[dshu]

- Enabled Cloud relay config on my scan server
- Installed client in cloud relay mode only, with key defined
- LSAgentLog errors with a 'The request channel timed out while waiting for a replay after 00:00:59.7812371. Increase the timeout value passed to the call to Reqeust"

I've got .NET 4.8 on the device, unrestricted internet access (no HTTPS inspection) and running LSAgent version 7.2.110.3.

Do I need to wait a little longer for the config to kick in at the back end or should it immediately work? On my web server I have the 'LsAgent access granted' green tick showing.

Thanks in advance

[nlertn]

Thanks. I followed the document below to ensure TLS 1.2 is enabled on the server as I saw errors on the errorlog under Service folder.

https://docs.microsoft.com/en-us/windows-server/security/tls/tls-registry-settings

Then rebooted the Lansweeper Server. Although the Cloud relay details finally showed up and no more errors.

However the LSAgent still spits the following error.

2020-03-27 09:10:01,516 [7] ERROR An unexpected exception was caused
System.TimeoutException: The request channel timed out while waiting for a reply after 00:00:59.8164935. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. ---> System.TimeoutException: The HTTP request to 'https://relay.lansweeper.com/EchoService.svc' has exceeded the allotted timeout of 00:00:59.9990000. The time allotted to this operation may have been a portion of a longer timeout. ---> System.Net.WebException: The operation has timed out
at System.Net.HttpWebRequest.GetResponse()
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
--- End of inner exception stack trace ---
at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason)
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
--- End of inner exception stack trace ---

Server stack trace:
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Lansweeper.Echo.Server.IService.IEchoService.GetAssetStatus(Guid agentKey, Guid assetId)
at Lansweeper.Echo.Client.SA.AssetServiceAgent..(IEchoService )
at Lansweeper.Echo.Client.SA.Wcf.WcfServiceInvoker.InvokeService[T,R](Func`2 invokeHandler)
at Lansweeper.Echo.Client.SA.AssetServiceAgent.GetAssetStatus(Guid agentKey, Guid assetId)
at Lansweeper.Echo.Client.SA.RelayCommunicationSender.GetAssetStatus()
at Lansweeper.Echo.Client.BL.ScanningManager.ExecuteScan(IAppSettingsClient appSettings, EchoIniInfo iniSettings, IIniFileManager iniFileManager)
at Lansweeper.Echo.Client.WindowsService.ServiceStart.StartCore()


I then note that Windows 10 has TLS 1.2 enabled by default, but will add the registry keys in anyway and reboot. Will see if this works.

[nlertn]

nlertn wrote:
Thanks. I followed the document below to ensure TLS 1.2 is enabled on the server as I saw errors on the errorlog under Service folder.

https://docs.microsoft.com/en-us/windows-server/security/tls/tls-registry-settings

Then rebooted the Lansweeper Server. Although the Cloud relay details finally showed up and no more errors.

However the LSAgent still spits the following error.

2020-03-27 09:10:01,516 [7] ERROR An unexpected exception was caused
System.TimeoutException: The request channel timed out while waiting for a reply after 00:00:59.8164935. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. ---> System.TimeoutException: The HTTP request to 'https://relay.lansweeper.com/EchoService.svc' has exceeded the allotted timeout of 00:00:59.9990000. The time allotted to this operation may have been a portion of a longer timeout. ---> System.Net.WebException: The operation has timed out
at System.Net.HttpWebRequest.GetResponse()
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
--- End of inner exception stack trace ---
at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason)
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
--- End of inner exception stack trace ---

Server stack trace:
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Lansweeper.Echo.Server.IService.IEchoService.GetAssetStatus(Guid agentKey, Guid assetId)
at Lansweeper.Echo.Client.SA.AssetServiceAgent..(IEchoService )
at Lansweeper.Echo.Client.SA.Wcf.WcfServiceInvoker.InvokeService[T,R](Func`2 invokeHandler)
at Lansweeper.Echo.Client.SA.AssetServiceAgent.GetAssetStatus(Guid agentKey, Guid assetId)
at Lansweeper.Echo.Client.SA.RelayCommunicationSender.GetAssetStatus()
at Lansweeper.Echo.Client.BL.ScanningManager.ExecuteScan(IAppSettingsClient appSettings, EchoIniInfo iniSettings, IIniFileManager iniFileManager)
at Lansweeper.Echo.Client.WindowsService.ServiceStart.StartCore()


I then note that Windows 10 has TLS 1.2 enabled by default, but will add the registry keys in anyway and reboot. Will see if this works.

After adding the registry key for TLS 1.2 on Windows 10 and rebooted, the error went away


SOLVED!!

[Bruce_B]

The increase in LsAgent relay usage due to COVID-19 has likely led to an increase in timeouts. We're investigating and making improvements.

Note that the clients will retry scanning every 15 minutes. If a timeout was thrown during the last attempt, a new attempt is made at that time. Similarly, the scanning service checks in with the relay every 10 minutes to retrieve assets. It's not critical for each connection attempt to be successful. If one fails, assets should be pulled in on the next successful one.

This can and will cause some delays though, which we're addressing, but no action is needed on the end of the user.

Note that outside of this, the same requirements still apply:
-The client and scanning server must have TLS 1.2 available
-Both the scanning server and client must have an internet connection
-The client must have a direct internet connection (no proxy)

[nlertn]

Did a Wireshark. Looks like the last part of the lsagent is a ping that failed.



No. Time Source Destination Protocol Length Info
329 4.125186 192.168.2.100 52.165.163.223 TCP 54 53806 → 443 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
895 11.198193 192.168.2.100 52.165.163.223 TCP 66 53951 → 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 WS=256 SACK_PERM=1
908 11.253855 192.168.2.100 52.165.163.223 TCP 54 53951 → 443 [ACK] Seq=1 Ack=1 Win=132352 Len=0
910 11.266721 192.168.2.100 52.165.163.223 TLSv1.2 230 Client Hello
921 11.331861 192.168.2.100 52.165.163.223 TCP 54 53951 → 443 [ACK] Seq=177 Ack=2921 Win=132352 Len=0
924 11.332885 192.168.2.100 52.165.163.223 TCP 54 53951 → 443 [ACK] Seq=177 Ack=5155 Win=132352 Len=0
925 11.334221 192.168.2.100 52.165.163.223 TLSv1.2 180 Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message
931 11.405754 192.168.2.100 52.165.163.223 TLSv1.2 330 Application Data
942 11.463470 192.168.2.100 52.165.163.223 TLSv1.2 322 Application Data
2145 71.087051 192.168.2.100 52.165.163.223 TCP 54 53951 → 443 [FIN, ACK] Seq=847 Ack=5260 Win=132352 Len=0
2711 83.285979 192.168.2.100 52.165.163.223 ICMP 74 Echo (ping) request id=0x0001, seq=8/2048, ttl=128 (no response found!)

[nlertn]

Support responded and asked to update license. No change. Its the cloud relay not the server or the lsagent, but doesn't seem to get through to them.

[nlertn]

Noted there is a new version 110.5. Still no help. I believe the lansweeper cloud relay for North America may be down. I tried to raise a ticket with support@lansweeper.com and have no response for 2 days.

[nlertn]

Hi

I am having the same error and no response from support. Anyone got a solution?

[nlertn]

nlertn wrote:
Hi

I am having the same error and no response from support. Anyone got a solution?

Still no response. I believe the server is down based on the error. Something to do with the server binding timeout.

2020-03-25 12:54:02,188 [7] ERROR An unexpected exception was caused
System.TimeoutException: The request channel timed out while waiting for a reply after 00:00:59.8892966. Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been a portion of a longer timeout. ---> System.TimeoutException: The HTTP request to 'https://relay.lansweeper.com/EchoService.svc' has exceeded the allotted timeout of 00:00:59.9980000. The time allotted to this operation may have been a portion of a longer timeout. ---> System.Net.WebException: The operation has timed out
at System.Net.HttpWebRequest.GetResponse()
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
--- End of inner exception stack trace ---
at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason)
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
--- End of inner exception stack trace ---

Server stack trace:
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Lansweeper.Echo.Server.IService.IEchoService.GetAssetStatus(Guid agentKey, Guid assetId)
at Lansweeper.Echo.Client.SA.AssetServiceAgent..(IEchoService )
at Lansweeper.Echo.Client.SA.Wcf.WcfServiceInvoker.InvokeService[T,R](Func`2 invokeHandler)
at Lansweeper.Echo.Client.SA.AssetServiceAgent.GetAssetStatus(Guid agentKey, Guid assetId)
at Lansweeper.Echo.Client.SA.RelayCommunicationSender.GetAssetStatus()
at Lansweeper.Echo.Client.BL.ScanningManager.ExecuteScan(IAppSettingsClient appSettings, EchoIniInfo iniSettings, IIniFileManager iniFileManager)
at Lansweeper.Echo.Client.WindowsService.ServiceStart.StartCore()

[dmorgan1]

I am seeing the same error in many LSagent logs. Most of my machines are checking in with LSagent correctly but not all. I have reinstalled LSagent on a number of machines and am still seeing this error. In some cases the machines that are giving this error later check in successfully, sometimes hours after the agent was installed. Let me know what else I can provide to best address the issue