diff --git a/Doc/ReleaseNotes-ISHRemote-8.3.md b/Doc/ReleaseNotes-ISHRemote-8.3.md index 2e78645..4465866 100644 --- a/Doc/ReleaseNotes-ISHRemote-8.3.md +++ b/Doc/ReleaseNotes-ISHRemote-8.3.md @@ -31,6 +31,7 @@ The below text describes the delta compared to fielded release ISHRemote v8.2. * Migrated all 58 `*.Tests.ps1` files from Pester v5 to Pester v6 (`Should -Be` to `Should-Be`, `Should -BeExactly` to `Should-BeString -CaseSensitive`, `Should -Not -BeNullOrEmpty` to `Should-NotBeNull`, `Should -Throw "msg"` to `Should-Throw -ExceptionMessage "msg"`, etc.). CI install gates updated to `-MinimumVersion 6.0.0`. Classic `Should -Not -Throw` retained as there is no `Should-NotThrow` equivalent in Pester 6. Hardened the library for parallel test execution by replacing the process-wide `TrisoftCmdletLogger` singleton with per-cmdlet `ILogger` routing and adding a double-checked lock on `IshSession._ishTypeFieldSetup` to eliminate Collection was modified races under `Run.Parallel = $true`. CI Pester invocations now use `New-PesterConfiguration` (with `Run.Parallel = $false`) so parallel mode can be toggled in one place when ready. See #242, #265, #266. * Fixed `New-IshSession` (protocol `WcfSoapWithOpenIdConnect`, PowerShell 7.2+/.NET 6.0+) throwing `FileLoadException: Could not load file or assembly 'Microsoft.IdentityModel.Tokens, Version=8.14.0.0, ...'. The located assembly's manifest definition does not match the assembly reference.` on machines where a different build of `Microsoft.IdentityModel.Tokens` (and related `Duende.IdentityModel.OidcClient`) is registered in the Global Assembly Cache (GAC) — observed on machines with Microsoft Intune Management Extension installed. `AppDomainModuleAssemblyInitializer` now force-loads ISHRemote's own bundled copies of `Duende.IdentityModel`, `Duende.IdentityModel.OidcClient`, `Microsoft.IdentityModel.Abstractions/.Logging/.Tokens/.Tokens.Saml/.Xml` as early as possible during module import, and `SessionCmdlet.BeginProcessing` now reports the full forced list over `-Verbose`. Root cause for the `Duende.IdentityModel.OidcClient` variant: `InfoShareOpenIdConnectSystemBrowser` was `public` and implemented `Duende.IdentityModel.OidcClient.Browser.IBrowser`, which put it in `Trisoft.ISHRemote.dll`'s exported types, forcing PowerShell's own binary-module cmdlet discovery (`Assembly.GetExportedTypes()`) to resolve `Duende.IdentityModel.OidcClient` before `IModuleAssemblyInitializer.OnImport()` ever ran — see Breaking Changes - Code. A new `TestPrerequisite.Tests.ps1` check asserts no assembly is ever loaded from the GAC on PowerShell Core. See #272. Thanks @ddemeyer * Fixed cmdlets over protocol `WcfSoapWithOpenIdConnect` occasionally throwing `An unsecured or incorrectly secured fault was received from the other party` on the first SOAP call after a channel fault, requiring the user to re-run the same cmdlet for it to succeed (the existing #201/#219 rebuild-on-next-call logic only kicked in on a second, separate call). Each `Get*25Channel()` method in `InfoShareWcfSoapWithOpenIdConnectConnection` now returns the channel wrapped in a new `RetryOnFaultProxy` (`System.Reflection.DispatchProxy`) that catches `CommunicationException`/`FaultException` on the actual SOAP call, rebuilds the channel via the existing rebuild logic, and retries exactly once within the same cmdlet invocation before propagating any further failure — with the original exception type/stack trace preserved. Requires a new `net48`-only NuGet dependency, `System.Reflection.DispatchProxy` (built into the BCL on `net6.0`/`net10.0`). See #273. Thanks @ddemeyer +* Extended `New-IshSession` with an automatic `User-Agent` fallback on PowerShell 7+ so it no longer gets blocked outright by cloud WAF bot-management rule groups (AWS WAF Bot Control, Azure Front Door/App Gateway WAF bot manager rules, GCP Cloud Armor adaptive protection/bot management) that inspect the `User-Agent` header on every request. Measured behavior against a live WAF-fronted environment: sending no `User-Agent` at all (today's default) or a bare `Product/Version` token (e.g. `ISHRemote/8.3.0`) both get bucketed with known non-browser tooling (`curl`, `python-requests`, ...) and blocked with `HTTP 403 Forbidden`. When WAF blocks the connection on the first `403` from the `connectionconfiguration.xml` probe, `New-IshSession` retries once with `Mozilla/5.0 (compatible; ISHRemote/{version}; +https://github.com/rws/ISHRemote)`; on success that header sticks for the session's lifetime and every subsequent request carries it. Windows PowerShell 5.1 (`net48`) will warn instead of a raw WCF exception, recommending you to ask your Tridion Docs administrator to adjust the WAF bot-management rule for this endpoint, or switch to PowerShell 7+ where the fallback is applied automatically. See #275. Thanks @ddemeyer diff --git a/Source/ISHRemote/Trisoft.ISHRemote/Connection/InfoShareWcfSoapUserAgentClientMessageInspector.cs b/Source/ISHRemote/Trisoft.ISHRemote/Connection/InfoShareWcfSoapUserAgentClientMessageInspector.cs new file mode 100644 index 0000000..cb8f2e0 --- /dev/null +++ b/Source/ISHRemote/Trisoft.ISHRemote/Connection/InfoShareWcfSoapUserAgentClientMessageInspector.cs @@ -0,0 +1,134 @@ +/* +* Copyright (c) 2014 All Rights Reserved by the SDL Group. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + +#if !NET48 +using System.Net; +using System.ServiceModel; +using System.ServiceModel.Channels; +using System.ServiceModel.Description; +using System.ServiceModel.Dispatcher; +#endif + +namespace Trisoft.ISHRemote.Connection +{ + /// + /// Mutable holder for a lazily-decided User-Agent value. Reference type on purpose: + /// creates exactly one instance and hands it to every + /// endpoint behavior at construction time. If a cloud WAF (AWS WAF Bot Control, Azure Front Door/App Gateway + /// WAF bot manager, GCP Cloud Armor) later blocks a bare/no User-Agent request, flips + /// once - every already-constructed InfoShareWcfSoapUserAgentClientMessageInspector + /// observes the change immediately on its next outgoing message, without rebuilding any channel. A plain + /// string field could not do this: strings are immutable, so passing one at construction time only ever + /// shares a snapshot, never the later mutation. See #275. + /// + /// + /// Declared unconditionally (not under #if !NET48) because - which builds for + /// net48/net6.0/net10.0 alike - owns and mutates the single instance from its HttpClient-based + /// LoadConnectionConfiguration fallback regardless of target framework. Only the WCF endpoint-behavior wiring + /// below that reads it is net6.0+/net10.0-only, because on net48 the WcfSoapWithOpenIdConnect SOAP channels are + /// built with ChannelFactory.CreateChannelWithIssuedToken(...), which does not go through EndpointBehaviors/ + /// IClientMessageInspector at all - see IshSession.CreateInfoShareWcfSoapWithOpenIdConnectConnection for the + /// resulting net48 limitation (SOAP calls cannot carry the fallback header; a Write-Warning is emitted instead). + /// + internal sealed class UserAgentState + { + /// + /// Null (so header omitted, today's default behavior) until a 403 is observed against a User-Agent-sensitive + /// WAF rule, at which point this becomes the RFC-sanctioned crawler self-identification convention + /// (same format Googlebot/Bingbot use) for the remaining lifetime of the owning . + /// + public string Value { get; set; } + } + +#if !NET48 + /// + /// Sets the outgoing SOAP message's HTTP User-Agent header whenever is + /// non-null. Added as a next to the existing + /// bearerCredentials endpoint behavior wiring in . + /// See #275. + /// + internal sealed class InfoShareWcfSoapUserAgentClientMessageInspector : IClientMessageInspector + { + private readonly UserAgentState _userAgentState; + + internal InfoShareWcfSoapUserAgentClientMessageInspector(UserAgentState userAgentState) + { + _userAgentState = userAgentState; + } + + public object BeforeSendRequest(ref Message request, IClientChannel channel) + { + if (_userAgentState.Value != null) + { + object propertyObject; + HttpRequestMessageProperty httpRequestMessageProperty; + if (request.Properties.TryGetValue(HttpRequestMessageProperty.Name, out propertyObject)) + { + httpRequestMessageProperty = (HttpRequestMessageProperty)propertyObject; + } + else + { + httpRequestMessageProperty = new HttpRequestMessageProperty(); + request.Properties[HttpRequestMessageProperty.Name] = httpRequestMessageProperty; + } + httpRequestMessageProperty.Headers[HttpRequestHeader.UserAgent] = _userAgentState.Value; + } + return null; + } + + public void AfterReceiveReply(ref Message reply, object correlationState) + { + // no-op, only outgoing requests need the User-Agent header + } + } + + /// + /// Standard passthrough wiring + /// into a WCF client channel's . Constructed once per / + /// holding a reference to the same + /// so a later flip is visible to every channel immediately. See #275. + /// + internal sealed class InfoShareWcfSoapUserAgentEndpointBehavior : IEndpointBehavior + { + private readonly UserAgentState _userAgentState; + + internal InfoShareWcfSoapUserAgentEndpointBehavior(UserAgentState userAgentState) + { + _userAgentState = userAgentState; + } + + public void AddBindingParameters(ServiceEndpoint endpoint, BindingParameterCollection bindingParameters) + { + // no-op + } + + public void ApplyClientBehavior(ServiceEndpoint endpoint, ClientRuntime clientRuntime) + { + clientRuntime.ClientMessageInspectors.Add(new InfoShareWcfSoapUserAgentClientMessageInspector(_userAgentState)); + } + + public void ApplyDispatchBehavior(ServiceEndpoint endpoint, EndpointDispatcher endpointDispatcher) + { + // no-op, client-side only + } + + public void Validate(ServiceEndpoint endpoint) + { + // no-op + } + } +#endif +} diff --git a/Source/ISHRemote/Trisoft.ISHRemote/Connection/InfoShareWcfSoapWithOpenIdConnectConnection.cs b/Source/ISHRemote/Trisoft.ISHRemote/Connection/InfoShareWcfSoapWithOpenIdConnectConnection.cs index 2097191..97fdade 100644 --- a/Source/ISHRemote/Trisoft.ISHRemote/Connection/InfoShareWcfSoapWithOpenIdConnectConnection.cs +++ b/Source/ISHRemote/Trisoft.ISHRemote/Connection/InfoShareWcfSoapWithOpenIdConnectConnection.cs @@ -233,6 +233,14 @@ internal sealed class InfoShareWcfSoapWithOpenIdConnectConnection : InfoShareOpe /// private BackgroundTask25ServiceReference.BackgroundTaskClient _backgroundTaskClient; private BackgroundTask25ServiceReference.BackgroundTask _backgroundTaskServiceReference; +#if !NET48 + /// + /// Endpoint behavior wiring the shared onto every SOAP channel next to the + /// existing bearerCredentials behavior, so a User-Agent fallback set later (see #275) is carried by every + /// subsequent call on this connection without rebuilding any channel. + /// + private readonly InfoShareWcfSoapUserAgentEndpointBehavior _userAgentEndpointBehavior; +#endif #endregion Private Members #region Constructors @@ -242,9 +250,13 @@ internal sealed class InfoShareWcfSoapWithOpenIdConnectConnection : InfoShareOpe /// Instance of Interfaces.ILogger implementation /// Incoming reused, probably Ssl/Tls initialized already. /// OpenIdConnect connection parameters to be shared with WcfSoapWithOpenIdConnect and OpenApiWithOpenIdConnect - public InfoShareWcfSoapWithOpenIdConnectConnection(ILogger logger, HttpClient httpClient, InfoShareOpenIdConnectConnectionParameters infoShareOpenIdConnectConnectionParameters) + /// Shared mutable User-Agent fallback holder (see #275); null-safe, but pass the same instance IshSession uses for LoadConnectionConfiguration so a WAF-triggered fallback is visible on both HttpClient and WCF traffic. + public InfoShareWcfSoapWithOpenIdConnectConnection(ILogger logger, HttpClient httpClient, InfoShareOpenIdConnectConnectionParameters infoShareOpenIdConnectConnectionParameters, UserAgentState userAgentState) : base(logger, httpClient, infoShareOpenIdConnectConnectionParameters) { +#if !NET48 + _userAgentEndpointBehavior = new InfoShareWcfSoapUserAgentEndpointBehavior(userAgentState ?? new UserAgentState()); +#endif _logger.WriteDebug($"InfoShareWcfSoapWithOpenIdConnectConnection InfoShareWSUrl[{_connectionParameters.InfoShareWSUrl}]"); if (_connectionParameters.Tokens == null) { @@ -579,6 +591,7 @@ private Annotation25ServiceReference.Annotation EnsureAnnotation25Channel() _annotationClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_annotationClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _annotationClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _annotationClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _annotationClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -648,6 +661,7 @@ private Application25ServiceReference.Application EnsureApplication25Channel() _applicationClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_applicationClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _applicationClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _applicationClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _applicationClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -718,6 +732,7 @@ private DocumentObj25ServiceReference.DocumentObj EnsureDocumentObj25Channel() _documentObjClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_documentObjClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _documentObjClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _documentObjClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _documentObjClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -788,6 +803,7 @@ private Folder25ServiceReference.Folder EnsureFolder25Channel() _folderClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_folderClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _folderClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _folderClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _folderClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -858,6 +874,7 @@ private User25ServiceReference.User EnsureUser25Channel() _userClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_userClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _userClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _userClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _userClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -928,6 +945,7 @@ private UserRole25ServiceReference.UserRole EnsureUserRole25Channel() _userRoleClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_userRoleClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _userRoleClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _userRoleClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _userRoleClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -998,6 +1016,7 @@ private UserGroup25ServiceReference.UserGroup EnsureUserGroup25Channel() _userGroupClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_userGroupClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _userGroupClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _userGroupClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _userGroupClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -1068,6 +1087,7 @@ private ListOfValues25ServiceReference.ListOfValues EnsureListOfValues25Channel( _listOfValuesClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_listOfValuesClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _listOfValuesClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _listOfValuesClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _listOfValuesClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -1138,6 +1158,7 @@ private PublicationOutput25ServiceReference.PublicationOutput EnsurePublicationO _publicationOutputClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_publicationOutputClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _publicationOutputClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _publicationOutputClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _publicationOutputClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -1208,6 +1229,7 @@ private OutputFormat25ServiceReference.OutputFormat EnsureOutputFormat25Channel( _outputFormatClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_outputFormatClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _outputFormatClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _outputFormatClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _outputFormatClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -1278,6 +1300,7 @@ private Settings25ServiceReference.Settings EnsureSettings25Channel() _settingsClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_settingsClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _settingsClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _settingsClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _settingsClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -1348,6 +1371,7 @@ private EDT25ServiceReference.EDT EnsureEDT25Channel() _EDTClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_EDTClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _EDTClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _EDTClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _EDTClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -1418,6 +1442,7 @@ private EventMonitor25ServiceReference.EventMonitor EnsureEventMonitor25Channel( _eventMonitorClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_eventMonitorClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _eventMonitorClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _eventMonitorClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _eventMonitorClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -1488,6 +1513,7 @@ private Baseline25ServiceReference.Baseline EnsureBaseline25Channel() _baselineClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_baselineClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _baselineClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _baselineClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _baselineClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -1558,6 +1584,7 @@ private MetadataBinding25ServiceReference.MetadataBinding EnsureMetadataBinding2 _metadataBindingClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_metadataBindingClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _metadataBindingClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _metadataBindingClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _metadataBindingClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -1628,6 +1655,7 @@ private Search25ServiceReference.Search EnsureSearch25Channel() _searchClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_searchClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _searchClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _searchClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _searchClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -1698,6 +1726,7 @@ private TranslationJob25ServiceReference.TranslationJob EnsureTranslationJob25Ch _translationJobClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_translationJobClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _translationJobClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _translationJobClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _translationJobClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -1768,6 +1797,7 @@ private TranslationTemplate25ServiceReference.TranslationTemplate EnsureTranslat _translationTemplateClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_translationTemplateClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _translationTemplateClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _translationTemplateClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _translationTemplateClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) @@ -1839,6 +1869,7 @@ private BackgroundTask25ServiceReference.BackgroundTask EnsureBackgroundTask25Ch _backgroundTaskClient.ChannelFactory.Endpoint.EndpointBehaviors.Remove(_backgroundTaskClient.ChannelFactory.Credentials); var bearerCredentials = GetBearerCredentials(); _backgroundTaskClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(bearerCredentials); + _backgroundTaskClient.ChannelFactory.Endpoint.EndpointBehaviors.Add(_userAgentEndpointBehavior); _backgroundTaskClient.ClientCredentials.ServiceCertificate.Authentication.CertificateValidationMode = System.ServiceModel.Security.X509CertificateValidationMode.None; if (_connectionParameters.IgnoreSslPolicyErrors) diff --git a/Source/ISHRemote/Trisoft.ISHRemote/Objects/Public/IshSession.cs b/Source/ISHRemote/Trisoft.ISHRemote/Objects/Public/IshSession.cs index 03b5eb0..2fe3112 100644 --- a/Source/ISHRemote/Trisoft.ISHRemote/Objects/Public/IshSession.cs +++ b/Source/ISHRemote/Trisoft.ISHRemote/Objects/Public/IshSession.cs @@ -73,6 +73,14 @@ public class IshSession : IDisposable // one HttpClient per IshSession with potential certificate overwrites which can be reused across requests private readonly HttpClient _httpClient; /// + /// Shared mutable holder for the User-Agent fallback value (see #275). Null until a cloud WAF (AWS WAF Bot + /// Control, Azure Front Door/App Gateway WAF bot manager, GCP Cloud Armor) is observed blocking a bare/no + /// User-Agent request with a 403, at which point it is set once for the lifetime of this IshSession and + /// every subsequent HttpClient/WCF request carries it. Reference type on purpose - see + /// Connection/InfoShareWcfSoapUserAgentClientMessageInspector.cs for why a plain string field cannot do this. + /// + private readonly UserAgentState _userAgentState = new UserAgentState(); + /// /// OpenIdConnect Client Application Id that is typically configured in Access Management (ISHID) to allow a local redirect (http://127.0.0.1:SomePort/) /// This option is not typically used but allows validating other applications like Tridion_Docs_Content_Importer /// @@ -259,7 +267,22 @@ private IshConnectionConfiguration LoadConnectionConfiguration(Uri connectionCon var responseMessage = _httpClient.GetAsync(connectionConfigurationUri).GetAwaiter().GetResult(); if (!responseMessage.IsSuccessStatusCode) { - throw new ArgumentException($"LoadConnectionConfiguration uri[{connectionConfigurationUri}] timeout[{_httpClient.Timeout}] failed with StatusCode[{responseMessage.StatusCode}]"); + // Some cloud WAFs (AWS WAF Bot Control, Azure Front Door/App Gateway WAF bot manager, GCP Cloud + // Armor) bucket a missing or bare Product/Version User-Agent with known non-browser tooling and + // return 403, while accepting the RFC-sanctioned crawler self-identification convention (same + // format Googlebot/Bingbot use). Retry exactly once with that fallback before giving up. See #275. + if (responseMessage.StatusCode == HttpStatusCode.Forbidden && _userAgentState.Value == null) + { + _userAgentState.Value = $"Mozilla/5.0 (compatible; ISHRemote/{ClientIshVersion}; +https://github.com/rws/ISHRemote)"; + _logger.WriteVerbose($"LoadConnectionConfiguration uri[{connectionConfigurationUri}] failed with StatusCode[Forbidden], retrying once with fallback User-Agent[{_userAgentState.Value}]"); + _httpClient.DefaultRequestHeaders.UserAgent.Clear(); + _httpClient.DefaultRequestHeaders.UserAgent.ParseAdd(_userAgentState.Value); + responseMessage = _httpClient.GetAsync(connectionConfigurationUri).GetAwaiter().GetResult(); + } + if (!responseMessage.IsSuccessStatusCode) + { + throw new ArgumentException($"LoadConnectionConfiguration uri[{connectionConfigurationUri}] timeout[{_httpClient.Timeout}] failed with StatusCode[{responseMessage.StatusCode}]"); + } } string response = responseMessage.Content.ReadAsStringAsync().GetAwaiter().GetResult(); //_logger.WriteDebug($"LoadConnectionConfiguration response[{response}]"); @@ -280,12 +303,63 @@ private void CreateInfoShareWcfSoapWithWsTrustConnection() private void CreateInfoShareWcfSoapWithOpenIdConnectConnection() { _logger.WriteVerbose($"CreateInfoShareWcfSoapWithOpenIdConnectConnection"); - _infoShareWcfSoapWithOpenIdConnectConnection = new InfoShareWcfSoapWithOpenIdConnectConnection(_logger, _httpClient, _infoShareOpenIdConnectConnectionParameters); + _infoShareWcfSoapWithOpenIdConnectConnection = new InfoShareWcfSoapWithOpenIdConnectConnection(_logger, _httpClient, _infoShareOpenIdConnectConnectionParameters, _userAgentState); // application proxy to get server version or authentication context init is a must as it also confirms credentials, can take up to 1s _logger.WriteDebug("CreateInfoShareWcfSoapWithOpenIdConnectConnection _serverVersion GetApplication25Channel"); var application25Proxy = _infoShareWcfSoapWithOpenIdConnectConnection.GetApplication25Channel(); _logger.WriteDebug("CreateInfoShareWcfSoapWithOpenIdConnectConnection _serverVersion GetApplication25Channel.GetVersion"); - _serverVersion = new IshVersion(application25Proxy.GetVersion()); + try + { + _serverVersion = new IshVersion(application25Proxy.GetVersion()); + } + catch (Exception ex) when (IsForbiddenWcfFault(ex)) + { +#if NET48 + // On net48, WcfSoapWithOpenIdConnect SOAP channels are built over ChannelFactory.CreateChannelWithIssuedToken(...) + // (see InfoShareWcfSoapWithOpenIdConnectConnection), which does not go through EndpointBehaviors/ + // IClientMessageInspector at all. There is no transport-level hook available here to attach a + // fallback User-Agent header to SOAP traffic, unlike the plain HttpClient calls in + // LoadConnectionConfiguration. So, unlike net6.0+/net10.0, we cannot retry our way out of this - the + // best we can do is fail with an actionable message instead of the raw WCF exception. See #275. + _logger.WriteWarning($"CreateInfoShareWcfSoapWithOpenIdConnectConnection GetVersion failed with a 403 Forbidden-shaped fault, most likely caused by a cloud WAF (AWS WAF Bot Control, Azure Front Door/App Gateway WAF bot manager, GCP Cloud Armor) bot-management rule blocking SOAP requests without a recognized browser User-Agent. On Windows PowerShell 5.1 (.NET Framework 4.8), ISHRemote cannot work around this for SOAP-based calls used by protocol WcfSoapWithOpenIdConnect. Ask your Tridion Docs administrator to adjust the WAF bot-management rule for this endpoint, or switch to PowerShell 7+ (pwsh) where this fallback is applied automatically."); + throw; +#else + if (_userAgentState.Value == null) + { + _userAgentState.Value = $"Mozilla/5.0 (compatible; ISHRemote/{ClientIshVersion}; +https://github.com/rws/ISHRemote)"; + } + _logger.WriteVerbose($"CreateInfoShareWcfSoapWithOpenIdConnectConnection GetVersion failed with a 403 Forbidden-shaped fault, retrying once with fallback User-Agent[{_userAgentState.Value}]"); + _serverVersion = new IshVersion(application25Proxy.GetVersion()); +#endif + } + } + + /// + /// Walks the exception chain (WCF's exception wrapping around a 403 is not fully consistent across .NET + /// targets - it can surface as CommunicationException/ProtocolException wrapping a WebException, or as + /// System.Net.Http.HttpRequestException wrapping a status code, depending on binding/transport) looking for + /// a Forbidden (403) signal. See #275. + /// + private static bool IsForbiddenWcfFault(Exception ex) + { + for (var current = ex; current != null; current = current.InnerException) + { + if (current is WebException webException && + webException.Response is HttpWebResponse httpWebResponse && + httpWebResponse.StatusCode == HttpStatusCode.Forbidden) + { + return true; + } + if (current is System.Net.Http.HttpRequestException && current.Message != null && current.Message.Contains("403")) + { + return true; + } + if (current.Message != null && current.Message.IndexOf("(403) Forbidden", StringComparison.OrdinalIgnoreCase) >= 0) + { + return true; + } + } + return false; } private void CreateOpenApiWithOpenIdConnectConnection()