SfB Hybrid setups: Remember to renew your OAuth certs with Online workloads, after updating your on-prem OAuth cert

Renewed my SfB on-prem OAuth cert and started getting these errors

Log Name:      Lync Server

Source:        LS Storage Service

Event ID:      32050

Storage Service had an OAuth STS request failure.

#CTX#{ctx:{traceId:2107372237, activityId:"db71b116-b4ea-430f-958f-12662b997bd4"}}#CTX#

Recv RST response, failed, sts=https://accounts.accesscontrol.windows.net/092a1ba4-a4fe-4172-970e-7ab3035e7c94/tokens/OAuth/2, resource=00000002-0000-0ff1-ce00-000000000000/autodiscover-s.outlook.com@domain.com, ex=The remote server returned an error: (401) Unauthorized….

You will also notice that the Test-CsExStorageConnectivity command fails (Test-CsExStorageConnectivity -SipUri Test_User@domain.com)

Renew OAuth with new Cert

1.  Export the new OAuth cert from MMC>Certificates on FE using BASE-64 without private key

2. Create a session with SfB online + MSOL 

$msolcred = get-credential admin@domain.com

$session = New-CsOnlineSession -Credential $msolcred -OverrideAdminDomain "domain.onmicrosoft.com"

Import-PSSession $session -AllowClobber_

Connect-MsolService -credential $msolcred

3.  Import and assign cert 

$certificate = New-Object System.Security.Cryptography.X509Certificates.X509Certificate

$certificate.Import("C:\temp\Oauth_2017.cer")

$binaryValue = $certificate.GetRawCertData()

$credentialsValue = [System.Convert]::ToBase64String($binaryValue)

3.  Get current KeyIDs for Certs (enter 0 for ReturnKeyValues)

Get-MsolServicePrincipal -AppPrincipalID 00000004-0000-0ff1-ce00-000000000000  #Lync

Get-MsolServicePrincipal -AppPrincipalID 00000002-0000-0ff1-ce00-000000000000  #Exchange

get-MsolServicePrincipalCredential -AppPrincipalId 00000004-0000-0ff1-ce00-000000000000

get-MsolServicePrincipalCredential -AppPrincipalId 00000002-0000-0ff1-ce00-000000000000

5.  Use the KeyIDs (that you got from above step) to remove current certs

Remove-MsolServicePrincipalCredential -AppPrincipalId 00000004-0000-0ff1-ce00-000000000000 -KeyIds @("00000000-0000-0000-0000-000000000000")

Remove-MsolServicePrincipalCredential -AppPrincipalId 00000002-0000-0ff1-ce00-000000000000 -KeyIds @("00000000-0000-0000-0000-000000000001")

6.  Assign new cert

New-MsolServicePrincipalCredential -AppPrincipalId 00000004-0000-0ff1-ce00-000000000000 -Type Asymmetric -Usage Verify -Value $credentialsValue

New-MsolServicePrincipalCredential -AppPrincipalId 00000002-0000-0ff1-ce00-000000000000 -Type Asymmetric -Usage Verify -Value $credentialsValue

7.  Make sure your edge servers are replicated (OAuth uses federation via them) 

Get-CsManagementStoreReplicationStatus | ft

Invoke-CsManagementStoreReplication

8.  Verify

Get-MsolServicePrincipal -AppPrincipalID 00000004-0000-0ff1-ce00-000000000000

Get-MsolServicePrincipal -AppPrincipalID 00000002-0000-0ff1-ce00-000000000000

Get-MsolServicePrincipal -AppPrincipalID 00000004-0000-0ff1-ce00-000000000000 | select serviceprincipalnames -ExpandProperty serviceprincipalnames

Get-MsolServicePrincipal -AppPrincipalID 00000002-0000-0ff1-ce00-000000000000 | select serviceprincipalnames -ExpandProperty serviceprincipalnames

Test-CsExStorageConnectivity –SipUri lync_tester1@domain.com # SfB on-prem user

Test-CsExStorageConnectivity –SipUri lync_tester2@domain.com # SfB online user

Cleared Log event (OAuth successful) 

Log Name:      Lync Server

Source:        LS Storage Service

Event ID:      32052

Task Category: (4006)

Description:

OAuth STS was properly configured for Storage Service.

#CTX#{ctx:{traceId:1596246623, activityId:"77278542-b703-4f56-9655-9f40fe99c04b"}}#CTX#

GetAppToken succeeded for request with sts=https://accounts.accesscontrol.windows.net/092a1ba4-a4fe-4172-970e-7ab3035e7c94/tokens/OAuth/2

Additional (verbose) references:

• http://www.ucprimer.com/tech-blog/provisioning-exchange-online-for-lync-hybrid-part-ii
• https://technet.microsoft.com/en-us/library/jj204990.aspx