Skip to content

Interoperating with AD FS

Jump to bottom Edit New page
Matthew X. Economou edited this page Apr 25, 2017 · 8 revisions

Background

Microsoft Active Directory Federation Services (AD FS) version 2.0 and 3.0 cannot read the federation metadata aggregates published by InCommon, eduGAIN, and others, as it expects claims provider or relying party trust relationships to be configured singly. In order to load (or update) federation metadata into AD FS, one generally downloads a suitable aggregate, verifies it somehow, extracts the desired EntityDescriptor elements, and creates the appropriate trust relationships using the extracted metadata. However, tools like sila and pysFemma partially duplicate AD FS functionality in that AD FS can monitor (and automatically update) an entity's metadata if it's published singly over HTTPS---AD FS relies on X.509 PKI to bootstrap trust. Given a mdq-server deployment accessible over HTTPS, one can synchronize AD FS trust relationships with the parent identity federation by walking the /x-entity-list endpoint, creating claims provider or relying party trust relationships for each of the listed entities.

Caveats

The mdq-server instance must be accessible over HTTPS using a certificate trusted by the service account under which AD FS runs. This may be accomplished using a proxy server to bridge requests made over HTTPS to an mdq-server instance. For stunnel, assuming that the mdq-server instance is accessible via http://192.0.2.100/global, one could use a configuration similar to the following:

cert = /path/to/certificate
key = /path/to/private-key
 
[https]
accept = 443
connect = 192.0.2.100:80

For Apache httpd 2.4, that configuration would look something like the following:

<IfModule ssl_module>
  <VirtualHost _default_:443>
    SSLEngine on
    SSLCertificateFile /path/to/certificate
    SSLCertificatekeyFile /path/to/private-key

    <IfModule proxy_http_module>
      ProxyPassReverse "/global" http://192.0.2.100/global
      ProxyPass "/global" http://192.0.20.100/global
    </IfModule>
  </VirtualHost>
</IfModule>

Given an mdq-server deployment accessible over HTTPS, Active Directory Federation Services versions 2.0 and 3.0 cannot download metadata for entities with the / (forward slash) character in their entity IDs. This happens because AD FS converts all occurrences of %2F in the URL-encoded (a/k/a percent-encoded) entity ID back into the / character, causing mdq-server to reject the entity metadata query with a HTTP 404 (not found) response. As a workaround, encode the entity ID using SHA-1 and use that in the metadata URL, e.g., http://md.iay.org.uk/static/entities/%7Bsha1%7D8dceff94e7b25f33596fe33e6e1543387bd373b0 for entity ID https://idp2.iay.org.uk/idp/shibboleth.

AD FS 2.0 expects relying parties' signing certificates to be unique, but some SP operators re-use keying material (e.g., the same signing certificate for development and production instances). To disable this constraint, install Update Rollup 3 for AD FS 2.0 and follow these instructions in the hotfix's release notes to update the farm's database schema.

Implementation

The below PowerShell script adds or removes claims provider and relying party trusts based on data provided by mdq-server. Key features:

  • uses SHA-1 encoded entity IDs to form the metadata URLs, as discussed
  • adds or removes trusts based on changes to the metadata feed or filter criteria presented by mdq-server (using the Notes field to track this)
  • adds claim acceptance transform rules for the REFEDS R&S attribute bundle that include the appropriate scope checks (only if the IdP specifies a scope in the metadata; at the time this was written, 91 IdPs in the eduGAIN metadata did not specify a scope, e.g., https://idp.geant.org)
  • adds issuance transform rules that pass through the REFEDS R&S attribute bundle (generating these using the Active Directory claims provider trust is left as an exercise for the reader)
#### SYNC-FEDERATION-METADATA.PS1 --- Manage federation metadata for AD FS 2.0/3.0

####
#### CUSTOMIZATIONS
####

## 0=quiet, 1=verbose
$log_level = 1

## set the base URL of the MDQ service (requires HTTPS)
$mdqBaseUrl = 'https://mdq-dev.ibrsp.org/global'

## set the farm's operating mode: 'idp', 'sp', or 'proxy'; this
## determines which kinds of trusts the script will maintain
## (respectively: relying party trusts, claims provider trusts, or
## both)
$mode = 'sp'

## accept the REFEDS R&S attribute bundle (NB: includes scope check)
$acceptance_transform_rules_scoped_template = '
@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through all Name claims"
c:[Type == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name"]
 => issue(claim = c);

@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through only properly scoped eduPerson Principal Name claims"
c:[Type == "urn:oid:1.3.6.1.4.1.5923.1.1.1.6", Value =~ getemailsuffixregex("IDP_SCOPE")]
 => issue(claim = c);

@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through only properly scoped eduPerson Scoped Affiliation claims"
c:[Type == "urn:oid:1.3.6.1.4.1.5923.1.1.1.9", Value =~ getemailsuffixregex("IDP_SCOPE")]
 => issue(claim = c);

@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through all Name ID claims"
c:[Type == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier"]
 => issue(claim = c);

@RuleName = "Transform ePPN to UPN"
c:[Type == "urn:oid:1.3.6.1.4.1.5923.1.1.1.6"]
 => issue(Type = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn", Issuer = c.Issuer, OriginalIssuer = c.OriginalIssuer, Value = c.Value, ValueType = c.ValueType);

@RuleName = "Transform mail to mail"
c:[Type == "urn:oid:0.9.2342.19200300.100.1.3"]
 => issue(Type = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress", Issuer = c.Issuer, OriginalIssuer = c.OriginalIssuer, Value = c.Value, ValueType = c.ValueType);
'

## accept commonly emitted attributes (DANGER: EXCLUDES SCOPE CHECKS)
$acceptance_transform_rules_unscoped_template = '
@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through all Name claims"
c:[Type == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name"]
 => issue(claim = c);

@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through all eduPerson Principal Name claims (DANGER: NO SCOPE CHECK)"
c:[Type == "urn:oid:1.3.6.1.4.1.5923.1.1.1.6"]
 => issue(claim = c);

@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through all eduPerson Scoped Affiliation claims (DANGER: NO SCOPE CHECK)"
c:[Type == "urn:oid:1.3.6.1.4.1.5923.1.1.1.9"]
 => issue(claim = c);

@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through all Name ID claims"
c:[Type == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier"]
 => issue(claim = c);

@RuleName = "Transform ePPN to UPN"
c:[Type == "urn:oid:1.3.6.1.4.1.5923.1.1.1.6"]
 => issue(Type = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn", Issuer = c.Issuer, OriginalIssuer = c.OriginalIssuer, Value = c.Value, ValueType = c.ValueType);

@RuleName = "Transform mail to mail"
c:[Type == "urn:oid:0.9.2342.19200300.100.1.3"]
 => issue(Type = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress", Issuer = c.Issuer, OriginalIssuer = c.OriginalIssuer, Value = c.Value, ValueType = c.ValueType);
'

## send the REFEDS R&S attribute bundle to trusted relying parties
$issuance_transform_rules = '
@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through all Name claims"
c:[Type == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name"]
 => issue(claim = c);

@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through all UPN claims"
c:[Type == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn"]
 => issue(claim = c);

@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through all E-mail Address claims"
c:[Type == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress"]
 => issue(claim = c);

@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through all eduPerson Principal Name claims"
c:[Type == "urn:oid:1.3.6.1.4.1.5923.1.1.1.6"]
 => issue(claim = c);

@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through all eduPerson Scoped Affiliation claims"
c:[Type == "urn:oid:1.3.6.1.4.1.5923.1.1.1.9"]
 => issue(claim = c);

@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass through all Name ID claims"
c:[Type == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier"]
 => issue(claim = c);
'

## permit anyone to access trusted relying parties by default
$issuance_authorization_rules = '
@RuleTemplate = "AllowAllAuthzRule"
 => issue(Type = "http://schemas.microsoft.com/authorization/claims/permit", Value = "true");
'

####
#### ENVIRONMENT PREP
####

## load PowerShell snap-in on Windows Server 2008 R2 and older
if (([System.Environment]::OSVersion.Version.Major -eq 6) -and ([System.Environment]::OSVersion.Version.Minor -lt 3)) {
    Add-PSSnapin Microsoft.Adfs.PowerShell
}

## define the EPPN and EPSA claim types
if ("urn:oid:1.3.6.1.4.1.5923.1.1.1.6" -notin @(Get-AdfsClaimDescription | foreach { $_.ClaimType }))
{
    Add-AdfsClaimDescription -ClaimType "urn:oid:1.3.6.1.4.1.5923.1.1.1.6" `
      -Name "eduPerson Principal Name" `
      -IsAccepted:$true `
      -IsOffered:$true `
      -IsRequired:$false `
      -Notes "A scoped identifier for a person. It should be represented in the form `"user@scope`" where 'user' is a name-based identifier for the person and where 'scope' defines a local security domain. Each value of `"scope`" defines a namespace within which the assigned identifiers MUST be unique. Given this rule, if two eduPersonPrincipalName (ePPN) values are the same at a given point in time, they refer to the same person. There must be one and only one `"@`" sign in valid values of eduPersonPrincipalName."
}
if ("urn:oid:1.3.6.1.4.1.5923.1.1.1.9" -notin @(Get-AdfsClaimDescription | foreach { $_.ClaimType }))
{
    Add-AdfsClaimDescription -ClaimType "urn:oid:1.3.6.1.4.1.5923.1.1.1.9" `
      -Name "eduPerson Scoped Affiliation" `
      -IsAccepted:$true `
      -IsOffered:$true `
      -IsRequired:$false `
      -Notes "Specifies the person's affiliation within a particular security domain in broad categories such as student, faculty, staff, alum, etc. The values consist of a left and right component separated by an `"@`" sign. The left component is one of the values from the eduPersonAffiliation controlled vocabulary. This right-hand side syntax of eduPersonScopedAffiliation intentionally matches that used for the right-hand side values for eduPersonPrincipalName since both identify a security domain."
}

## tag federated trusts by putting the following in the `Notes` field
$notes = "DO NOT CHANGE THIS TEXT - Federated Trust - C992C076-D84D-11E5-ACFB-F6D137852114"

## convenience variables describing the farm's mode of operation
$imaSP = ($mode -in ('sp', 'proxy'))
$imaIdP = ($mode -in ('idp', 'proxy'))

## get list of existing claims providers
if ($imaSP)
{
    $existing_cp_trusts = @(Get-ADFSClaimsProviderTrust | ForEach-Object { $_.Identifier } )
}

## get list of existing relying parties
if ($imaIdP)
{
    $existing_rp_trusts = @(Get-ADFSRelyingPartyTrust | ForEach-Object { $_.Identifier } )
}

## for hashing and encoding entity IDs used in MDQ queries
$sha1 = New-Object -TypeName System.Security.Cryptography.SHA1CryptoServiceProvider
$utf8enc = New-Object -TypeName System.Text.UTF8Encoding

####
#### ENTITY PROCESSING AND TRUST ESTABLISHMENT
####

## query the MDQ service for the full list of entity IDs
$entities = Invoke-RestMethod -Uri "$mdqBaseUrl/x-entity-list" -Method Get

## if the response exceeds the 2-MiB default length limit for input
## strings, parse the JSON ourselves
if ($entities.GetType().Name -eq "String") {
    [void][System.Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions")
    $parser = New-Object -TypeName System.Web.Script.Serialization.JavaScriptSerializer
    $parser.MaxJsonLength = $entities.Length
    $entities = $parser.DeserializeObject($entities)
}

## process the entity list
$federated_entities = @()
$count = -1
foreach ($entity in $entities)
{
    ## track progress for debugging purposes
    $count++

    ## display name defaults to entity ID
    $displayName = $entity.entityID

    ## decide what to do with the entity
    $install_cp_trust = $false
    $install_rp_trust = $false
    foreach ($role in $entity.roles)
    {
        ## update the display name if set in the role descriptor
        ## (FIXME: assumes that this is what users expect to see in
        ## their native language)
        if ($role.displayName -ne $null)
        {
            $displayName = $role.displayName
        }

        ## if we are a SP and this is an IdP, install the CP trust
        if ($imaSP -and ($role.type -eq 'IDPSSODescriptor'))
        {
            $install_cp_trust = $true
            break
        }

        ## if we are an IdP and this is a SP, install the RP trust
        if ($imaIdP -and ($role.type -eq 'SPSSODescriptor'))
        {
            $install_rp_trust = $true
            break
        }
    }
    if (($install_cp_trust -eq $false) `
        -and ($install_rp_trust -eq $false))
    {
        continue
    }

    ## compute the SHA-1 hash of the entity ID, then convert it to
    ## UTF-8 encoding
    $hashedEntityId = [System.BitConverter]::ToString(
        $sha1.ComputeHash(
            $utf8enc.GetBytes($entity.entityID))).Replace("-", "").ToLower()

    ## download the entity's metadata
    $metadataUrl = "$mdqBaseUrl/entities/{sha1}$hashedEntityId"
    $metadata = Invoke-RestMethod -Uri $metadataUrl -Method Get

    ## filter out blacklisted entities
    ## https://spaces.internet2.edu/display/InCFederation/Hide+From+Discovery+Category
    $blacklisted = $false
    foreach ($entityAttribute in $metadata.EntityDescriptor.Extensions.EntityAttributes.Attribute)
    {
        if ($entityAttribute.Name -eq 'http://macedir.org/entity-category' `
            -and $entityAttribute.NameFormat -eq 'urn:oasis:names:tc:SAML:2.0:attrname-format:uri' `
            -and $entityAttribute.AttributeValue -eq 'http://refeds.org/category/hide-from-discovery')
        {
            $blacklisted = $true
            break
        }
    }
    if ($blacklisted)
    {
        continue
    }

    ## filter out entities that do not support R&S attribute release
    ## or consumption (NB: ignores distinction in the entity metadata
    ## for the R&S category between IdPs and SPs; this might be a
    ## bug)
    ## https://spaces.internet2.edu/display/InCFederation/Research+and+Scholarship+Category
    ## https://spaces.internet2.edu/display/InCFederation/Research+and+Scholarship+Entity+Metadata
    $not_rs = $true
    foreach ($entityAttribute in $metadata.EntityDescriptor.Extensions.EntityAttributes.Attribute)
    {
        if (($entityAttribute.Name -eq 'http://macedir.org/entity-category' `
             -or $entityAttribute.Name -eq 'http://macedir.org/entity-category-support') `
            -and $entityAttribute.NameFormat -eq 'urn:oasis:names:tc:SAML:2.0:attrname-format:uri' `
            -and ($entityAttribute.AttributeValue -contains 'http://id.incommon.org/category/research-and-scholarship' `
                  -or $entityAttribute.AttributeValue -contains 'http://refeds.org/category/research-and-scholarship'))
        {
            $not_rs = $false
            break
        }
    }
    if ($not_rs)
    {
        continue
    }

    ## make a list of federated identites whose trusts this script
    ## maintains; this will be used later to remove decommissioned
    ## IdPs and SPs
    $federated_entities += $entity.entityID

    ## create a new CP trust if appropriate/necessary
    if ($install_cp_trust -and ($entity.entityID -notin $existing_cp_trusts))
    {
        if (($metadata.EntityDescriptor.IDPSSODescriptor.Extensions `
            | Get-Member -MemberType Properties).Name -contains 'Scope')
        {
            $scope = $metadata.EntityDescriptor.IDPSSODescriptor.Extensions.Scope.InnerText
            $acceptance_transform_rules = $acceptance_transform_rules_scoped_template -replace 'IDP_SCOPE', $scope
        }
        else
        {
            $acceptance_transform_rules = $acceptance_transform_rules_unscoped_template;
        }
        if ($log_level -gt 0) { Write-Host ('{0}/{1}: creating CP trust for {2}' `
            -f $count, $entities.Count, $entity.entityID) }
        Add-AdfsClaimsProviderTrust -Name $displayName `
            -MetadataUrl $metadataUrl `
            -MonitoringEnabled:$true `
            -AutoUpdateEnabled:$true `
            -AcceptanceTransformRules $acceptance_transform_rules `
            -Notes $notes
    }

    ## create a new RP trust if appropriate/necessary
    if ($install_rp_trust -and ($entity.entityID -notin $existing_rp_trusts))
    {
        if ($log_level -gt 0) { Write-Host ('{0}/{1}: creating RP trust for {2}' `
            -f $count, $entities.Count, $entity.entityID) }
        Add-AdfsRelyingPartyTrust -Name $displayName `
            -MetadataUrl $metadataUrl `
            -MonitoringEnabled:$true `
            -AutoUpdateEnabled:$true `
            -IssuanceTransformRules $issuance_transform_rules `
            -IssuanceAuthorizationRules $issuance_authorization_rules `
            -Notes $notes
    }
}

####
#### CLEANUP
####

## remove trusts for any federated claims providers no longer listed in the
## metadata feed
Get-ADFSClaimsProviderTrust `
    | Where-Object {($_.Notes -eq $notes) -and ($_.Identifier -notin $federated_entities)} `
    | Remove-ADFSClaimsProviderTrust

## repeat, but for federated relying parties
Get-ADFSRelyingPartyTrust `
    | Where-Object {($_.Notes -eq $notes) -and ($_.Identifier -notin $federated_entities)} `
    | Remove-ADFSRelyingPartyTrust

#### SYNC-FEDERATION-METADATA.PS1 ends here.
Add a custom footer
Add a custom sidebar
Clone this wiki locally