IDEM MDX Instructions¶
The following instructions are meant to be used to retrive metadata by the IDEM MDX service.
For all the configuration a cache duration value of 1 hour (3600 seconds) is chosen. This choice was made taking into account that all the metadata are signed every hour and the validUntil field in the metadata file is 48 hours.
For Shibboleth, we set a maxCacheDuration value equal to 48 hours (172800 seconds), with a refreshDelayFactor of 0.025 to get in case of changes the metadata refresh every hour, like the other configurations.
The configurations implicitly formulates a Metadata Query Protocol URL from the given baseURL.
For example, if the entityID is https://wiki.idem.garr.it/rp
, the provider will request the following resource:
https://mdx.idem.garr.it/idem/entities/https:%2F%2Fwiki.idem.garr.it%2Frp
Avvertimento
Change the baseURL according to the metadata stream you need:
https://mdx.idem.garr.it/idem-test/
for IDEM Test Federationhttps://mdx.idem.garr.it/idem/
for IDEM Federationhttps://mdx.idem.garr.it/edugain/
for IDEM + eduGAIN Interfederation
Shibboleth IdP v.3.1+¶
Configure the IdP to retrieve the Federation Metadata:¶
Retrieve the Federation Certificate used to verify signed metadata:
wget https://mdx.idem.garr.it/idem-mdx-service-crt.pem -O /opt/shibboleth-idp/credentials/idem-mdx-service-crt.pem
Check the validity:
openssl x509 -in /opt/shibboleth-idp/credentials/idem-mdx-service-crt.pem -fingerprint -sha1 -noout
must return:
(sha1: 46:FC:EB:7B:D0:67:46:EA:0C:B1:B2:61:4C:DC:37:DA:BD:B4:8A:95)
openssl x509 -in /opt/shibboleth-idp/credentials/idem-mdx-service-crt.pem -fingerprint -md5 -noout
must return:
(md5: 5D:19:CC:AA:1E:63:E9:50:9D:C7:BE:99:60:0F:1F:96)
Edit the SAML metadata source configuration file opportunely:
vim /opt/shibboleth-idp/conf/metadata-providers.xml
and add before the last
</MetadataProvider>
this piece of code, commenting/uncommenting the<MetadataQueryProtocol>
value depending on the Metadata flow used:<!-- MDX Service --> <MetadataProvider id="DynamicEntityMetadata" xsi:type="DynamicHTTPMetadataProvider" connectionRequestTimeout="PT2S" connectionTimeout="PT2S" socketTimeout="PT4S" refreshDelayFactor="0.025" maxCacheDuration="PT48H"> <!-- Verify the signature on the root element of the metadata using a trusted metadata signing certificate. --> <MetadataFilter xsi:type="SignatureValidation" requireSignedRoot="true" certificateFile="%{idp.home}/credentials/idem-mdx-service-crt.pem"/> <!-- Require a validUntil XML attribute on the root element and make sure its value is no more than 3 days into the future. --> <MetadataFilter xsi:type="RequiredValidUntil" maxValidityInterval="P3D"/> <!-- Base URL for MDQ --> <!-- IDEM Production Federation --> <!--<MetadataQueryProtocol>https://mdx.idem.garr.it/idem/</MetadataQueryProtocol>--> <!-- IDEM Test Federation --> <MetadataQueryProtocol>https://mdx.idem.garr.it/idem-test/</MetadataQueryProtocol> <!-- eduGAIN --> <!--<MetadataQueryProtocol>https://mdx.idem.garr.it/edugain/</MetadataQueryProtocol>--> </MetadataProvider>
Reload service with id shibboleth.MetadataResolverService to apply previous changes:
bash /opt/shibboleth-idp/bin/reload-service.sh -id shibboleth.MetadataResolverService
Shibboleth SP v3¶
Configure the SP to retrieve the Federation Metadata:¶
Retrieve the Federation Certificate used to verify signed metadata:
wget https://mdx.idem.garr.it/idem-mdx-service-crt.pem -O /etc/shibboleth/idem-mdx-service-crt.pem
Check the validity:
openssl x509 -in /etc/shibboleth/idem-mdx-service-crt.pem -fingerprint -sha1 -noout
must return:
(sha1: 46:FC:EB:7B:D0:67:46:EA:0C:B1:B2:61:4C:DC:37:DA:BD:B4:8A:95)
openssl x509 -in /etc/shibboleth/idem-mdx-service-crt.pem -fingerprint -md5 -noout
must return:
(md5: 5D:19:CC:AA:1E:63:E9:50:9D:C7:BE:99:60:0F:1F:96)
Edit the Shibboleth SP configuration file opportunely:
vim /etc/shibboleth/shibboleth2.xml
adding before the last
</MetadataProvider>
this piece of code:<!-- MDX Service --> <MetadataProvider type="MDQ" id="mdx" cacheDirectory="mdq-cache" baseUrl="https://mdx.idem.garr.it/idem-test/" maxCacheDuration="172800" refreshDelayFactor="0.025" ignoreTransport="true"> <MetadataFilter type="RequireValidUntil" maxValidityInterval="259200"/> <MetadataFilter type="Signature" certificate="idem-mdx-service-crt.pem"/> </MetadataProvider>
and changing the
baseUrl
value depending on the Metadata flow used:IDEM Test Federation:
https://mdx.idem.garr.it/idem-test/
IDEM Production Federation:
https://mdx.idem.garr.it/idem/
eduGAIN + IDEM Interfederation:
https://mdx.idem.garr.it/edugain/
Restart
shibd
daemon:sudo systemctl restart shibd
SimpleSAMLphp v.1.14+¶
Configure the Identity Provider to retrieve the Federation Metadata:¶
Edit the SimpleSAMLphp configuration file opportunely:
vim /var/simplesamlphp/config/config.php
by changing the configuration of
metadata.sources
to load the new MDQ metadata provider, commenting/uncommenting theserver
value depending on the Metadata flow used:'metadata.sources' => [ ['type' => 'flatfile'], ['type' => 'mdq', // IDEM Production Federation //'server' => 'https://mdx.idem.garr.it/idem', // IDEM Test Federation 'server' => 'https://mdx.idem.garr.it/idem-test', // eduGAIN //'server' => 'https://mdx.idem.garr.it/edugain', 'validateFingerprint' => '46:FC:EB:7B:D0:67:46:EA:0C:B1:B2:61:4C:DC:37:DA:BD:B4:8A:95', 'cachedir' => '/var/simplesamlphp/mdq-cache', 'cachelength' => 3600], ],
Nota
The
metarefresh
module needs to be disabled and thesaml20-idp-hosted.php
file needs to be the only one into/metadata
directory.Create the
mdq-cache
folder:sudo mkdir /var/simplesamlphp/mdq-cache
chown www-data /var/simplesamlphp/mdq-cache
Remove not needed files:
cd /var/simplesamlphp/metadata ; rm !(saml20-idp-hosted.php)
Configure the Service Provider to retrieve the Federation Metadata:¶
Edit the SimpleSAMLphp configuration file opportunely:
vim /var/simplesamlphp/config/config.php
by changing the configuration of
metadata.sources
to load the new MDQ metadata provider, commenting/uncommenting theserver
value depending on the Metadata flow used:'metadata.sources' => [ ['type' => 'mdq', // IDEM Production Federation //'server' => 'https://mdx.idem.garr.it/idem', // IDEM Test Federation 'server' => 'https://mdx.idem.garr.it/idem-test', // eduGAIN //'server' => 'https://mdx.idem.garr.it/edugain', 'validateFingerprint' => '46:FC:EB:7B:D0:67:46:EA:0C:B1:B2:61:4C:DC:37:DA:BD:B4:8A:95', 'cachedir' => '/var/simplesamlphp/mdq-cache', 'cachelength' => 3600], ['type' => 'flatfile'], ],
Avvertimento
The
metarefresh
module needs to be disabled and no file should be present in the/metadata
directory, unless it is thesaml20-idp-remote.php
file used by the embedded discovery service.Create the
mdq-cache
folder:sudo mkdir /var/simplesamlphp/mdq-cache
chown www-data /var/simplesamlphp/mdq-cache
Satosa¶
Configure the IdP/SP to retrieve the Federation Metadata:¶
Retrieve the Federation Certificate used to verify the signature of signed metadata:
wget https://mdx.idem.garr.it/idem-mdx-service-crt.pem -O /opt/satosa/etc/idem-mdx-service-crt.pem
Check the validity:
openssl x509 -in /opt/satosa/etc/idem-mdx-service-crt.pem -fingerprint -sha1 -noout
must return:
(sha1: 46:FC:EB:7B:D0:67:46:EA:0C:B1:B2:61:4C:DC:37:DA:BD:B4:8A:95)
openssl x509 -in /opt/satosa/etc/idem-mdx-service-crt.pem -fingerprint -md5 -noout
must return:
(md5: 5D:19:CC:AA:1E:63:E9:50:9D:C7:BE:99:60:0F:1F:96)
Depending on your configuration (backends/frontends), edit the configuration file:
Backends
vim /opt/satosa/etc/plugins/backends/saml2_backend.yaml
adding the following configuration for metadata:
metadata: mdq: - { url: "https://mdx.idem.garr.it/idem-test/", cert: idem-mdx-service-crt.pem, freshness_period: P0Y0M0DT1H0M0S }
and changing the
url
value depending on the Metadata flow used:IDEM Test Federation:
https://mdx.idem.garr.it/idem-test/
IDEM Production Federation:
https://mdx.idem.garr.it/idem/
eduGAIN + IDEM Interfederation:
https://mdx.idem.garr.it/edugain/
Frontends
vim /opt/satosa/etc/plugins/frontends/saml2_frontend.yaml
and add the following configuration for metadata:
metadata: mdq: - { url: "https://mdx.idem.garr.it/idem-test/", cert: idem-mdx-service-crt.pem, freshness_period: P0Y0M0DT1H0M0S }
and changing the
url
value depending on the Metadata flow used:IDEM Test Federation:
https://mdx.idem.garr.it/idem-test/
IDEM Production Federation:
https://mdx.idem.garr.it/idem/
eduGAIN + IDEM Interfederation:
https://mdx.idem.garr.it/edugain/
MDX Embedded Shibboleth Discovery Service Configuration¶
To be able to use Shibboleth Embedded Discovery Service with MDX, it is needed an «ad hoc» configuration as explained into the next section.
Shibboleth EDS¶
DOC: https://shibboleth.atlassian.net/wiki/spaces/EDS10/overview
Configure the EDS to use the Federation JSON file:¶
Create the folder to obtain the JSON file from the Jason Web Token:
sudo mkdir /opt/idem_jwt_to_json
Retrieve the public key needed to decode:
sudo wget "https://mdx.idem.garr.it/idem-mdx-service-pubkey.pem" -O /opt/idem_jwt_to_json/idem-mdx-service-pubkey.pem
and check the validity:
sudo openssl rsa -pubin -in /opt/idem_jwt_to_json/idem-mdx-service-pubkey.pem -pubout -outform DER | openssl sha1 -c
must return:
((stdin)= 30:75:93:37:d0:05:55:19:9f:76:e1:5a:73:db:45:7f:5e:66:11:4b)
sudo openssl rsa -pubin -in /opt/idem_jwt_to_json/idem-mdx-service-pubkey.pem -pubout -outform DER | openssl md5 -c
must return:
((stdin)= 84:5f:69:99:c5:f6:bb:e6:5f:ff:32:39:9a:a6:bb:85)
Install the needed Python modules:
Python >= 3.9:
sudo apt-get install python3-pip
sudo pip install pyjwt pyjwt[crypto] pem requests
Python >= 3.11:
sudo apt-get install python3-jwt python3-requests python3-pem
Retrieve the
decodeToken.py
file in the/opt/idem_jwt_to_json
folder:cd /opt/idem_jwt_to_json
sudo wget https://registry.idem.garr.it/idem-conf/shibboleth/IDP4/decodeToken.py
sudo chmod +x decodeToken.py
which contains the following code block:
#!/usr/bin/env python3 import jwt import json import pem import sys, getopt import requests def main(argv): try: opts, args = getopt.getopt(sys.argv[1:], 'j:o:k:hd', ['jwt=','output=','publickey=','help','debug' ]) except getopt.GetoptError as err: print (str(err)) print ("Usage: ./decodeToken.py -j <jwt_inputurl> -o <output_path> -k <publickey_path>") print ("The JSON file will be put in the output directory") sys.exit(2) inputurl = None outputpath = None publickey = None for opt, arg in opts: if opt in ('-h', '--help'): print ("Usage: ./decodeToken.py -j <jwt_inputurl> -o <output_path> -k <publickey_path>") print ("The JSON file will be put in the output directory") sys.exit() elif opt in ('-j', '--jwt'): inputurl = arg elif opt in ('-o', '--output'): outputpath = arg elif opt in ('-k', '--publickey'): publickey = arg elif opt == '-d': global _debug _debug = 1 else: print ("Usage: ./decodeToken.py -j <jwt_inputurl> -o <output_path> -k <publickey_path>") print ("The JSON file will be put in the output directory") sys.exit() if inputurl == None: print ("Token file is missing!\n") print ("Usage: ./decodeToken.py -j <jwt_inputurl> -o <output_path> -k <publickey_path>") sys.exit() if outputpath == None: print ("Output path is missing!\n") print ("Usage: ./decodeToken.py -j <jwt_inputurl> -o <output_path> -k <publickey_path>") sys.exit() if publickey == None: print ("Public Key path is missing!\n") print ("Usage: ./decodeToken.py -j <jwt_inputurl> -o <output_path> -k <publickey_path>") sys.exit() with open(publickey, 'r') as rsa_pubkey: pubkey = rsa_pubkey.read() jwt_token = requests.get(inputurl, allow_redirects=True) token = jwt_token.content decode = jwt.decode(token, pubkey, algorithms=["RS256"]) x = decode["data"] json_decoded = json.dumps(x, indent=4, ensure_ascii=False) result_path = open(outputpath, "w", encoding="utf-8") result_path.write(json_decoded) result_path.close() if __name__ == "__main__": main(sys.argv[1:])
Set up a cron job to regularly update the JSON file:
vim /etc/cron.d/eds-refresh
and insert one of the following line into the cron file created, according to the metadata stream and the
DocumentRoot
used (default:/var/www/html
):IDEM Production Federation:
*/30 * * * * /opt/idem_jwt_to_json/decodeToken.py -j https://mdx.idem.garr.it/idem-token -o /var/www/html/idem-eds.json -k /opt/idem_jwt_to_json/idem-mdx-service-pubkey.pem > /opt/idem_jwt_to_json/jwt_to_json.log 2>&1
eduGAIN + IDEM Interfederation:
*/30 * * * * /opt/idem_jwt_to_json/decodeToken.py -j https://mdx.idem.garr.it/edugain2idem-token -o /var/www/html/edugain2idem-eds.json -k /opt/idem_jwt_to_json/idem-mdx-service-pubkey.pem > /opt/idem_jwt_to_json/jwt_to_json.log 2>&1
versione senza loghi:
*/30 * * * * /opt/idem_jwt_to_json/decodeToken.py -j https://mdx.idem.garr.it/edugain2idem-token-nologo -o /var/www/html/edugain2idem-eds-nologo.json -k /opt/idem_jwt_to_json/idem-mdx-service-pubkey.pem > /opt/idem_jwt_to_json/jwt_to_json.log 2>&1
IDEM Test Federation:
*/30 * * * * /opt/idem_jwt_to_json/decodeToken.py -j https://mdx.idem.garr.it/idem-test-token -o /var/www/html/idem-test-eds.json -k /opt/idem_jwt_to_json/idem-mdx-service-pubkey.pem > /opt/idem_jwt_to_json/jwt_to_json.log 2>&1
Modify the EDS configuration:
sudo vim /etc/shibboleth-ds/idpselect_config.js
by changing the value of
this.dataSource
with a value chosen between:IDEM Test Federation:
/idem-test-eds.json
IDEM Production Federation:
/idem-eds.json
eduGAIN + IDEM Interfederation:
/edugain2idem-eds.json
Avvertimento
The
this.dataSource
value is the URL of the source of the data feed of IdPs to the Discovery Service. This feed MUST be at the same location as the DS itself and so it is usual for the protocol and host part of the URL (https://example.org
) to be dropped. The data source is a JSON file. The schema of this file is defined in EDSDetails.
SimpleSAMLphp¶
Configure the creation of EDS’s files:¶
Create the folder to obtain the JSON file from the Jason Web Token:
sudo mkdir /opt/idem_jwt_to_php
Retrieve the public key needed to decode:
sudo wget "https://mdx.idem.garr.it/idem-mdx-service-pubkey.pem" -O /opt/idem_jwt_to_php/idem-mdx-service-pubkey.pem
and check the validity:
sudo openssl rsa -pubin -in /opt/idem_jwt_to_php/idem-mdx-service-pubkey.pem -pubout -outform DER | openssl sha1 -c
must return:
((stdin)= 30:75:93:37:d0:05:55:19:9f:76:e1:5a:73:db:45:7f:5e:66:11:4b)
sudo openssl rsa -pubin -in /opt/idem_jwt_to_php/idem-mdx-service-pubkey.pem -pubout -outform DER | openssl md5 -c
must return:
((stdin)= 84:5f:69:99:c5:f6:bb:e6:5f:ff:32:39:9a:a6:bb:85)
Install the needed Python modules:
Python >= 3.9:
sudo apt-get install python3-pip
sudo pip install pyjwt pyjwt[crypto] pem requests
Python >= 3.11:
sudo apt-get install python3-jwt python3-requests python3-pem
Retrieve the
decodeToken.py
file in the/opt/idem_jwt_to_php folder
:cd /opt/idem_jwt_to_php
sudo wget https://registry.idem.garr.it/idem-conf/shibboleth/IDP4/decodeToken.py
sudo chmod +x decodeToken.py
which contains the following code block:
#!/usr/bin/env python3 import jwt import json import pem import sys, getopt import requests def main(argv): try: opts, args = getopt.getopt(sys.argv[1:], 'j:o:k:hd', ['jwt=','output=','publickey=','help','debug' ]) except getopt.GetoptError as err: print (str(err)) print ("Usage: ./decodeToken.py -j <jwt_inputurl> -o <output_path> -k <publickey_path>") print ("The JSON file will be put in the output directory") sys.exit(2) inputurl = None outputpath = None publickey = None for opt, arg in opts: if opt in ('-h', '--help'): print ("Usage: ./decodeToken.py -j <jwt_inputurl> -o <output_path> -k <publickey_path>") print ("The JSON file will be put in the output directory") sys.exit() elif opt in ('-j', '--jwt'): inputurl = arg elif opt in ('-o', '--output'): outputpath = arg elif opt in ('-k', '--publickey'): publickey = arg elif opt == '-d': global _debug _debug = 1 else: print ("Usage: ./decodeToken.py -j <jwt_inputurl> -o <output_path> -k <publickey_path>") print ("The JSON file will be put in the output directory") sys.exit() if inputurl == None: print ("Token file is missing!\n") print ("Usage: ./decodeToken.py -j <jwt_inputurl> -o <output_path> -k <publickey_path>") sys.exit() if outputpath == None: print ("Output path is missing!\n") print ("Usage: ./decodeToken.py -j <jwt_inputurl> -o <output_path> -k <publickey_path>") sys.exit() if publickey == None: print ("Public Key path is missing!\n") print ("Usage: ./decodeToken.py -j <jwt_inputurl> -o <output_path> -k <publickey_path>") sys.exit() with open(publickey, 'r') as rsa_pubkey: pubkey = rsa_pubkey.read() jwt_token = requests.get(inputurl, allow_redirects=True) token = jwt_token.content decode = jwt.decode(token, pubkey, algorithms=["RS256"]) x = decode["data"] json_decoded = json.dumps(x, indent=4, ensure_ascii=False) result_path = open(outputpath, "w", encoding="utf-8") result_path.write(json_decoded) result_path.close() if __name__ == "__main__": main(sys.argv[1:])
Retrieve the script to convert JSON file to PHP:
cd /opt/idem_jwt_to_php
sudo wget https://registry.idem.garr.it/idem-conf/simplesamlphp/json_to_php_converter.php
sudo chmod +x json_to_php_converter.php
which contains the following code block:
<?php $jsondata = file_get_contents('/opt/idem_jwt_to_php/edugain2idem-eds.json'); if (!empty($jsondata)) { $entities = json_decode($jsondata, true); $text = ''; $displaynames = ''; foreach ($entities as $entity => $entityMetadata) { // remove the unused elements of json unset($entityMetadata['Logos']); $entityMetadata['entityid'] = $entityMetadata['entityID']; unset($entityMetadata['entityID']); $displaynames = $entityMetadata['DisplayNames']; unset($entityMetadata['DisplayNames']); if (!empty($displaynames)) { $entityMetadata['name'] = array(); foreach ($displaynames as $displayname => $entityName) { $entityMetadata['name'][$entityName['lang']] = $entityName['value']; } } $text .= '$metadata['.var_export($entityMetadata['entityid'], true).'] ='. var_export($entityMetadata, true).";\n"; $entities = $text; } } $file = '/var/simplesamlphp/metadata/saml20-idp-remote.php'; file_put_contents($file,"<?php\n"); file_put_contents($file, print_r($entities, true),FILE_APPEND); ?>
and change the
file_get_contents
value according to your configuration:IDEM Test Federation:
/opt/idem_jwt_to_php/idem-test-eds.json
IDEM Production Federation:
/opt/idem_jwt_to_php/idem-eds.json
eduGAIN + IDEM Interfederation:
/opt/idem_jwt_to_php/edugain2idem-eds.json
Create a cronjob to refresh files:
vim /etc/cron.d/eds-refresh
and add the following jobs, according to the metadata flow followed:
IDEM Production Federation:
*/30 * * * * root /opt/idem_jwt_to_php/decodeToken.py -j https://mdx.idem.garr.it/idem-token -o /opt/idem_jwt_to_php/idem-eds.json -k /opt/idem_jwt_to_php/idem-mdx-service-pubkey.pem > /opt/idem_jwt_to_php/jwt_to_php.log 2>&1 */30 * * * * root /usr/bin/php /opt/idem_jwt_to_php/json_to_php_converter.php
eduGAIN + IDEM Interfederation:
*/30 * * * * root /opt/idem_jwt_to_php/decodeToken.py -j https://mdx.idem.garr.it/edugain2idem-token -o /opt/idem_jwt_to_php/edugain2idem-eds.json -k /opt/idem_jwt_to_php/idem-mdx-service-pubkey.pem > /opt/idem_jwt_to_php/jwt_to_php.log 2>&1 */30 * * * * root /usr/bin/php /opt/idem_jwt_to_php/json_to_php_converter.php
IDEM Test Federation:
*/30 * * * * root /opt/idem_jwt_to_php/decodeToken.py -j https://mdx.idem.garr.it/idem-test-token -o /opt/idem_jwt_to_php/idem-test-eds.json -k /opt/idem_jwt_to_php/idem-mdx-service-pubkey.pem > /opt/idem_jwt_to_php/jwt_to_php.log 2>&1 */30 * * * * root /usr/bin/php /opt/idem_jwt_to_php/json_to_php_converter.php
IDEM Metadata Aggregates¶
If you want to see the aggregated metadata file, you can download it with:
IDEM Federation¶
Metadata aggregate for the IDEM Federation: https://mdx.idem.garr.it/idem/entities
eduGAIN (+ IDEM) Interfederation¶
Metadata aggregate for the eduGAIN (+ IDEM) Interfederation: https://mdx.idem.garr.it/edugain/entities
IDEM Test Federation¶
Metadata aggregate for the IDEM Test Federation: https://mdx.idem.garr.it/idem-test/entities