From 9021452de0a61c73f4f6e9b466ee752932c2f6b5 Mon Sep 17 00:00:00 2001 From: Showmic Islam <57932760+showmic09@users.noreply.github.com> Date: Wed, 15 May 2024 16:21:11 -0500 Subject: [PATCH 01/32] Create UNL_Turner.yaml --- projects/UNL_Turner.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 projects/UNL_Turner.yaml diff --git a/projects/UNL_Turner.yaml b/projects/UNL_Turner.yaml new file mode 100644 index 000000000..946829fd9 --- /dev/null +++ b/projects/UNL_Turner.yaml @@ -0,0 +1,8 @@ +Department: Mechanical and Materials Engineering +Description: 'Working with ultrasonic wave propagation in polycrystalline materials. + Website: https://engineering.unl.edu/quisp/' +FieldOfScience: Engineering +FieldOfScienceID: '14.1901' +InstitutionID: Unknown +Organization: University of Nebraska - Lincoln +PIName: Joseph Turner From 8eb0525c742771797734c2dc71986fde8cfac913 Mon Sep 17 00:00:00 2001 From: Showmic Islam <57932760+showmic09@users.noreply.github.com> Date: Wed, 15 May 2024 16:28:09 -0500 Subject: [PATCH 02/32] Create UMiss_Bennett.yaml --- projects/UMiss_Bennett.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 projects/UMiss_Bennett.yaml diff --git a/projects/UMiss_Bennett.yaml b/projects/UMiss_Bennett.yaml new file mode 100644 index 000000000..be75126dc --- /dev/null +++ b/projects/UMiss_Bennett.yaml @@ -0,0 +1,8 @@ +Department: Physics and Astronomy +Description: 'In the decay of Xi_c baryon to lambda kaon pion, I am performing a likelihood + fitting to extract the coupling to each potential resonance. ' +FieldOfScience: Physics and astronomy +FieldOfScienceID: '40.0804' +InstitutionID: https://osg-htc.org/iid/bigpgrrmxblz +Organization: University of Mississippi +PIName: Jake Bennett From 5079fcc704830f34d96810551e9a066a76e3852e Mon Sep 17 00:00:00 2001 From: Fabio Andrijauskas Date: Wed, 15 May 2024 15:26:11 -0700 Subject: [PATCH 03/32] changing to lets encrypt changing to lets encrypt --- .../Nebraska-Lincoln/UNLCachingInfrastructure.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topology/University of Nebraska/Nebraska-Lincoln/UNLCachingInfrastructure.yaml b/topology/University of Nebraska/Nebraska-Lincoln/UNLCachingInfrastructure.yaml index 4daa9c33b..b40074f1c 100644 --- a/topology/University of Nebraska/Nebraska-Lincoln/UNLCachingInfrastructure.yaml +++ b/topology/University of Nebraska/Nebraska-Lincoln/UNLCachingInfrastructure.yaml @@ -122,7 +122,7 @@ Resources: ID: e86d60dc841731aebfd879d5883d66ebab20cbd2 FQDN: unl-cache.nationalresearchplatform.org - DN: /DC=org/DC=incommon/C=US/ST=California/O=University of California, San Diego/CN=unl-cache.nationalresearchplatform.org + DN: /CN=unl-cache.nationalresearchplatform.org Services: XRootD cache server: Description: Nebraska NRP OSDF Cache From a28ea27bd332391c70c4c237bbe0da483dbe1682 Mon Sep 17 00:00:00 2001 From: glushkov Date: Mon, 20 May 2024 20:18:59 +0200 Subject: [PATCH 04/32] BNL: Retiring gridgk07,8.racf.bnl.gov and Introducing gridgk07,8.sdcc.bnl.gov @bbockelm , @brianhlin - thanks for the support in https://github.com/opensciencegrid/topology/pull/3865 Hopefully now everything is correct --- .../BNL ATLAS Tier1/BNL-ATLAS.yaml | 87 ++++++++++++++++++- 1 file changed, 84 insertions(+), 3 deletions(-) diff --git a/topology/Brookhaven National Laboratory/BNL ATLAS Tier1/BNL-ATLAS.yaml b/topology/Brookhaven National Laboratory/BNL ATLAS Tier1/BNL-ATLAS.yaml index f9c061f92..adaa19aec 100644 --- a/topology/Brookhaven National Laboratory/BNL ATLAS Tier1/BNL-ATLAS.yaml +++ b/topology/Brookhaven National Laboratory/BNL ATLAS Tier1/BNL-ATLAS.yaml @@ -201,7 +201,7 @@ Resources: KSI2KMin: 0 StorageCapacityMax: 0 StorageCapacityMin: 0 - BNL_ATLAS_7: + BNL_ATLAS_07: Active: true ContactLists: Administrative Contact: @@ -219,7 +219,7 @@ Resources: Name: Robert Hancock ID: OSG1000212 Description: An OSG CE resource at BNL, mainly serving USATLAS production jobs, - not open to general OSG users. + not open to general OSG users. EL9 FQDN: gridgk07.sdcc.bnl.gov ID: 438 Services: @@ -242,7 +242,48 @@ Resources: StorageCapacityMax: 0 StorageCapacityMin: 0 TapeCapacity: 0 - BNL_ATLAS_8: + BNL_ATLAS_7: + Active: false + ContactLists: + Administrative Contact: + Primary: + ID: 1e47726bf41f2128059de31cf7bc9c8d5dcadad6 + Name: BNL Atlas Tier 1 support + Secondary: + ID: OSG1000638 + Name: Ivan Glushkov + Security Contact: + Primary: + Name: BNL Atlas Tier 1 support + ID: 1e47726bf41f2128059de31cf7bc9c8d5dcadad6 + Secondary: + Name: Robert Hancock + ID: OSG1000212 + Description: An OSG CE resource at BNL, mainly serving USATLAS production jobs, + not open to general OSG users. + FQDN: gridgk07.racf.bnl.gov + ID: 438 + Services: + CE: + Description: Compute Element + Details: + hidden: false + sam_uri: htcondor://gridgk07.racf.bnl.gov + VOOwnership: + ATLAS: 100 + WLCGInformation: + APELNormalFactor: 12.69 + AccountingName: BNL_ATLAS_1 + HEPSPEC: 0 + InteropAccounting: true + InteropBDII: true + InteropMonitoring: true + KSI2KMax: 0 + KSI2KMin: 0 + StorageCapacityMax: 0 + StorageCapacityMin: 0 + TapeCapacity: 0 + BNL_ATLAS_08: Active: true ContactLists: Administrative Contact: @@ -282,6 +323,46 @@ Resources: KSI2KMin: 0 StorageCapacityMax: 0 StorageCapacityMin: 0 + BNL_ATLAS_8: + Active: false + ContactLists: + Administrative Contact: + Primary: + ID: 1e47726bf41f2128059de31cf7bc9c8d5dcadad6 + Name: BNL Atlas Tier 1 support + Secondary: + ID: OSG1000638 + Name: Ivan Glushkov + Security Contact: + Primary: + Name: BNL Atlas Tier 1 support + ID: 1e47726bf41f2128059de31cf7bc9c8d5dcadad6 + Secondary: + Name: Robert Hancock + ID: OSG1000212 + Description: This CE will serve grid jobs submitted to the *local* USATLAS resources, + e.g. for panda queue ANALY_BNL_LOCAL, at BNL. + FQDN: gridgk08.racf.bnl.gov + ID: 545 + Services: + CE: + Description: Compute Element + Details: + hidden: false + sam_uri: htcondor://gridgk08.racf.bnl.gov + VOOwnership: + ATLAS: 100 + WLCGInformation: + APELNormalFactor: 12.69 + AccountingName: BNL_ATLAS_1 + HEPSPEC: 0 + InteropAccounting: false + InteropBDII: false + InteropMonitoring: false + KSI2KMax: 0 + KSI2KMin: 0 + StorageCapacityMax: 0 + StorageCapacityMin: 0 BNL_ATLAS_Frontier_Squid: Active: true ContactLists: From 380adfe096c48c129853482e35ecffec36261dc4 Mon Sep 17 00:00:00 2001 From: glushkov Date: Mon, 20 May 2024 20:25:40 +0200 Subject: [PATCH 05/32] BNL: Removing IDs for gridk07,8.sdcc --- .../BNL ATLAS Tier1/BNL-ATLAS.yaml | 2 -- 1 file changed, 2 deletions(-) diff --git a/topology/Brookhaven National Laboratory/BNL ATLAS Tier1/BNL-ATLAS.yaml b/topology/Brookhaven National Laboratory/BNL ATLAS Tier1/BNL-ATLAS.yaml index adaa19aec..9221d00c3 100644 --- a/topology/Brookhaven National Laboratory/BNL ATLAS Tier1/BNL-ATLAS.yaml +++ b/topology/Brookhaven National Laboratory/BNL ATLAS Tier1/BNL-ATLAS.yaml @@ -221,7 +221,6 @@ Resources: Description: An OSG CE resource at BNL, mainly serving USATLAS production jobs, not open to general OSG users. EL9 FQDN: gridgk07.sdcc.bnl.gov - ID: 438 Services: CE: Description: Compute Element @@ -303,7 +302,6 @@ Resources: Description: This CE will serve grid jobs submitted to the *local* USATLAS resources, e.g. for panda queue ANALY_BNL_LOCAL, at BNL. FQDN: gridgk08.sdcc.bnl.gov - ID: 545 Services: CE: Description: Compute Element From 0747c54c0e35f23fcececfe1bbedc37239836b00 Mon Sep 17 00:00:00 2001 From: Fabio Andrijauskas Date: Mon, 20 May 2024 14:13:33 -0700 Subject: [PATCH 06/32] Add downtime for ComputeCanada-Cedar-Cache due to overloaded Add downtime for ComputeCanada-Cedar-Cache due to overloaded --- .../ComputeCanada-Cedar_downtime.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/topology/Compute Canada/ComputeCanada - Cedar/ComputeCanada-Cedar_downtime.yaml b/topology/Compute Canada/ComputeCanada - Cedar/ComputeCanada-Cedar_downtime.yaml index 8e53972ba..12c7e8588 100644 --- a/topology/Compute Canada/ComputeCanada - Cedar/ComputeCanada-Cedar_downtime.yaml +++ b/topology/Compute Canada/ComputeCanada - Cedar/ComputeCanada-Cedar_downtime.yaml @@ -31,3 +31,14 @@ Services: - XRootD cache server # --------------------------------------------------------- +- Class: UNSCHEDULED + ID: 1812395767 + Description: overloaded + Severity: Outage + StartTime: May 19, 2024 08:01 +0000 + EndTime: May 27, 2024 19:30 +0000 + CreatedTime: May 20, 2024 21:12 +0000 + ResourceName: ComputeCanada-Cedar-Cache + Services: + - XRootD cache server +# --------------------------------------------------------- From 2517b2ff02dd15802cee6cd6ef49f6a8213a0e66 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Mon, 20 May 2024 17:06:08 -0500 Subject: [PATCH 07/32] Update projects/UNL_Turner.yaml --- projects/UNL_Turner.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/projects/UNL_Turner.yaml b/projects/UNL_Turner.yaml index 946829fd9..8e2311cf7 100644 --- a/projects/UNL_Turner.yaml +++ b/projects/UNL_Turner.yaml @@ -3,6 +3,6 @@ Description: 'Working with ultrasonic wave propagation in polycrystalline materi Website: https://engineering.unl.edu/quisp/' FieldOfScience: Engineering FieldOfScienceID: '14.1901' -InstitutionID: Unknown +InstitutionID: 'https://osg-htc.org/iid/q9k1b8dfrw25' Organization: University of Nebraska - Lincoln PIName: Joseph Turner From 1e6c74cf1c064a7526fcde9edeab4c32cd674bc4 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 21 May 2024 08:24:33 +0000 Subject: [PATCH 08/32] --- updated-dependencies: - dependency-name: requests dependency-type: direct:production ... Signed-off-by: dependabot[bot] --- .github/scripts/check_project_fos_precision/requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/scripts/check_project_fos_precision/requirements.txt b/.github/scripts/check_project_fos_precision/requirements.txt index 1e8b52de3..96082b7ff 100644 --- a/.github/scripts/check_project_fos_precision/requirements.txt +++ b/.github/scripts/check_project_fos_precision/requirements.txt @@ -6,7 +6,7 @@ pandas==2.2.2 python-dateutil==2.9.0.post0 pytz==2024.1 PyYAML==6.0.1 -requests==2.31.0 +requests==2.32.0 six==1.16.0 tzdata==2024.1 urllib3==2.2.1 From 3dc1bde38fbd1abb7f77ad2b665aaf03c50ed01c Mon Sep 17 00:00:00 2001 From: Jadir Marra da Silva Date: Tue, 21 May 2024 08:21:56 -0300 Subject: [PATCH 09/32] Update SPRACE_downtime.yaml --- .../SPRACE/SPRACE_downtime.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/topology/Universidade Estadual Paulista/SPRACE/SPRACE_downtime.yaml b/topology/Universidade Estadual Paulista/SPRACE/SPRACE_downtime.yaml index eff8c25ae..d68162d40 100644 --- a/topology/Universidade Estadual Paulista/SPRACE/SPRACE_downtime.yaml +++ b/topology/Universidade Estadual Paulista/SPRACE/SPRACE_downtime.yaml @@ -3412,3 +3412,14 @@ Services: - Squid # --------------------------------------------------------- +- Class: SCHEDULED + ID: 1812904442 + Description: Host update to Alma 9 and OSG 23 + Severity: Outage + StartTime: May 23, 2024 11:00 +0000 + EndTime: May 23, 2024 21:00 +0000 + CreatedTime: May 21, 2024 11:20 +0000 + ResourceName: SPRACE + Services: + - CE +# --------------------------------------------------------- From bc6934de0fa64b6f95ebf942f5a85871475e41bb Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Tue, 21 May 2024 14:08:28 -0500 Subject: [PATCH 10/32] Rename some project files to match their project names --- projects/{GSU_Maimon.yaml => GSU_Wang.yaml} | 0 projects/{TNTTech_ITS.yaml => TNTech_ITS.yaml} | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename projects/{GSU_Maimon.yaml => GSU_Wang.yaml} (100%) rename projects/{TNTTech_ITS.yaml => TNTech_ITS.yaml} (100%) diff --git a/projects/GSU_Maimon.yaml b/projects/GSU_Wang.yaml similarity index 100% rename from projects/GSU_Maimon.yaml rename to projects/GSU_Wang.yaml diff --git a/projects/TNTTech_ITS.yaml b/projects/TNTech_ITS.yaml similarity index 100% rename from projects/TNTTech_ITS.yaml rename to projects/TNTech_ITS.yaml From 6d9fcfb17abe7537ef9db5777ee0e4264f6a2096 Mon Sep 17 00:00:00 2001 From: Fabio Andrijauskas Date: Tue, 21 May 2024 12:53:21 -0700 Subject: [PATCH 11/32] Setting FQDN back - I2 issuers is working Setting FQDN back - I2 issuers is working --- .../Internet2/Internet2Chicago/I2ChicagoInfrastructure.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topology/Internet2/Internet2Chicago/I2ChicagoInfrastructure.yaml b/topology/Internet2/Internet2Chicago/I2ChicagoInfrastructure.yaml index 56b02345f..456cf92dc 100644 --- a/topology/Internet2/Internet2Chicago/I2ChicagoInfrastructure.yaml +++ b/topology/Internet2/Internet2Chicago/I2ChicagoInfrastructure.yaml @@ -18,7 +18,7 @@ Resources: Name: Fabio Andrijauskas ID: OSG1000162 FQDN: osg-chicago-stashcache.nrp.internet2.edu - DN: /CN=osg-chicago-stashcache.nrp.internet2.edu + DN: /DC=org/DC=incommon/C=US/ST=Michigan/O=University Corporation For Advanced Internet Development/CN=osg-chicago-stashcache.nrp.internet2.edu Services: XRootD cache server: Description: Internet2 Chicago Cache From a6ceecc5bfb0404733629998075751680588f7f3 Mon Sep 17 00:00:00 2001 From: edubach <124012043+edubach@users.noreply.github.com> Date: Wed, 22 May 2024 13:49:38 +0100 Subject: [PATCH 12/32] Update NET2.yaml Adding a varnish proxy for NET2 at NRP. --- .../NET2/NET2.yaml | 33 +++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-) diff --git a/topology/University of Massachusetts - Amherst/NET2/NET2.yaml b/topology/University of Massachusetts - Amherst/NET2/NET2.yaml index 12b32e69b..635cd3395 100644 --- a/topology/University of Massachusetts - Amherst/NET2/NET2.yaml +++ b/topology/University of Massachusetts - Amherst/NET2/NET2.yaml @@ -153,14 +153,43 @@ Resources: ID: OSG1000284 Name: Eduardo Bach Description: - This is the NET2 squid + This is a NET2 squid proxy. FQDN: squid.compute.net2.mghpcc.org ID: 1480 Services: Squid: - Description: Generic squid service + Description: This is the Frontier distribution of Squid. Details: uri_override: squid.compute.net2.mghpcc.org:3128 Monitored: true VOOwnership: ATLAS: 100 + NET2_NRP_VARNISH_FRONTIER: + Active: true + ContactLists: + Administrative Contact: + Primary: + ID: OSG1000443 + Name: William Leight + Secondary: + ID: OSG1000284 + Name: Eduardo Bach + Security Contact: + Primary: + ID: OSG1000443 + Name: William Leight + Secondary: + ID: OSG1000284 + Name: Eduardo Bach + Description: + This is a NET2 Varnish proxy at NRP. + FQDN: storage-01.nrp.mghpcc.org + ID: + Services: + Squid: + Description: This is a Varnish proxy flavour. + Details: + uri_override: storage-01.nrp.mghpcc.org:6082 + Monitored: true + VOOwnership: + ATLAS: 100 From 4e86b35c1965a61fe91dbf18539d367011f88ae5 Mon Sep 17 00:00:00 2001 From: Brian Lin Date: Wed, 22 May 2024 09:21:43 -0500 Subject: [PATCH 13/32] Remove superfluous ID --- topology/University of Massachusetts - Amherst/NET2/NET2.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/topology/University of Massachusetts - Amherst/NET2/NET2.yaml b/topology/University of Massachusetts - Amherst/NET2/NET2.yaml index 635cd3395..01ddabbf0 100644 --- a/topology/University of Massachusetts - Amherst/NET2/NET2.yaml +++ b/topology/University of Massachusetts - Amherst/NET2/NET2.yaml @@ -184,7 +184,6 @@ Resources: Description: This is a NET2 Varnish proxy at NRP. FQDN: storage-01.nrp.mghpcc.org - ID: Services: Squid: Description: This is a Varnish proxy flavour. From e15dc21cf7127319c56647ed0db6ea5194d08e48 Mon Sep 17 00:00:00 2001 From: Eric Appelt Date: Wed, 22 May 2024 19:40:26 -0500 Subject: [PATCH 14/32] Add T2_US_Vanderbilt May 2024 Downtime --- .../Vanderbilt ACCRE/Vanderbilt_downtime.yaml | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/topology/Vanderbilt University/Vanderbilt ACCRE/Vanderbilt_downtime.yaml b/topology/Vanderbilt University/Vanderbilt ACCRE/Vanderbilt_downtime.yaml index 854709c0f..148f700fd 100644 --- a/topology/Vanderbilt University/Vanderbilt ACCRE/Vanderbilt_downtime.yaml +++ b/topology/Vanderbilt University/Vanderbilt ACCRE/Vanderbilt_downtime.yaml @@ -655,3 +655,34 @@ ResourceName: Vanderbilt_Xrootd Services: - XRootD component +# ---------------------------------------------------------- +- Class: SCHEDULED + ID: 1812904443 + Description: Network upgrades + Severity: Outage + StartTime: May 28, 2024 05:00 +0000 + EndTime: May 31, 2024 05:00 +0000 + CreatedTime: May 23, 2024 02:00 +0000 + ResourceName: Vanderbilt_CE5 + Services: + - CE +- Class: SCHEDULED + ID: 1812904444 + Description: Network upgrades + Severity: Outage + StartTime: May 28, 2024 05:00 +0000 + EndTime: May 31, 2024 05:00 +0000 + CreatedTime: May 23, 2024 02:00 +0000 + ResourceName: Vanderbilt_CE6 + Services: + - CE +- Class: SCHEDULED + ID: 1812904445 + Description: Network upgrades + Severity: Outage + StartTime: May 28, 2024 05:00 +0000 + EndTime: May 31, 2024 05:00 +0000 + CreatedTime: May 23, 2024 02:00 +0000 + ResourceName: Vanderbilt_Xrootd + Services: + - XRootD component From 8cee80f8d50b9f454b5b1a0c816315daac2d6af4 Mon Sep 17 00:00:00 2001 From: Tim Cartwright Date: Thu, 23 May 2024 11:01:06 -0500 Subject: [PATCH 15/32] Added MI-HORUS local contacts --- .../Michigan HORUS/MI-HORUS.yaml | 55 ++++++++----------- 1 file changed, 23 insertions(+), 32 deletions(-) diff --git a/topology/University of Michigan/Michigan HORUS/MI-HORUS.yaml b/topology/University of Michigan/Michigan HORUS/MI-HORUS.yaml index 148637d81..ef5d9d311 100644 --- a/topology/University of Michigan/Michigan HORUS/MI-HORUS.yaml +++ b/topology/University of Michigan/Michigan HORUS/MI-HORUS.yaml @@ -27,48 +27,39 @@ Resources: # If you cannot find the contact above XML, please register the contact: # https://opensciencegrid.org/docs/common/registration/#registering-contacts ContactLists: - # Administrative Contacts are persons or groups of people (i.e., - # mailing lists) that are directly responsible for the - # maintenance of the resource Administrative Contact: Primary: Name: Jeffrey Michael Dost ID: 3a8eb6436a8b78ca50f7e93bb2a4d1f0141212ba - # Secondary: - # Name: - # ID: - # Tertiary: - # Name: - # ID: - # Security Contact are persons or groups of people (i.e., - # mailing lists) that are responsible for handling security - # issues related to the resource Security Contact: Primary: Name: Jeffrey Michael Dost ID: 3a8eb6436a8b78ca50f7e93bb2a4d1f0141212ba - # Secondary: - # Name: - # ID: - # Tertiary: - # Name: - # ID: - # Site contact (optional) are persons or groups of people (i.e., - # mailing lists) that are generally responsible for a site's - # relationship with the OSG (e.g., principal investigators, - # local administrator contact for OSG Hosted CEs) - # Site Contact: - # Primary: - # Name: - # ID: - # Secondary: - # Name: - # ID: - # Tertiary: - # Name: - # ID: + Local Operational Contact: + Primary: + Name: Muhammad Akhdhor + ID: OSG1000654 + Secondary: + Name: Michael Thompson + ID: f84214412f7b3dbf848808c4ba30d179c5743766 + Tertiary: + Name: Andrew Keen + ID: OSG1000660 + + Local Security Contact: + Primary: + Name: HORUS Security Mailing List + ID: d7ec710e472267ffcc69de4070491f48582ccc2f + + Local Executive Contact: + Primary: + Name: Shawn McKee + ID: a833f94ad3840fe779702409c6b778d79aaed76f + Secondary: + Name: Robert Eugene Stovall + ID: OSG1000685 # FQDN is the fully qualified domain name of the host running this resource FQDN: mi-horus-ce1.svc.opensciencegrid.org From a54acd2923b7452c53c140f74134fd590242dc5b Mon Sep 17 00:00:00 2001 From: Fabio Andrijauskas Date: Wed, 22 May 2024 18:21:18 -0700 Subject: [PATCH 16/32] Adding ANY to ESnet cache Adding ANY to ESnet cache --- topology/Energy Sciences Network/Amsterdam/EsnetAmsterdam.yaml | 2 +- topology/Energy Sciences Network/London/ESnetLondon.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/topology/Energy Sciences Network/Amsterdam/EsnetAmsterdam.yaml b/topology/Energy Sciences Network/Amsterdam/EsnetAmsterdam.yaml index b81bda81b..f2ed868fd 100644 --- a/topology/Energy Sciences Network/Amsterdam/EsnetAmsterdam.yaml +++ b/topology/Energy Sciences Network/Amsterdam/EsnetAmsterdam.yaml @@ -22,4 +22,4 @@ Resources: XRootD cache server: Description: ESnet Amsterdam Cache AllowedVOs: - - LIGO + - ANY \ No newline at end of file diff --git a/topology/Energy Sciences Network/London/ESnetLondon.yaml b/topology/Energy Sciences Network/London/ESnetLondon.yaml index 5fe582291..934ded268 100644 --- a/topology/Energy Sciences Network/London/ESnetLondon.yaml +++ b/topology/Energy Sciences Network/London/ESnetLondon.yaml @@ -22,4 +22,4 @@ Resources: XRootD cache server: Description: Internet2 London Cache AllowedVOs: - - LIGO + - ANY From 2bb7448e1d97a494100f86daab86728b8ef2a362 Mon Sep 17 00:00:00 2001 From: Tim Cartwright Date: Fri, 24 May 2024 10:06:10 -0500 Subject: [PATCH 17/32] Added remaining HORUS contact; FD #74888 --- topology/University of Michigan/Michigan HORUS/MI-HORUS.yaml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/topology/University of Michigan/Michigan HORUS/MI-HORUS.yaml b/topology/University of Michigan/Michigan HORUS/MI-HORUS.yaml index ef5d9d311..5ed2ade22 100644 --- a/topology/University of Michigan/Michigan HORUS/MI-HORUS.yaml +++ b/topology/University of Michigan/Michigan HORUS/MI-HORUS.yaml @@ -60,6 +60,9 @@ Resources: Secondary: Name: Robert Eugene Stovall ID: OSG1000685 + Tertiary: + Name: Robert Michael Thompson + ID: OSG1000697 # FQDN is the fully qualified domain name of the host running this resource FQDN: mi-horus-ce1.svc.opensciencegrid.org From 646eaff2e560cb9641e4db1b588352b9efa04de8 Mon Sep 17 00:00:00 2001 From: Christina K Date: Fri, 24 May 2024 13:03:56 -0500 Subject: [PATCH 18/32] Create UWMadison_OConnor.yaml --- projects/UWMadison_OConnor.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 projects/UWMadison_OConnor.yaml diff --git a/projects/UWMadison_OConnor.yaml b/projects/UWMadison_OConnor.yaml new file mode 100644 index 000000000..36a01870e --- /dev/null +++ b/projects/UWMadison_OConnor.yaml @@ -0,0 +1,8 @@ +Department: Pathology and Laboratory Medicine +Description: "The overarching goal of the O\u2019Connor lab is to contribute meaningfully\ + \ to the global response to viral infections impacting human health." +FieldOfScience: Biological and Biomedical Sciences +FieldOfScienceID: 26.0999b +InstitutionID: Unknown +Organization: University of Wisconsin-Madison +PIName: David O'Connor From 35cfcd93efe275ee6d32987cca52d3294f7134bb Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 24 May 2024 13:19:44 -0500 Subject: [PATCH 19/32] Add InstitutionID --- projects/UWMadison_OConnor.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/projects/UWMadison_OConnor.yaml b/projects/UWMadison_OConnor.yaml index 36a01870e..ec1ccba51 100644 --- a/projects/UWMadison_OConnor.yaml +++ b/projects/UWMadison_OConnor.yaml @@ -3,6 +3,6 @@ Description: "The overarching goal of the O\u2019Connor lab is to contribute mea \ to the global response to viral infections impacting human health." FieldOfScience: Biological and Biomedical Sciences FieldOfScienceID: 26.0999b -InstitutionID: Unknown +InstitutionID: 'https://osg-htc.org/iid/fq8thqsj99zh' Organization: University of Wisconsin-Madison PIName: David O'Connor From 1f3b8792c9cf376d0ce966ef19eeb085b4ebad2d Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 24 May 2024 20:46:38 -0500 Subject: [PATCH 20/32] Cache results of DN -> DN hash conversion --- src/webapp/x509.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/webapp/x509.py b/src/webapp/x509.py index 5724cb60f..ba6d939dc 100644 --- a/src/webapp/x509.py +++ b/src/webapp/x509.py @@ -1,3 +1,4 @@ +import functools import hashlib import re @@ -17,6 +18,7 @@ } +@functools.lru_cache(maxsize=2048) def generate_dn_hash(dn: str) -> str: """ Given a DN one-liner as commonly encoded in the grid world From c4ae64f8d8fc731a1692c4272571fc31602a4c00 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 24 May 2024 21:08:30 -0500 Subject: [PATCH 21/32] Cache and pre-calculate authfile lines to speed up generation --- src/stashcache.py | 14 +++++------ src/webapp/data_federation.py | 46 +++++++++++++++++------------------ 2 files changed, 30 insertions(+), 30 deletions(-) diff --git a/src/stashcache.py b/src/stashcache.py index f2db0de99..a9ac3a5a7 100644 --- a/src/stashcache.py +++ b/src/stashcache.py @@ -172,10 +172,10 @@ def fetch_ligo_authz_list_if_needed(): for authz in extended_authz_list: if authz.used_in_authfile: - self.id_to_paths[authz.get_authfile_id()].add(path) - self.id_to_str[authz.get_authfile_id()] = str(authz) + self.id_to_paths[authz.authfile_id].add(path) + self.id_to_str[authz.authfile_id] = str(authz) if authz.used_in_grid_mapfile: - self.grid_mapfile_lines.add(authz.get_grid_mapfile_line()) + self.grid_mapfile_lines.add(authz.grid_mapfile_line) return self @@ -220,10 +220,10 @@ def for_origin(cls, topology: Topology, vos_data: VOsData, for authz in authz_list: if authz.used_in_authfile: - self.id_to_paths[authz.get_authfile_id()].add(path) - self.id_to_str[authz.get_authfile_id()] = str(authz) + self.id_to_paths[authz.authfile_id].add(path) + self.id_to_str[authz.authfile_id] = str(authz) if authz.used_in_grid_mapfile: - self.grid_mapfile_lines.add(authz.get_grid_mapfile_line()) + self.grid_mapfile_lines.add(authz.grid_mapfile_line) return self @@ -534,7 +534,7 @@ def get_credential_generation_dict_for_namespace(ns: Namespace) -> Optional[Dict def get_scitokens_list_for_namespace(ns: Namespace) -> List[Dict]: """Return the list of scitokens issuer info for the .namespaces[*].scitokens attribute in the namespaces JSON""" return list( - filter(None, (a.get_namespaces_scitokens_block() for a in ns.authz_list)) + filter(None, (a.namespaces_scitokens_block for a in ns.authz_list)) ) diff --git a/src/webapp/data_federation.py b/src/webapp/data_federation.py index 5f6ab796a..f7ae234ef 100644 --- a/src/webapp/data_federation.py +++ b/src/webapp/data_federation.py @@ -1,3 +1,4 @@ +import functools import re import urllib import urllib.parse @@ -12,22 +13,20 @@ class AuthMethod: + __slots__ = ("authfile_id", "grid_mapfile_line", "namespaces_scitokens_block") is_public = False used_in_authfile = False used_in_scitokens_conf = False used_in_grid_mapfile = False - def get_authfile_id(self): - return "" + def __init__(self): + self.authfile_id = "" + self.grid_mapfile_line = "" + self.namespaces_scitokens_block = None def get_scitokens_conf_block(self, service_name: str): return "" - def get_grid_mapfile_line(self): - return "" - - def get_namespaces_scitokens_block(self): - return None class NullAuth(AuthMethod): pass @@ -37,59 +36,60 @@ class PublicAuth(AuthMethod): is_public = True used_in_authfile = True + def __init__(self): + super().__init__() + self.authfile_id = "u *" + def __str__(self): return "PUBLIC" - def get_authfile_id(self): - return "u *" - class DNAuth(AuthMethod): + __slots__ = ("dn", "dn_hash") used_in_authfile = True used_in_grid_mapfile = True def __init__(self, dn: str): + super().__init__() self.dn = dn + self.dn_hash = generate_dn_hash(dn) + self.authfile_id = f"u {self.dn_hash}" + self.grid_mapfile_line = f'"{self.dn}" {self.dn_hash}' def __str__(self): return "DN: " + self.dn - def get_dn_hash(self): - return generate_dn_hash(self.dn) - - def get_authfile_id(self): - return f"u {self.get_dn_hash()}" - - def get_grid_mapfile_line(self): - return f'"{self.dn}" {self.get_dn_hash()}' - class FQANAuth(AuthMethod): + __slots__ = ("fqan",) used_in_authfile = True def __init__(self, fqan: str): + super().__init__() self.fqan = fqan + self.authfile_id = f"g {self.fqan}" def __str__(self): return "FQAN: " + self.fqan - def get_authfile_id(self): - return f"g {self.fqan}" - class SciTokenAuth(AuthMethod): + __slots__ = ("issuer", "base_path", "restricted_path", "map_subject") used_in_scitokens_conf = True def __init__(self, issuer: str, base_path: str, restricted_path: Optional[str], map_subject: bool): + super().__init__() self.issuer = issuer self.base_path = base_path self.restricted_path = restricted_path self.map_subject = map_subject + self.namespaces_scitokens_block = self._get_namespaces_scitokens_block() def __str__(self): return f"SciToken: issuer={self.issuer} base_path={self.base_path} restricted_path={self.restricted_path} " \ f"map_subject={self.map_subject}" + @functools.lru_cache(4) def get_scitokens_conf_block(self, service_name: str): if service_name not in [XROOTD_CACHE_SERVER, XROOTD_ORIGIN_SERVER]: raise ValueError(f"service_name must be '{XROOTD_CACHE_SERVER}' or '{XROOTD_ORIGIN_SERVER}'") @@ -103,7 +103,7 @@ def get_scitokens_conf_block(self, service_name: str): return block - def get_namespaces_scitokens_block(self): + def _get_namespaces_scitokens_block(self): base_path = re.split(r"\s*,\s*", self.base_path) restricted_path = re.split(r"\s*,\s*", self.restricted_path) if self.restricted_path else [] return { From 33b56f7dfa7cdb039d814e9f3d7c163d4b76cdc3 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 24 May 2024 21:45:12 -0500 Subject: [PATCH 22/32] Don't cache get_scitokens_conf_block() care must be taken when caching methods (see https://docs.python.org/3/faq/programming.html#how-do-i-cache-method-calls) and it's not called frequently enough to be worth it. --- src/webapp/data_federation.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/webapp/data_federation.py b/src/webapp/data_federation.py index f7ae234ef..8542f527f 100644 --- a/src/webapp/data_federation.py +++ b/src/webapp/data_federation.py @@ -1,4 +1,3 @@ -import functools import re import urllib import urllib.parse @@ -89,7 +88,6 @@ def __str__(self): return f"SciToken: issuer={self.issuer} base_path={self.base_path} restricted_path={self.restricted_path} " \ f"map_subject={self.map_subject}" - @functools.lru_cache(4) def get_scitokens_conf_block(self, service_name: str): if service_name not in [XROOTD_CACHE_SERVER, XROOTD_ORIGIN_SERVER]: raise ValueError(f"service_name must be '{XROOTD_CACHE_SERVER}' or '{XROOTD_ORIGIN_SERVER}'") From 8d918b8511e4ed1e11f402d11adc283fdf56a87b Mon Sep 17 00:00:00 2001 From: smithnp <60900055+smithnp@users.noreply.github.com> Date: Tue, 28 May 2024 13:13:03 -0400 Subject: [PATCH 23/32] Update Purdue_downtime.yaml --- .../Purdue University/Purdue CMS/Purdue_downtime.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/topology/Purdue University/Purdue CMS/Purdue_downtime.yaml b/topology/Purdue University/Purdue CMS/Purdue_downtime.yaml index 270898dcb..5ba31f0a8 100644 --- a/topology/Purdue University/Purdue CMS/Purdue_downtime.yaml +++ b/topology/Purdue University/Purdue CMS/Purdue_downtime.yaml @@ -2131,3 +2131,14 @@ Services: - net.perfSONAR.Latency # --------------------------------------------------------- +- Class: SCHEDULED + ID: 1819162937 + Description: Power transfer for datacenter circuits + Severity: Severe + StartTime: May 30, 2024 11:00 +0000 + EndTime: May 30, 2024 21:00 +0000 + CreatedTime: May 28, 2024 17:11 +0000 + ResourceName: Purdue-EOS-SE + Services: + - EOS +# --------------------------------------------------------- From a8ad9c03db0e7face3bda8e0fa425d350215a9a1 Mon Sep 17 00:00:00 2001 From: jlstephen Date: Tue, 28 May 2024 15:40:19 -0500 Subject: [PATCH 24/32] Add downtime for MWT2 for EL9 upgrades --- .../MWT2 ATLAS UC/MWT2_downtime.yaml | 99 +++++++++++++++++++ 1 file changed, 99 insertions(+) diff --git a/topology/University of Chicago/MWT2 ATLAS UC/MWT2_downtime.yaml b/topology/University of Chicago/MWT2 ATLAS UC/MWT2_downtime.yaml index 96a13392e..2da2fcf9c 100644 --- a/topology/University of Chicago/MWT2 ATLAS UC/MWT2_downtime.yaml +++ b/topology/University of Chicago/MWT2 ATLAS UC/MWT2_downtime.yaml @@ -1574,3 +1574,102 @@ Services: - XRootD component # --------------------------------------------------------- +- Class: SCHEDULED + ID: 1819285909 + Description: EL9 Upgrades + Severity: Outage + StartTime: Jun 10, 2024 14:00 +0000 + EndTime: Jun 10, 2024 22:00 +0000 + CreatedTime: May 28, 2024 20:36 +0000 + ResourceName: MWT2_CE_IU + Services: + - CE +# --------------------------------------------------------- +- Class: SCHEDULED + ID: 1819286180 + Description: EL9 Upgrades + Severity: Outage + StartTime: Jun 10, 2024 14:00 +0000 + EndTime: Jun 10, 2024 22:00 +0000 + CreatedTime: May 28, 2024 20:36 +0000 + ResourceName: MWT2_CE_IU2 + Services: + - CE +# --------------------------------------------------------- +- Class: SCHEDULED + ID: 1819286287 + Description: EL9 Upgrades + Severity: Outage + StartTime: Jun 10, 2024 14:00 +0000 + EndTime: Jun 10, 2024 22:00 +0000 + CreatedTime: May 28, 2024 20:37 +0000 + ResourceName: MWT2_CE_UC + Services: + - CE +# --------------------------------------------------------- +- Class: SCHEDULED + ID: 1819286377 + Description: EL9 Upgrades + Severity: Outage + StartTime: Jun 10, 2024 14:00 +0000 + EndTime: Jun 10, 2024 22:00 +0000 + CreatedTime: May 28, 2024 20:37 +0000 + ResourceName: MWT2_CE_UC2 + Services: + - CE +# --------------------------------------------------------- +- Class: SCHEDULED + ID: 1819286612 + Description: EL9 Upgrades + Severity: Outage + StartTime: Jun 10, 2024 14:00 +0000 + EndTime: Jun 10, 2024 22:00 +0000 + CreatedTime: May 28, 2024 20:37 +0000 + ResourceName: MWT2_CE_UIUC + Services: + - CE +# --------------------------------------------------------- +- Class: SCHEDULED + ID: 1819286698 + Description: EL9 Upgrades + Severity: Outage + StartTime: Jun 10, 2024 14:00 +0000 + EndTime: Jun 10, 2024 22:00 +0000 + CreatedTime: May 28, 2024 20:37 +0000 + ResourceName: MWT2_CE_UIUC2 + Services: + - CE +# --------------------------------------------------------- +- Class: SCHEDULED + ID: 1819286864 + Description: EL9 Upgrades + Severity: Outage + StartTime: Jun 10, 2024 14:00 +0000 + EndTime: Jun 10, 2024 22:00 +0000 + CreatedTime: May 28, 2024 20:38 +0000 + ResourceName: MWT2_UC_SE + Services: + - SRMv2 +# --------------------------------------------------------- +- Class: SCHEDULED + ID: 1819286951 + Description: EL9 Upgrades + Severity: Outage + StartTime: Jun 10, 2024 14:00 +0000 + EndTime: Jun 10, 2024 22:00 +0000 + CreatedTime: May 28, 2024 20:38 +0000 + ResourceName: MWT2_UC_WebDAV + Services: + - WebDAV +# --------------------------------------------------------- +- Class: SCHEDULED + ID: 1819287109 + Description: EL9 Upgrades + Severity: Outage + StartTime: Jun 10, 2024 14:00 +0000 + EndTime: Jun 10, 2024 22:00 +0000 + CreatedTime: May 28, 2024 20:38 +0000 + ResourceName: MWT2_UC_XRootD_door + Services: + - XRootD component +# --------------------------------------------------------- From 735a1c8cd5a58c8c3642286079b21491afc88a42 Mon Sep 17 00:00:00 2001 From: John Thiltges Date: Tue, 28 May 2024 21:10:18 -0500 Subject: [PATCH 25/32] IN2P3: Update stashcache DN to match CN (no SAN) --- topology/IN2P3/IN2P3-CC/IN2P3-OSDF.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topology/IN2P3/IN2P3-CC/IN2P3-OSDF.yaml b/topology/IN2P3/IN2P3-CC/IN2P3-OSDF.yaml index d63c69d50..38a16649a 100644 --- a/topology/IN2P3/IN2P3-CC/IN2P3-OSDF.yaml +++ b/topology/IN2P3/IN2P3-CC/IN2P3-OSDF.yaml @@ -15,7 +15,7 @@ Resources: Primary: Name: IN2P3 Security ID: 33fcdfcbfc32afbe7691bb90d29d1a48bcda8918 - DN: /DC=org/DC=terena/DC=tcs/C=FR/ST=Paris/O=Centre national de la recherche scientifique/CN=ccstashcache01.in2p3.fr + DN: /DC=org/DC=terena/DC=tcs/C=FR/ST=Paris/O=Centre national de la recherche scientifique/CN=ccstashcacheli01.in2p3.fr FQDN: ccstashcacheli01.in2p3.fr Services: XRootD cache server: From 5c0a33b6f897c6f790dd8d79dee1d2023d7eebcb Mon Sep 17 00:00:00 2001 From: Wei Yang Date: Tue, 28 May 2024 23:04:35 -0700 Subject: [PATCH 26/32] Update SLAC resource --- topology/Stanford University/SLAC/WT2.yaml | 35 +++++++++++++++++----- 1 file changed, 27 insertions(+), 8 deletions(-) diff --git a/topology/Stanford University/SLAC/WT2.yaml b/topology/Stanford University/SLAC/WT2.yaml index 1bd5fed96..fa73e0bfd 100644 --- a/topology/Stanford University/SLAC/WT2.yaml +++ b/topology/Stanford University/SLAC/WT2.yaml @@ -3,7 +3,7 @@ GroupID: 91 Production: true Resources: SLAC: - Active: true + Active: false ContactLists: Administrative Contact: Primary: @@ -28,7 +28,7 @@ Resources: VOOwnership: SLAC: 100 SLAC-ATLAS-T3_GRIDFTP: - Active: true + Active: false ContactLists: Administrative Contact: Primary: @@ -60,7 +60,7 @@ Resources: StorageCapacityMin: 5 TapeCapacity: 0 WT2: - Active: true + Active: false ContactLists: Administrative Contact: Primary: @@ -97,7 +97,7 @@ Resources: StorageCapacityMax: 0 StorageCapacityMin: 0 WT2_CE_2: - Active: true + Active: false ContactLists: Administrative Contact: Primary: @@ -118,7 +118,7 @@ Resources: VOOwnership: ATLAS: 100 WT2_SE: - Active: true + Active: false ContactLists: Administrative Contact: Primary: @@ -150,7 +150,7 @@ Resources: StorageCapacityMax: 3890 StorageCapacityMin: 3890 WT2_SE_2: - Active: true + Active: false ContactLists: Administrative Contact: Primary: @@ -182,7 +182,7 @@ Resources: StorageCapacityMin: 1 TapeCapacity: 0 perfSONAR_bw_SLAC: - Active: true + Active: false ContactLists: Administrative Contact: Primary: @@ -206,7 +206,7 @@ Resources: VOOwnership: ATLAS: 100 perfSONAR_lat_SLAC: - Active: true + Active: false ContactLists: Administrative Contact: Primary: @@ -226,5 +226,24 @@ Resources: endpoint: psnr-farm10.slac.stanford.edu VOOwnership: ATLAS: 100 + S3DFsquid: + Active: true + ContactLists: + Administrative Contact: + Primary: + ID: cf2de3eba56782b901bade70fe8bc1c18c36dd90 + Name: Wei Yang + Security Contact: + Primary: + ID: cf2de3eba56782b901bade70fe8bc1c18c36dd90 + Name: Wei Yang + Description: SLAC S3DF squid proxy + FQDN: sdfproxy.slac.stanford.edu + ID: + Services: + Squid: + Description: SLAC S3DF primary squid proxy + Details: + endpoint: sdfproxy.slac.stanford.edu SupportCenter: USATLAS From d7874461211978eeaed293283b65de70102b5ecd Mon Sep 17 00:00:00 2001 From: kurtjstrosahl-jlab <63591612+kurtjstrosahl-jlab@users.noreply.github.com> Date: Wed, 29 May 2024 10:20:25 -0400 Subject: [PATCH 27/32] Update JLab-Farm.yaml adding osg-ce-3, which will replace osg-ce-1 in June. --- .../JLAB/JLab-Farm.yaml | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/topology/Thomas Jefferson National Accelerator Facility/JLAB/JLab-Farm.yaml b/topology/Thomas Jefferson National Accelerator Facility/JLAB/JLab-Farm.yaml index f085f5ac0..ecee5c95c 100644 --- a/topology/Thomas Jefferson National Accelerator Facility/JLAB/JLab-Farm.yaml +++ b/topology/Thomas Jefferson National Accelerator Facility/JLAB/JLab-Farm.yaml @@ -262,5 +262,29 @@ Resources: hidden: false VOOwnership: JLab: 100 + + JLab-FARM-CE-3: + Active: true + ContactLists: + Administrative Contact: + Primary: + ID: 818acb7ebcd1865a17551799f0cfeb5256e0db20 + Name: Kurt Strosahl + Security Contact: + Primary: + ID: ea9db644eb6f8632cf87fd2cbac8a6e393c21f71 + Name: Wesley Moore + Description: RHEL9 HTcondor-CE for jlab farm + FQDN: osg-ce-3.jlab.org + FQDNAliases: + - osg-ce-3.jlab.org + ID: + Services: + CE: + Description: Compute Entry Point + Details: + hidden: false + VOOwnership: + JLab: 100 SupportCenter: Self Supported From 2934cd34570696441befa9464e6d54e1327ec78f Mon Sep 17 00:00:00 2001 From: Brian Lin Date: Wed, 29 May 2024 10:31:31 -0500 Subject: [PATCH 28/32] Retire /ospool/PROTECTED (INF-1883) --- .../CHTC/CHTC_OSPOOL.yaml | 2 +- virtual-organizations/OSG.yaml | 28 ------------------- 2 files changed, 1 insertion(+), 29 deletions(-) diff --git a/topology/University of Wisconsin/CHTC/CHTC_OSPOOL.yaml b/topology/University of Wisconsin/CHTC/CHTC_OSPOOL.yaml index 732dd235c..1b8571bac 100644 --- a/topology/University of Wisconsin/CHTC/CHTC_OSPOOL.yaml +++ b/topology/University of Wisconsin/CHTC/CHTC_OSPOOL.yaml @@ -5,7 +5,7 @@ GroupID: 1125 Resources: CHTC_OSPOOL_ORIGIN: - Active: true + Active: false Description: Authenticated origin server for OSPool Users at UW-Madison ID: 1194 ContactLists: diff --git a/virtual-organizations/OSG.yaml b/virtual-organizations/OSG.yaml index b612ea1e7..0f8413d44 100644 --- a/virtual-organizations/OSG.yaml +++ b/virtual-organizations/OSG.yaml @@ -117,34 +117,6 @@ DataFederations: AllowedCaches: - ANY - # HACK: enormous hack to get us going on a demo - # https://opensciencegrid.atlassian.net/browse/SOFTWARE-5398 - # FIXME: rip this out after the demo - # TODO: Redesign namespace interface (take 3?). See commit body - # for considerations. - - # NOTE: The SciTokens blocks for Issuer "https://osg-htc.org/ospool" must be the same - # between the paths /ospool/PROTECTED, /s3.amazonaws.com/us-east-1, and - # /s3.amazonaws.com/us-west-1 below or we will see problems. - # See c3524138ac8d46eee2a3c33cb75fac50acab41c4 for more information. - - - Path: /ospool/PROTECTED - Authorizations: - - SciTokens: - Issuer: https://osg-htc.org/ospool - Base Path: /ospool/PROTECTED,/s3.amazonaws.com/us-east-1,/s3.amazonaws.com/us-west-1 - Map Subject: True - AllowedOrigins: - - CHTC_OSPOOL_ORIGIN - AllowedCaches: - - ANY - Writeback: https://origin-auth2001.chtc.wisc.edu:1095 - DirList: https://origin-auth2001.chtc.wisc.edu:1095 - CredentialGeneration: - Strategy: OAuth2 - Issuer: https://osg-htc.org/ospool - MaxScopeDepth: 4 - - Path: /s3.amazonaws.com/us-east-1 Authorizations: - PUBLIC From 0b5b5777c6e9e59474fa744e11a2680a21a9f3e6 Mon Sep 17 00:00:00 2001 From: Brian Lin Date: Wed, 29 May 2024 10:32:34 -0500 Subject: [PATCH 29/32] Also retire the ITB S3 origins --- .../CHTC/CHTC-ITB.yaml | 4 +-- virtual-organizations/OSG.yaml | 36 ------------------- 2 files changed, 2 insertions(+), 38 deletions(-) diff --git a/topology/University of Wisconsin/CHTC/CHTC-ITB.yaml b/topology/University of Wisconsin/CHTC/CHTC-ITB.yaml index a7af50742..fc6d114db 100644 --- a/topology/University of Wisconsin/CHTC/CHTC-ITB.yaml +++ b/topology/University of Wisconsin/CHTC/CHTC-ITB.yaml @@ -311,7 +311,7 @@ Resources: Description: OSG VO backfill containers on the Tiger cluster, serving the ITB pool CHTC-ITB-S3-AWS-EAST-ORIGIN: - Active: true + Active: false Description: >- This is an origin used for demonstrating integration with AWS East S3 buckets @@ -343,7 +343,7 @@ Resources: - OSG CHTC-ITB-S3-AWS-WEST-ORIGIN: - Active: true + Active: false Description: >- This is an origin used for demonstrating integration with AWS West S3 buckets diff --git a/virtual-organizations/OSG.yaml b/virtual-organizations/OSG.yaml index 0f8413d44..515ef2326 100644 --- a/virtual-organizations/OSG.yaml +++ b/virtual-organizations/OSG.yaml @@ -117,42 +117,6 @@ DataFederations: AllowedCaches: - ANY - - Path: /s3.amazonaws.com/us-east-1 - Authorizations: - - PUBLIC - - SciTokens: - Issuer: https://osg-htc.org/ospool - Base Path: /ospool/PROTECTED,/s3.amazonaws.com/us-east-1,/s3.amazonaws.com/us-west-1 - Map Subject: True - AllowedOrigins: - - CHTC-ITB-S3-AWS-EAST-ORIGIN - AllowedCaches: - - ANY - Writeback: https://s3-us-east-1.osgdev.chtc.io:1095 - DirList: https://s3-us-east-1.osgdev.chtc.io:1095 - - - Path: /s3.amazonaws.com/us-west-1 - Authorizations: - - PUBLIC - - SciTokens: - Issuer: https://osg-htc.org/ospool - Base Path: /ospool/PROTECTED,/s3.amazonaws.com/us-east-1,/s3.amazonaws.com/us-west-1 - Map Subject: True - AllowedOrigins: - - CHTC-ITB-S3-AWS-WEST-ORIGIN - AllowedCaches: - - ANY - Writeback: https://s3-us-west-1.osgdev.chtc.io:1095 - DirList: https://s3-us-west-1.osgdev.chtc.io:1095 - - - Path: /osn-sdsc/us-west-1/bp3d-pelican - Authorizations: - - PUBLIC - AllowedOrigins: - - SDSC_NRP_OSDF_S3_ORIGIN - AllowedCaches: - - ANY - - Path: /nrdstor/protected Authorizations: - SciTokens: From 1ae9ac913bcf43dd155e0688a896c336bfaaf300 Mon Sep 17 00:00:00 2001 From: Brian Lin Date: Wed, 29 May 2024 10:44:03 -0500 Subject: [PATCH 30/32] Use an old-style OSDF origin --- src/tests/test_api.py | 2 +- src/tests/test_stashcache.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/tests/test_api.py b/src/tests/test_api.py index a93672c9b..251b8249c 100644 --- a/src/tests/test_api.py +++ b/src/tests/test_api.py @@ -203,7 +203,7 @@ def test_institution_accept_type(self, client: flask.Flask): def test_origin_grid_mapfile(self, client: flask.Flask): - TEST_ORIGIN = "origin-auth2001.chtc.wisc.edu" # This origin serves protected data + TEST_ORIGIN = "ap20.uc.osg-htc.org" # This origin serves protected data response = client.get("/origin/grid-mapfile") assert response.status_code == 400 # fqdn not specified diff --git a/src/tests/test_stashcache.py b/src/tests/test_stashcache.py index 53c0d1b72..1318dd318 100644 --- a/src/tests/test_stashcache.py +++ b/src/tests/test_stashcache.py @@ -201,7 +201,7 @@ def test_origin_grid_mapfile_nohost(self, client: flask.Flask): assert EMPTY_LINE_REGEX.match(line), f'Unexpected text "{line}".\nFull text:\n{text}\n' def test_origin_grid_mapfile_with_host(self, client: flask.Flask): - text = stashcache.generate_origin_grid_mapfile(global_data, "origin-auth2001.chtc.wisc.edu", + text = stashcache.generate_origin_grid_mapfile(global_data, "ap20.uc.osg-htc.org", suppress_errors=False) num_mappings = 0 for line in text.split("\n"): From 10f26f355f634ab6aff6933a58f8529be8cc7e21 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Wed, 29 May 2024 13:49:21 -0500 Subject: [PATCH 31/32] Update JLab-Farm.yaml ID not needed for new resources --- .../JLAB/JLab-Farm.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/topology/Thomas Jefferson National Accelerator Facility/JLAB/JLab-Farm.yaml b/topology/Thomas Jefferson National Accelerator Facility/JLAB/JLab-Farm.yaml index ecee5c95c..fdcefd5d7 100644 --- a/topology/Thomas Jefferson National Accelerator Facility/JLAB/JLab-Farm.yaml +++ b/topology/Thomas Jefferson National Accelerator Facility/JLAB/JLab-Farm.yaml @@ -278,7 +278,6 @@ Resources: FQDN: osg-ce-3.jlab.org FQDNAliases: - osg-ce-3.jlab.org - ID: Services: CE: Description: Compute Entry Point From aaaaad8c90fa709aff378f033c739dde84c041db Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Wed, 29 May 2024 14:47:53 -0500 Subject: [PATCH 32/32] Update WT2.yaml ID not needed for new resources --- topology/Stanford University/SLAC/WT2.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/topology/Stanford University/SLAC/WT2.yaml b/topology/Stanford University/SLAC/WT2.yaml index fa73e0bfd..1977924f2 100644 --- a/topology/Stanford University/SLAC/WT2.yaml +++ b/topology/Stanford University/SLAC/WT2.yaml @@ -239,7 +239,6 @@ Resources: Name: Wei Yang Description: SLAC S3DF squid proxy FQDN: sdfproxy.slac.stanford.edu - ID: Services: Squid: Description: SLAC S3DF primary squid proxy