PAT Based Authentication not working

[akulagrawal]

Getting the following error when trying to use VSS GitHttpClient using PAT based authentication:
{
"Message": "An error has occurred.",
"ExceptionMessage": "The resource cannot be found.",
"ExceptionType": "Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException",
"StackTrace": " at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.d__53.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.d__51.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.d__471.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.<GetConnectionDataAsync>d__6.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.<GetConnectionDataAsync>d__56.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.<ConnectAsync>d__41.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.<EnsureConnectedAsync>d__39.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.<GetInstanceIdAsync>d__11.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.VisualStudio.Services.WebApi.Location.LocationService.<GetLocationDataAsync>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.VisualStudio.Services.WebApi.VssConnection.<GetClientInstanceAsync>d__20.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.VisualStudio.Services.WebApi.VssConnection.<GetClientServiceImplAsync>d__17.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.VisualStudio.Services.WebApi.VssConnection.<GetClientAsync>d__141.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Microsoft.VisualStudio.Services.WebApi.TaskExtensions.SyncResult[T](Task1 task)\r\n at Microsoft.VisualStudio.Services.WebApi.VssConnection.GetClient[T]()\r\n at Microsoft.M365.ConfigMgtTool.Orchestrator.ReplaceManager.<GetMock>d__11.MoveNext() in C:\\Users\\akula\\Documents\\repos\\config_mgt_tool\\sources\\dev\\ConfigMgtTool\\Orchestrator\\ReplaceManager.cs:line 105\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult()\r\n at Microsoft.M365.ConfigMgtTool.ConfigManager.Controllers.ReplaceController.d__3.MoveNext() in C:\Users\akula\Documents\repos\config_mgt_tool\sources\dev\ConfigMgtTool\Controllers\ReplaceController.cs:line 63\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Threading.Tasks.TaskHelpersExtensions.d__1`1.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Controllers.ApiControllerActionInvoker.d__1.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Controllers.ActionFilterResult.d__5.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at System.Web.Http.Dispatcher.HttpControllerDispatcher.d__15.MoveNext()"
}

Original code I am using:
public async Task GetMock()
{
VssConnection connection = new VssConnection(new Uri(ConfigurationManager.AppSettings["ADOBaseAddress"]), new VssBasicCredential(string.Empty, GetSecretClient().GetSecret(Constants.ADOSecret).Value.ToString()));
GitHttpClient gitClient = connection.GetClient();
List repos = await gitClient.GetRepositoriesAsync(project).ConfigureAwait(false);
foreach (GitRepository repo in repos)
{
if (repo.Name == repository)
{
return repo.Id.ToString();
}
}
return string.Empty;
}