Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System job port collision causes service job deployments to get stuck #18509

Open
andrba opened this issue Sep 15, 2023 · 1 comment
Open

System job port collision causes service job deployments to get stuck #18509

andrba opened this issue Sep 15, 2023 · 1 comment
Labels
hcc/jira stage/accepted Confirmed, and intend to work on. No timeline committment though. theme/scheduling type/bug

Comments

@andrba
Copy link

andrba commented Sep 15, 2023

Nomad version

Output from nomad version

Nomad v1.6.1
BuildDate 2023-07-21T13:49:42Z
Revision 515895c7690cdc72278018dc5dc58aca41204ccc

The issue was also reproduced on the previous Nomad versions:

  • 1.4.12
  • 1.5.7

Versions 1.1-1.3 behave as expected.

Operating system and Environment details

  • Debian GNU/Linux 10 (buster) Linux 5.10.0-0.deb10.24-amd64
  • MacOS Ventura 13.5.2 (22G91)

Issue

When a system job with a static port mapping is updated, its previous allocation may still be holding the port by the time the new allocation is started.

If such system job update is followed by an update of a service job that changes its list of datacenters, the service job evaluation reports a port collision error and its deployment gets stuck indefinitely.

Reproduction steps

  1. Start nomad agent -dev
  2. Create a system job with a task that runs a docker container listening on a static port (see sys.hcl). The container should have a delay on receiving a SIGINT and SIGTERM signals to simulate graceful shutdown
  3. Create a service job (see serv.hcl)
  4. Run the system and the service jobs back to back
  5. Verify that both jobs are running
  6. Change the system job template so that it would cause a new allocation to be created, e.g. change its env vars
  7. Change the datacenters property of the service job by adding or removing a datacenter from its list
  8. Run both jobs in quick succession
  9. The system job evaluation will not report any errors in the console output, however there will be a port collision record in the Nomad logs
  10. The service job evaluation will report port collisions and will be stuck

Here's a link to the repository with the Nomad jobs, the sample Docker image and the script that reliably reproduces this issue: https://github.com/andrba/nomad-port-collision-repro.

Expected Result

  • System job should not have port collisions
  • Service job evaluation should not report port collision errors
  • Service job deployment should update its allocations in-place
  • Service job should be successfully deployed

Actual Result

  • System job evaluation output:
==> 2023-09-15T11:18:42+10:00: Monitoring evaluation "5af82c13"
    2023-09-15T11:18:42+10:00: Evaluation triggered by job "sys"
    2023-09-15T11:18:42+10:00: Allocation "99c8bf78" created: node "7649f1e8", group "sleepy"
    2023-09-15T11:18:43+10:00: Evaluation status changed: "pending" -> "complete"
==> 2023-09-15T11:18:43+10:00: Evaluation "5af82c13" finished with status "complete"
  • Service job evaluation output:
==> 2023-09-15T11:18:43+10:00: Monitoring evaluation "6885b310"
    2023-09-15T11:18:43+10:00: Evaluation triggered by job "serv"
    2023-09-15T11:18:44+10:00: Evaluation within deployment: "93bb7c0b"
    2023-09-15T11:18:44+10:00: Evaluation status changed: "pending" -> "complete"
==> 2023-09-15T11:18:44+10:00: Evaluation "6885b310" finished with status "complete" but failed to place all allocations:
    2023-09-15T11:18:44+10:00: Task Group "group" (failed to place 1 allocation):
      * Resources exhausted on 1 nodes
      * Dimension "network: port collision" exhausted on 1 nodes
    2023-09-15T11:18:44+10:00: Evaluation "b2c76131" waiting for additional capacity to place remainder
  • Service job attempts to create canary allocations
==> 2023-09-15T11:18:44+10:00: Monitoring deployment "93bb7c0b"
  ⠧ Deployment "93bb7c0b" in progress...

    2023-09-15T11:18:57+10:00
    ID          = 93bb7c0b
    Job ID      = serv
    Job Version = 1
    Status      = running
    Description = Deployment is running pending automatic promotion

    Deployed
    Task Group  Auto Revert  Promoted  Desired  Canaries  Placed  Healthy  Unhealthy  Progress Deadline
    group       true         false     1        1         1       1        0          2023-09-15T11:20:57+10:00
  • Service job does not respect the deployment progress_deadline

Job file (if appropriate)

sys.hcl
job "sys" {
  datacenters = ["dc1"]
  type = "system"

  group "sleepy" {
    network {
      port "http" {
        static = 3002
      }
    }

    task "sleepy" {
      driver = "docker"

      config {
        network_mode = "host"
        image = "sleepy:1.0.0"
        command = "/go/bin/sleepy"
        args = [
          "-port",
          "${NOMAD_PORT_http}",
          "-shutdown-delay",
          "30",
        ]
      }

      env {
        ITERATION = 0
      }
    }
  }
}
serv.hcl
job "serv" {
  datacenters = ["dc1"]

  group "group" {
    count = 1

    update {
      canary            = 1
      max_parallel      = 1
      healthy_deadline  = "1m"
      progress_deadline = "2m"
      auto_revert       = true
      auto_promote      = true
    }

    task "task" {
      driver = "docker"
      config {
        network_mode = "host"
        image = "sleepy:1.0.0"
        command = "/go/bin/sleepy"
        args = ["-shutdown-delay", "0"]
      }
    }
  }
}

Nomad Server logs (if appropriate)

Port collision log record

2023-09-15T11:18:43.271+1000 [WARN] nomad: unexpected node port collision, refer to https://www.nomadproject.io/s/port-plan-failure for more information: node_id=7649f1e8-14ef-8d17-19b9-c723cc1ac19a reason="collision when reserving port for alloc b9884fca-02f8-e031-9d2c-aca274b6c01d: [port 3002 already in use]" event="{\"Reason\":\"collision when reserving port for alloc b9884fca-02f8-e031-9d2c-aca274b6c01d: [port 3002 already in use]\",\"Node\":{\"ID\":\"7649f1e8-14ef-8d17-19b9-c723cc1ac19a\",\"SecretID\":\"\",\"Datacenter\":\"dc1\",\"Name\":\"Andreys-MBP.localdomain\",\"CgroupParent\":\"\",\"HTTPAddr\":\"127.0.0.1:4646\",\"TLSEnabled\":false,\"Attributes\":{\"cpu.arch\":\"arm64\",\"cpu.frequency.efficiency\":\"2064\",\"cpu.frequency.power\":\"3228\",\"cpu.modelname\":\"Apple M1 Pro\",\"cpu.numcores.efficiency\":\"2\",\"cpu.numcores.power\":\"8\",\"cpu.reservablecores\":\"0\",\"cpu.totalcompute\":\"29952\",\"driver.docker\":\"1\",\"driver.docker.bridge_ip\":\"172.17.0.1\",\"driver.docker.os_type\":\"linux\",\"driver.docker.runtimes\":\"io.containerd.runc.v2,runc\",\"driver.docker.version\":\"24.0.5\",\"driver.raw_exec\":\"1\",\"kernel.arch\":\"arm64\",\"kernel.name\":\"darwin\",\"kernel.version\":\"22.6.0\",\"memory.totalbytes\":\"34359738368\",\"nomad.advertise.address\":\"127.0.0.1:4646\",\"nomad.revision\":\"515895c7690cdc72278018dc5dc58aca41204ccc\",\"nomad.service_discovery\":\"true\",\"nomad.version\":\"1.6.1\",\"os.name\":\"darwin\",\"os.signals\":\"SIGFPE,SIGNULL,SIGBUS,SIGTTOU,SIGIOT,SIGTRAP,SIGPIPE,SIGSTOP,SIGSYS,SIGTERM,SIGUSR1,SIGUSR2,SIGILL,SIGIO,SIGTTIN,SIGABRT,SIGKILL,SIGTSTP,SIGCONT,SIGPROF,SIGXCPU,SIGQUIT,SIGSEGV,SIGINT,SIGWINCH,SIGXFSZ,SIGALRM,SIGHUP\",\"os.version\":\"13.5.2\",\"unique.hostname\":\"Andreys-MBP.localdomain\",\"unique.network.ip-address\":\"127.0.0.1\",\"unique.storage.bytesfree\":\"873562386432\",\"unique.storage.bytestotal\":\"994662584320\",\"unique.storage.volume\":\"/dev/disk3s5\"},\"NodeResources\":{\"Cpu\":{\"CpuShares\":29952,\"TotalCpuCores\":10,\"ReservableCpuCores\":null},\"Memory\":{\"MemoryMB\":32768},\"Disk\":{\"DiskMB\":833094},\"Devices\":null,\"NodeNetworks\":[{\"Mode\":\"host\",\"Device\":\"lo0\",\"MacAddress\":\"\",\"Speed\":1000,\"Addresses\":[{\"Family\":\"ipv4\",\"Alias\":\"default\",\"Address\":\"127.0.0.1\",\"ReservedPorts\":\"\",\"Gateway\":\"\"},{\"Family\":\"ipv6\",\"Alias\":\"default\",\"Address\":\"::1\",\"ReservedPorts\":\"\",\"Gateway\":\"\"}]}],\"Networks\":[{\"Mode\":\"host\",\"Device\":\"lo0\",\"CIDR\":\"127.0.0.1/32\",\"IP\":\"127.0.0.1\",\"MBits\":1000,\"DNS\":null,\"ReservedPorts\":null,\"DynamicPorts\":null},{\"Mode\":\"host\",\"Device\":\"lo0\",\"CIDR\":\"::1/128\",\"IP\":\"::1\",\"MBits\":1000,\"DNS\":null,\"ReservedPorts\":null,\"DynamicPorts\":null}],\"MinDynamicPort\":20000,\"MaxDynamicPort\":32000},\"ReservedResources\":{\"Cpu\":{\"CpuShares\":0,\"ReservedCpuCores\":null},\"Memory\":{\"MemoryMB\":0},\"Disk\":{\"DiskMB\":0},\"Networks\":{\"ReservedHostPorts\":\"\"}},\"Resources\":{\"CPU\":29952,\"Cores\":0,\"MemoryMB\":32768,\"MemoryMaxMB\":0,\"DiskMB\":833094,\"IOPS\":0,\"Networks\":[{\"Mode\":\"host\",\"Device\":\"lo0\",\"CIDR\":\"127.0.0.1/32\",\"IP\":\"127.0.0.1\",\"MBits\":1000,\"DNS\":null,\"ReservedPorts\":null,\"DynamicPorts\":null},{\"Mode\":\"host\",\"Device\":\"lo0\",\"CIDR\":\"::1/128\",\"IP\":\"::1\",\"MBits\":1000,\"DNS\":null,\"ReservedPorts\":null,\"DynamicPorts\":null}],\"Devices\":null},\"Reserved\":{\"CPU\":0,\"Cores\":0,\"MemoryMB\":0,\"MemoryMaxMB\":0,\"DiskMB\":0,\"IOPS\":0,\"Networks\":null,\"Devices\":null},\"Links\":{},\"Meta\":{},\"NodeClass\":\"\",\"NodePool\":\"default\",\"ComputedClass\":\"v1:18054280369589802710\",\"DrainStrategy\":null,\"SchedulingEligibility\":\"eligible\",\"Status\":\"ready\",\"StatusDescription\":\"\",\"StatusUpdatedAt\":1694740693,\"Events\":[{\"Message\":\"Node registered\",\"Subsystem\":\"Cluster\",\"Details\":null,\"Timestamp\":\"2023-09-15T11:18:12+10:00\",\"CreateIndex\":0}],\"Drivers\":{\"docker\":{\"Attributes\":{\"driver.docker\":\"true\",\"driver.docker.bridge_ip\":\"172.17.0.1\",\"driver.docker.os_type\":\"linux\",\"driver.docker.runtimes\":\"io.containerd.runc.v2,runc\",\"driver.docker.version\":\"24.0.5\"},\"Detected\":true,\"Healthy\":true,\"HealthDescription\":\"Healthy\",\"UpdateTime\":\"2023-09-15T11:18:12.120507+10:00\"},\"exec\":{\"Attributes\":{},\"Detected\":false,\"Healthy\":false,\"HealthDescription\":\"exec driver unsupported on client OS\",\"UpdateTime\":\"2023-09-15T11:18:05.500889+10:00\"},\"java\":{\"Attributes\":{},\"Detected\":false,\"Healthy\":false,\"HealthDescription\":\"\",\"UpdateTime\":\"2023-09-15T11:18:05.518994+10:00\"},\"qemu\":{\"Attributes\":{},\"Detected\":false,\"Healthy\":false,\"HealthDescription\":\"\",\"UpdateTime\":\"2023-09-15T11:18:05.50298+10:00\"},\"raw_exec\":{\"Attributes\":{\"driver.raw_exec\":\"true\"},\"Detected\":true,\"Healthy\":true,\"HealthDescription\":\"Healthy\",\"UpdateTime\":\"2023-09-15T11:18:05.500834+10:00\"}},\"CSIControllerPlugins\":{},\"CSINodePlugins\":{},\"HostVolumes\":null,\"HostNetworks\":null,\"LastDrain\":null,\"LastMissedHeartbeatIndex\":0,\"LastAllocUpdateIndex\":27,\"CreateIndex\":8,\"ModifyIndex\":10},\"Allocations\":[{\"ID\":\"99c8bf78-48cd-72f1-b7c6-b58227324740\",\"Namespace\":\"default\",\"EvalID\":\"5af82c13-8c98-b8a1-e61d-79528d6d3eef\",\"Name\":\"sys.sleepy[0]\",\"NodeID\":\"7649f1e8-14ef-8d17-19b9-c723cc1ac19a\",\"NodeName\":\"Andreys-MBP.localdomain\",\"JobID\":\"sys\",\"Job\":null,\"TaskGroup\":\"sleepy\",\"Resources\":{\"CPU\":100,\"Cores\":0,\"MemoryMB\":300,\"MemoryMaxMB\":300,\"DiskMB\":300,\"IOPS\":0,\"Networks\":[{\"Mode\":\"\",\"Device\":\"\",\"CIDR\":\"\",\"IP\":\"127.0.0.1\",\"MBits\":0,\"DNS\":null,\"ReservedPorts\":[{\"Label\":\"http\",\"Value\":3002,\"To\":0,\"HostNetwork\":\"default\"}],\"DynamicPorts\":null}],\"Devices\":null},\"SharedResources\":{\"CPU\":0,\"Cores\":0,\"MemoryMB\":0,\"MemoryMaxMB\":0,\"DiskMB\":300,\"IOPS\":0,\"Networks\":[{\"Mode\":\"\",\"Device\":\"\",\"CIDR\":\"\",\"IP\":\"127.0.0.1\",\"MBits\":0,\"DNS\":null,\"ReservedPorts\":[{\"Label\":\"http\",\"Value\":3002,\"To\":0,\"HostNetwork\":\"default\"}],\"DynamicPorts\":null}],\"Devices\":null},\"TaskResources\":{\"sleepy\":{\"CPU\":100,\"Cores\":0,\"MemoryMB\":300,\"MemoryMaxMB\":0,\"DiskMB\":0,\"IOPS\":0,\"Networks\":null,\"Devices\":null}},\"AllocatedResources\":{\"Tasks\":{\"sleepy\":{\"Cpu\":{\"CpuShares\":100,\"ReservedCores\":null},\"Memory\":{\"MemoryMB\":300,\"MemoryMaxMB\":0},\"Networks\":null,\"Devices\":null}},\"TaskLifecycles\":{\"sleepy\":null},\"Shared\":{\"Networks\":[{\"Mode\":\"\",\"Device\":\"\",\"CIDR\":\"\",\"IP\":\"127.0.0.1\",\"MBits\":0,\"DNS\":null,\"ReservedPorts\":[{\"Label\":\"http\",\"Value\":3002,\"To\":0,\"HostNetwork\":\"default\"}],\"DynamicPorts\":null}],\"DiskMB\":300,\"Ports\":[{\"Label\":\"http\",\"Value\":3002,\"To\":0,\"HostIP\":\"127.0.0.1\"}]}},\"Metrics\":{\"NodesEvaluated\":1,\"NodesFiltered\":0,\"NodesInPool\":1,\"NodesAvailable\":{\"dc1\":1},\"ClassFiltered\":null,\"ConstraintFiltered\":null,\"NodesExhausted\":0,\"ClassExhausted\":null,\"DimensionExhausted\":null,\"QuotaExhausted\":null,\"ResourcesExhausted\":null,\"Scores\":null,\"ScoreMetaData\":[{\"NodeID\":\"7649f1e8-14ef-8d17-19b9-c723cc1ac19a\",\"Scores\":{\"binpack\":0.03141262554317809},\"NormScore\":0.03141262554317809}],\"AllocationTime\":29333,\"CoalescedFailures\":0},\"DesiredStatus\":\"run\",\"DesiredDescription\":\"\",\"DesiredTransition\":{\"Migrate\":null,\"Reschedule\":null,\"ForceReschedule\":null,\"NoShutdownDelay\":null},\"ClientStatus\":\"pending\",\"ClientDescription\":\"\",\"TaskStates\":null,\"AllocStates\":null,\"PreviousAllocation\":\"b9884fca-02f8-e031-9d2c-aca274b6c01d\",\"NextAllocation\":\"\",\"DeploymentID\":\"\",\"DeploymentStatus\":null,\"RescheduleTracker\":null,\"NetworkStatus\":null,\"FollowupEvalID\":\"\",\"PreemptedAllocations\":null,\"PreemptedByAllocation\":\"\",\"SigningKeyID\":\"47abed21-1e77-4115-2d37-a64d3828d165\",\"CreateIndex\":25,\"ModifyIndex\":25,\"AllocModifyIndex\":25,\"CreateTime\":1694740722170418000,\"ModifyTime\":1694740722170418000},{\"ID\":\"b9884fca-02f8-e031-9d2c-aca274b6c01d\",\"Namespace\":\"default\",\"EvalID\":\"0242b8ba-f412-26a9-6534-0d21e84347e8\",\"Name\":\"sys.sleepy[0]\",\"NodeID\":\"7649f1e8-14ef-8d17-19b9-c723cc1ac19a\",\"NodeName\":\"Andreys-MBP.localdomain\",\"JobID\":\"sys\",\"Job\":null,\"TaskGroup\":\"sleepy\",\"Resources\":{\"CPU\":100,\"Cores\":0,\"MemoryMB\":300,\"MemoryMaxMB\":300,\"DiskMB\":300,\"IOPS\":0,\"Networks\":[{\"Mode\":\"\",\"Device\":\"\",\"CIDR\":\"\",\"IP\":\"127.0.0.1\",\"MBits\":0,\"DNS\":null,\"ReservedPorts\":[{\"Label\":\"http\",\"Value\":3002,\"To\":0,\"HostNetwork\":\"default\"}],\"DynamicPorts\":null}],\"Devices\":null},\"SharedResources\":{\"CPU\":0,\"Cores\":0,\"MemoryMB\":0,\"MemoryMaxMB\":0,\"DiskMB\":300,\"IOPS\":0,\"Networks\":[{\"Mode\":\"\",\"Device\":\"\",\"CIDR\":\"\",\"IP\":\"127.0.0.1\",\"MBits\":0,\"DNS\":null,\"ReservedPorts\":[{\"Label\":\"http\",\"Value\":3002,\"To\":0,\"HostNetwork\":\"default\"}],\"DynamicPorts\":null}],\"Devices\":null},\"TaskResources\":{\"sleepy\":{\"CPU\":100,\"Cores\":0,\"MemoryMB\":300,\"MemoryMaxMB\":0,\"DiskMB\":0,\"IOPS\":0,\"Networks\":null,\"Devices\":null}},\"AllocatedResources\":{\"Tasks\":{\"sleepy\":{\"Cpu\":{\"CpuShares\":100,\"ReservedCores\":null},\"Memory\":{\"MemoryMB\":300,\"MemoryMaxMB\":0},\"Networks\":null,\"Devices\":null}},\"TaskLifecycles\":{\"sleepy\":null},\"Shared\":{\"Networks\":[{\"Mode\":\"\",\"Device\":\"\",\"CIDR\":\"\",\"IP\":\"127.0.0.1\",\"MBits\":0,\"DNS\":null,\"ReservedPorts\":[{\"Label\":\"http\",\"Value\":3002,\"To\":0,\"HostNetwork\":\"default\"}],\"DynamicPorts\":null}],\"DiskMB\":300,\"Ports\":[{\"Label\":\"http\",\"Value\":3002,\"To\":0,\"HostIP\":\"127.0.0.1\"}]}},\"Metrics\":{\"NodesEvaluated\":1,\"NodesFiltered\":0,\"NodesInPool\":1,\"NodesAvailable\":{\"dc1\":1},\"ClassFiltered\":null,\"ConstraintFiltered\":null,\"NodesExhausted\":0,\"ClassExhausted\":null,\"DimensionExhausted\":null,\"QuotaExhausted\":null,\"ResourcesExhausted\":null,\"Scores\":null,\"ScoreMetaData\":[{\"NodeID\":\"7649f1e8-14ef-8d17-19b9-c723cc1ac19a\",\"Scores\":{\"binpack\":0.015843477348731087},\"NormScore\":0.015843477348731087}],\"AllocationTime\":211292,\"CoalescedFailures\":0},\"DesiredStatus\":\"stop\",\"DesiredDescription\":\"alloc is being updated due to job update\",\"DesiredTransition\":{\"Migrate\":null,\"Reschedule\":null,\"ForceReschedule\":null,\"NoShutdownDelay\":null},\"ClientStatus\":\"running\",\"ClientDescription\":\"Tasks are running\",\"TaskStates\":{\"sleepy\":{\"State\":\"running\",\"Failed\":false,\"Restarts\":0,\"LastRestart\":\"0001-01-01T00:00:00Z\",\"StartedAt\":\"2023-09-15T01:18:24.316895Z\",\"FinishedAt\":\"0001-01-01T00:00:00Z\",\"Events\":[{\"Type\":\"Received\",\"Time\":1694740704132151000,\"Message\":\"\",\"DisplayMessage\":\"Task received by client\",\"Details\":{},\"FailsTask\":false,\"RestartReason\":\"\",\"SetupError\":\"\",\"DriverError\":\"\",\"ExitCode\":0,\"Signal\":0,\"KillTimeout\":0,\"KillError\":\"\",\"KillReason\":\"\",\"StartDelay\":0,\"DownloadError\":\"\",\"ValidationError\":\"\",\"DiskLimit\":0,\"FailedSibling\":\"\",\"VaultError\":\"\",\"TaskSignalReason\":\"\",\"TaskSignal\":\"\",\"DriverMessage\":\"\",\"GenericSource\":\"\"},{\"Type\":\"Task Setup\",\"Time\":1694740704132789000,\"Message\":\"Building Task Directory\",\"DisplayMessage\":\"Building Task Directory\",\"Details\":{\"message\":\"Building Task Directory\"},\"FailsTask\":false,\"RestartReason\":\"\",\"SetupError\":\"\",\"DriverError\":\"\",\"ExitCode\":0,\"Signal\":0,\"KillTimeout\":0,\"KillError\":\"\",\"KillReason\":\"\",\"StartDelay\":0,\"DownloadError\":\"\",\"ValidationError\":\"\",\"DiskLimit\":0,\"FailedSibling\":\"\",\"VaultError\":\"\",\"TaskSignalReason\":\"\",\"TaskSignal\":\"\",\"DriverMessage\":\"\",\"GenericSource\":\"\"},{\"Type\":\"Started\",\"Time\":1694740704316863000,\"Message\":\"\",\"DisplayMessage\":\"Task started by client\",\"Details\":{},\"FailsTask\":false,\"RestartReason\":\"\",\"SetupError\":\"\",\"DriverError\":\"\",\"ExitCode\":0,\"Signal\":0,\"KillTimeout\":0,\"KillError\":\"\",\"KillReason\":\"\",\"StartDelay\":0,\"DownloadError\":\"\",\"ValidationError\":\"\",\"DiskLimit\":0,\"FailedSibling\":\"\",\"VaultError\":\"\",\"TaskSignalReason\":\"\",\"TaskSignal\":\"\",\"DriverMessage\":\"\",\"GenericSource\":\"\"},{\"Type\":\"Killing\",\"Time\":1694740722171274000,\"Message\":\"\",\"DisplayMessage\":\"Sent interrupt. Waiting 5s before force killing\",\"Details\":{\"kill_timeout\":\"5s\"},\"FailsTask\":false,\"RestartReason\":\"\",\"SetupError\":\"\",\"DriverError\":\"\",\"ExitCode\":0,\"Signal\":0,\"KillTimeout\":5000000000,\"KillError\":\"\",\"KillReason\":\"\",\"StartDelay\":0,\"DownloadError\":\"\",\"ValidationError\":\"\",\"DiskLimit\":0,\"FailedSibling\":\"\",\"VaultError\":\"\",\"TaskSignalReason\":\"\",\"TaskSignal\":\"\",\"DriverMessage\":\"\",\"GenericSource\":\"\"}],\"TaskHandle\":null}},\"AllocStates\":null,\"PreviousAllocation\":\"\",\"NextAllocation\":\"99c8bf78-48cd-72f1-b7c6-b58227324740\",\"DeploymentID\":\"\",\"DeploymentStatus\":null,\"RescheduleTracker\":null,\"NetworkStatus\":{\"InterfaceName\":\"\",\"Address\":\"\",\"DNS\":null},\"FollowupEvalID\":\"\",\"PreemptedAllocations\":null,\"PreemptedByAllocation\":\"\",\"SigningKeyID\":\"47abed21-1e77-4115-2d37-a64d3828d165\",\"CreateIndex\":12,\"ModifyIndex\":27,\"AllocModifyIndex\":25,\"CreateTime\":1694740704130450000,\"ModifyTime\":1694740722632551000}],\"NetIndex\":{\"TaskNetworks\":[{\"Mode\":\"host\",\"Device\":\"lo0\",\"CIDR\":\"127.0.0.1/32\",\"IP\":\"127.0.0.1\",\"MBits\":1000,\"DNS\":null,\"ReservedPorts\":null,\"DynamicPorts\":null},{\"Mode\":\"host\",\"Device\":\"lo0\",\"CIDR\":\"::1/128\",\"IP\":\"::1\",\"MBits\":1000,\"DNS\":null,\"ReservedPorts\":null,\"DynamicPorts\":null}],\"GroupNetworks\":null,\"HostNetworks\":{\"default\":[{\"Family\":\"ipv4\",\"Alias\":\"default\",\"Address\":\"127.0.0.1\",\"ReservedPorts\":\"\",\"Gateway\":\"\"},{\"Family\":\"ipv6\",\"Alias\":\"default\",\"Address\":\"::1\",\"ReservedPorts\":\"\",\"Gateway\":\"\"}]},\"UsedPorts\":{\"127.0.0.1\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\",\"::1\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\"},\"AvailBandwidth\":{\"lo0\":1000},\"UsedBandwidth\":{},\"MinDynamicPort\":20000,\"MaxDynamicPort\":32000}}"

Nomad Client logs (if appropriate)

@tgross
Copy link
Member

tgross commented Sep 15, 2023

Hi @andrba and thanks for this detailed breakdown! There's been a couple variants of this same kind of bug and I thought we had it root caused and fixed in Nomad 1.5.1 with #16401 but apparently not quite.

I'm going to bring this to the attention of @mikenomitch and @jrasell in particular, and mark it for roadmapping.

@tgross tgross added stage/accepted Confirmed, and intend to work on. No timeline committment though. theme/scheduling labels Sep 15, 2023
@tgross tgross moved this to Needs Roadmapping in Nomad - Community Issues Triage Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hcc/jira stage/accepted Confirmed, and intend to work on. No timeline committment though. theme/scheduling type/bug
Projects
Status: Needs Roadmapping
Development

No branches or pull requests

3 participants