crypto/aes: speedup CTR mode on AMD64 and ARM64

[starius]

The implementation runs up to 8 AES instructions in different registers
one after another in ASM code. Because CPU has instruction pipelining
and the instructions do not depend on each other, they can run in
parallel with this layout of code. This results in significant speedup
compared to the regular implementation in which blocks are processed in
the same registers so AES instructions do not run in parallel.

GCM mode already utilizes the approach.

The ASM implementation of ctrAble has most of its code in XORKeyStreamAt
method which has an additional argument, offset. It allows to use it
in a stateless way and to jump to any location in the stream. The method
does not exist in pure Go and boringcrypto implementations.

AES CTR benchmark delta.

$ go test crypto/cipher -bench 'BenchmarkAESCTR*'

AMD64. Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
name old time/op new time/op delta
BenchmarkAESCTR1K-2 1259ns 266.9ns -78.8%
BenchmarkAESCTR8K-2 9859ns 1953ns -80.1%

ARM64. ARM Neoverse-N1 (AWS EC2 t4g.small instance)
name old time/op new time/op delta
BenchmarkAESCTR1K-2 1098ns 481.1ns -56.2%
BenchmarkAESCTR8K-2 8447ns 3452ns -59.1%

ARM64. Apple M1
name old time/op new time/op delta
BenchmarkAESCTR1K-2 455.3ns 154.3ns -66.1%
BenchmarkAESCTR8K-2 3491ns 1116ns -68.0%

Fixes #20967

Investigation and initial implementation:
https://github.com/mmcloughlin/aesnix/
Full implementation in external repo:
https://github.com/starius/aesctrat

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[gopherbot]

This PR (HEAD: d9b6a72) has been imported to Gerrit for code review.

Please visit https://go-review.googlesource.com/c/go/+/413594 to see it.

Tip: You can toggle comments from me using the comments slash command (e.g. /comments off)
See the Wiki page for more info

[gopherbot]

Message from Gopher Robot:

Patch Set 1:

Congratulations on opening your first change. Thank you for your contribution!

Next steps:
A maintainer will review your change and provide feedback. See
https://go.dev/doc/contribute#review for more info and tips to get your
patch through code review.

Most changes in the Go project go through a few rounds of revision. This can be
surprising to people new to the project. The careful, iterative review process
is our way of helping mentor contributors and ensuring that their contributions
have a lasting impact.

During May-July and Nov-Jan the Go project is in a code freeze, during which
little code gets reviewed or merged. If a reviewer responds with a comment like
R=go1.11 or adds a tag like "wait-release", it means that this CL will be
reviewed as part of the next development cycle. See https://go.dev/s/release
for more details.

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Борис Нагаев:

Patch Set 1:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Ian Lance Taylor:

Patch Set 2:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[andig]

@starius would be great if you were following up on the python vs go topic?

[starius]

@andig I'm rewriting the generators in Go. Want to make a single Go program producing for all architectures.

[gopherbot]

This PR (HEAD: e24634a) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/413594.

Important tips:

• Don't comment on this PR. All discussion takes place in Gerrit.
• You need a Gmail or other Google account to register for Gerrit.
• To change your code in response to feedback:
  • Push a new commit to the branch used by your GitHub PR.
  • A new "patch set" will then appear in Gerrit.
  • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
  • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
  • Multiple commits in the PR will be squashed by GerritBot.
• The title and description of the GitHub PR are used to construct the final commit message.
  • Edit these as needed via the GitHub web interface (not via Gerrit or git).
  • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
• See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

[starius]

@andig I finished with generators and added correctness test. Waiting for review in Gerrit :-)

[gopherbot]

Message from Борис Нагаев:

Patch Set 3:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Ian Lance Taylor:

Patch Set 3: Commit-Queue+1

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Go LUCI:

Patch Set 3:

Dry run: CV is trying the patch.

Bot data: {"action":"start","triggered_at":"2023-09-12T01:14:29Z","revision":"e73f75e3161f92b39819bc5ff5b915f7a9c87bda"}

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Ian Lance Taylor:

Patch Set 3: -Commit-Queue

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Go LUCI:

Patch Set 3:

This CL has failed the run. Reason:

Tryjob golang/try/gotip-linux-arm64 has failed

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Go LUCI:

Patch Set 3: LUCI-TryBot-Result-1

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

This PR (HEAD: 9683c85) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/413594.

Important tips:

• Don't comment on this PR. All discussion takes place in Gerrit.
• You need a Gmail or other Google account to register for Gerrit.
• To change your code in response to feedback:
  • Push a new commit to the branch used by your GitHub PR.
  • A new "patch set" will then appear in Gerrit.
  • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
  • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
  • Multiple commits in the PR will be squashed by GerritBot.
• The title and description of the GitHub PR are used to construct the final commit message.
  • Edit these as needed via the GitHub web interface (not via Gerrit or git).
  • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
• See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

[gopherbot]

Message from Борис Нагаев:

Patch Set 4:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Gopher Robot:

Patch Set 1:

Congratulations on opening your first change. Thank you for your contribution!

Next steps:
A maintainer will review your change and provide feedback. See
https://go.dev/doc/contribute#review for more info and tips to get your
patch through code review.

Most changes in the Go project go through a few rounds of revision. This can be
surprising to people new to the project. The careful, iterative review process
is our way of helping mentor contributors and ensuring that their contributions
have a lasting impact.

During May-July and Nov-Jan the Go project is in a code freeze, during which
little code gets reviewed or merged. If a reviewer responds with a comment like
R=go1.11 or adds a tag like "wait-release", it means that this CL will be
reviewed as part of the next development cycle. See https://go.dev/s/release
for more details.

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Борис Нагаев:

Patch Set 1:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Ian Lance Taylor:

Patch Set 2:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Борис Нагаев:

Patch Set 3:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Ian Lance Taylor:

Patch Set 3: Commit-Queue+1

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Go LUCI:

Patch Set 3:

Dry run: CV is trying the patch.

Bot data: {"action":"start","triggered_at":"2023-09-12T01:14:29Z","revision":"e73f75e3161f92b39819bc5ff5b915f7a9c87bda"}

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Ian Lance Taylor:

Patch Set 3: -Commit-Queue

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Go LUCI:

Patch Set 3:

This CL has failed the run. Reason:

Tryjob golang/try/gotip-linux-arm64 has failed

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Go LUCI:

Patch Set 3: LUCI-TryBot-Result-1

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Борис Нагаев:

Patch Set 4:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Nicola Murino:

Patch Set 6: Run-TryBot+1

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Gopher Robot:

Patch Set 6:

(2 comments)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Gopher Robot:

Patch Set 6:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Gopher Robot:

Patch Set 6: TryBot-Result-1

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

This PR (HEAD: 92848a1) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/413594.

Important tips:

• Don't comment on this PR. All discussion takes place in Gerrit.
• You need a Gmail or other Google account to log in to Gerrit.
• To change your code in response to feedback:
  • Push a new commit to the branch used by your GitHub PR.
  • A new "patch set" will then appear in Gerrit.
  • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
  • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
  • Multiple commits in the PR will be squashed by GerritBot.
• The title and description of the GitHub PR are used to construct the final commit message.
  • Edit these as needed via the GitHub web interface (not via Gerrit or git).
  • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
• See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

[gopherbot]

Message from Nicola Murino:

Patch Set 7: Run-TryBot+1

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Gopher Robot:

Patch Set 7:

(2 comments)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Борис Нагаев:

Patch Set 6:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Gopher Robot:

Patch Set 7: TryBot-Result+1

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from qiulaidongfeng:

Patch Set 8: Run-TryBot+1

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from qiulaidongfeng:

Patch Set 8: -Run-TryBot

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Ben Darnell:

Patch Set 8:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from qiulaidongfeng:

Patch Set 8:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Ben Darnell:

Patch Set 8:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from qiulaidongfeng:

Patch Set 8:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

This PR (HEAD: e087300) has been imported to Gerrit for code review.

Please visit Gerrit at https://go-review.googlesource.com/c/go/+/413594.

Important tips:

• Don't comment on this PR. All discussion takes place in Gerrit.
• You need a Gmail or other Google account to log in to Gerrit.
• To change your code in response to feedback:
  • Push a new commit to the branch used by your GitHub PR.
  • A new "patch set" will then appear in Gerrit.
  • Respond to each comment by marking as Done in Gerrit if implemented as suggested. You can alternatively write a reply.
  • Critical: you must click the blue Reply button near the top to publish your Gerrit responses.
  • Multiple commits in the PR will be squashed by GerritBot.
• The title and description of the GitHub PR are used to construct the final commit message.
  • Edit these as needed via the GitHub web interface (not via Gerrit or git).
  • You should word wrap the PR description at ~76 characters unless you need longer lines (e.g., for tables or URLs).
• See the Sending a change via GitHub and Reviews sections of the Contribution Guide as well as the FAQ for details.

[gopherbot]

Message from Борис Нагаев:

Patch Set 8:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Борис Нагаев:

Patch Set 9:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from qiulaidongfeng:

Patch Set 9:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Ben Darnell:

Patch Set 9:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Борис Нагаев:

Patch Set 9:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Filippo Valsorda:

Patch Set 9:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!

[gopherbot]

Message from Filippo Valsorda:

Patch Set 9:

(1 comment)

---

Please don’t reply on this GitHub thread. Visit golang.org/cl/413594.
After addressing review feedback, remember to publish your drafts!