UPDATE 2021-04-18
I've built gcc-arm 10.2.1 aarch64 in anticipation for an upcoming Particle deviceOS release.
I've been testing Manjaro ARM XFCE on Raspberry Pi and I discovered that because it's a 64-bit operating system armv7l binaries do not work.
I found prebuilt gcc-arm 9.2.1 aarch64 and I have included it in this release so that it can be downloaded within neopo.
Because compiling gcc-arm 9.2.1 aarch64 kept filling up the 32GB micro SD card I was using I decided to buy a 256GB Micro SD card and complete the compilation using the new card. I have uploaded the compiled toolchain on GitHub.
I successfully built gcc-arm 5.3.1 for aarch64. I had to make changes to several config.sub files to add support for aarch64.
I also had to make a change to src/gdb/gdb/configure.
Finally, on line 604 of build-toolchain.sh I had to change -perm +111 to -perm -111. This was stopping the script before strip could be used to reduce the size of all binaries. I'm certain that I can reduce the size of the armv7l 5.3.1 binaries by using strip.