Linker issue break Lingua Franca CI

[schoeberl]

Lingua Franca (LF) has Patmos support and includes Patmos and the compiler in the CI. Therefore, this is an important issue.

How to reproduce:
Just with a simple "hello world"

patmos-clang -c hello.c
patmos-llvm-link hello.o

errors into:

patmos-llvm-link: hello.o:1:1: error: expected top-level entity
ELF???4?|7?|8?=??????????@A?'?@A?(?@hello
patmos-llvm-link: error:  loading file 'hello.o'

[schoeberl]

see: https://github.com/lf-lang/lingua-franca/actions/runs/12200973294/job/34038357391

[Emoun]

I see this in the log:

/home/runner/t-crest/local/bin/patmos-clang    -MD -MT CMakeFiles/cmTC_8af5b.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_8af5b.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_8af5b.dir/testCCompiler.c.o -c /home/runner/work/lingua-franca/lingua-franca/test/C/src-gen/StructParallel/build/CMakeFiles/CMakeScratch/TryCompile-c4QmtP/testCCompiler.c

I don't know what -MD -MT do and have not tested them, so no guarantee they work or do the same as the old compiler.

CMakeFiles/cmTC_8af5b.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_8af5b.dir/testCCompiler.c.o.d are these object files? If so, this would mean we are using the new compiler as a linker, which cannot be done in this way. (the old compiler was weirdly setup to work as a linker too, I have not done this for the new one). If the files are LLVM-IR object files, patmos-llvm-link must be used to link them. If they are patmos machine code, patmos-ld.lld must be used to link them.

-c /home/runner/work/lingua-franca/lingua-franca/test/C/src-gen/StructParallel/build/CMakeFiles/CMakeScratch/TryCompile-c4QmtP/testCCompiler.c:
The -c flag outputs bitcode in the old compiler, but outputs machine code in the new (as is the case for all other targets clang supports). So if LLVM-bitcode is needed, use the -emit-llvm flag in addition to -c.

In general, this command likely needs to be split in two. First the c file is compiled to LLVM-IR using patmos-clang, then (assuming the other files are already LLVM-IR), patmos-llvm-link must be used to link them together. Otherwise, they can all be compiled to machine code and pamtos-ld.lld can be used.

[Emoun]

@schoeberl In your example, the problem is patmos-clang -c hello.c produces a patmos machine code object file, which you are then trying to link with patmos-llvm-link, which expects a LLVM-IR file. Adding -emit-llvm should make it work.

[schoeberl]

Do we not have a linker of the machine code object file?

[schoeberl]

BTW, with clang, this works (as in GCC and as usual in general C compilers.)

[Emoun]

When using Clang and LLVM, there are 2 linkers: llvm-link is an LLVM-IR-level linker. ld.lld is the machine-code-linker and is the one we usually talk about when talking anout a "linker". This is the case for all targets, x86, ARM, RISCV, Patmos.

When using Clang for other targets (x86, ARM, RISCV), using the -c flag will produce a machine-code object file that must be linked with ld-lld.
Our old Patmos compiler behaved differently, making -c produce LLVM-IR object files that must be linked with llvm-link (patmos-llvm-link to be exact).
The new compiler behaves like the standard for other targets, meaning -c produces a patmos machine-code object file. This file must therefore be linked with patmos-ld.lld.

From what I can see, the LF command above was chosen for the old compiler, and thus produce and comsumed llvm-ir-object code. The new compiler does not do that and will therefore get it wrong with the same command. The command must be corrected to work with the new compiler. I think the solution is to just add the -emit-llvm flag, but might also need manual linking at the LLVM-IR level.

If some of the flags, that I don't know, do something important, we need to test if they work on the new compiler (it is not a given that they would work, so I suspect they wouldn't).

[schoeberl]

No it is not standard clang/llvm. When I use clang on may Mac (native not Patmos) it can compile to object files and link those files, both with clang. I never needed to call an "extra" linker.

[schoeberl]

The flags are not important. That comes from cmake and maybe Ehsan has copied some from other targets. cmake is a very bad choice for crosscompiling. It tries to guess the target, the OS and so forth from the host. Orginily it had even a flag with -arm64 or so...

[schoeberl]

I am having troubles both ways:

patmos-clang -c -emit-llvm -O2 hello.c
patmos-clang hello.bc 
...
13 patmos-llc               0x00000001032e4a78 llvm::legacy::PassManagerImpl::run(llvm::Module&) + 1692
14 patmos-llc               0x0000000102a31844 main + 7576
15 dyld                     0x000000018450f154 start + 2476
patmos-clang-12: error: unable to execute command: Abort trap: 6
patmos-clang-12: error: Link final executable command failed due to signal (use -v to see invocation)

The other way round also fails:

patmos-clang -c -O2 hello.c
patmos-ld.lld hello.o                 
patmos-ld.lld: error: undefined symbol: puts
>>> referenced by hello.c
>>>               hello.o:(main)

Do you have an example where compiling and linking is separate?

[Emoun]

I have investigated the behavior of the new compiler and how things can be done in GCC (don't have pure clang installed).
The new compiler is not able to compile and link at the same time like GCC (and probably pure clang) can. I will look into adding this feature next week. For now, the solution is either to created dynamic libraries and link them in using the -l flags (see the patmos repo make files), or to revert LF to using the old compiler.

[Emoun]

I have update the patmos compiler to now allow compiling and linking in the same step (as you described at the top).

I cannot release a new prebuilt of the compiler, as we are having unrelated issues that fail the CI. However, if the compiler is built locally, it should now be able to do what is needed for LF. Please test this on your machine and maybe the LF CI (assuming it doesn't use prebuilts).