Building on Fedora Core 25 x64

[frno7]

First of all, many thanks for putting this together so far! I tried to build a live Linux image using ./build-all.sh from scratch and had to make a few minor changes:

There’s a problem with the upstream hash for buildroot/package/iperf/iperf.hash, as explained here.

The error cp: cannot create regular file '../usb-stick/boot-overlay/BOOT/buildrootconfig.txt': No such file or directory seems to be solved after a manual mkdir usb-stick/boot-overlay/BOOT. Similarly, cp: cannot create regular file '../linux/firmware/ps2/ps2dev9.irx': No such file or directory Makefile:29: recipe for target '../linux/firmware/ps2/ps2dev9.irx' failed seems to need a manual mkdir linux/firmware/ps2.

kernelloader/png2rgb/png2rgb.c needs some simple updates (could be improved) to compile with a newer PNG library:

cc -g -Wall -Werror-implicit-function-declaration -c -o png2rgb.o png2rgb.c
png2rgb.c: In function ‘loadPng’:
png2rgb.c:48:37: error: ‘png_infopp_NULL’ undeclared (first use in this function)
   png_destroy_read_struct(&png_ptr, png_infopp_NULL, png_infopp_NULL);
                                     ^~~~~~~~~~~~~~~
png2rgb.c:48:37: note: each undeclared identifier is reported only once for each function it appears in
png2rgb.c:72:52: error: ‘png_voidp_NULL’ undeclared (first use in this function)
   PNG_TRANSFORM_SWAP_ENDIAN /* png_transforms */ , png_voidp_NULL);
                                                    ^~~~~~~~~~~~~~
png2rgb.c:75:59: error: dereferencing pointer to incomplete type ‘png_info {aka struct png_info_def}’
  printf("Width: %d Height: %d Depth: %d\n", (int) info_ptr->width,
                                                           ^~
png2rgb.c:103:2: error: implicit declaration of function ‘memset’ [-Werror=implicit-function-declaration]
  memset(dest, 0, info_ptr->width * info_ptr->height * *depth);
  ^~~~~~
png2rgb.c:103:2: warning: incompatible implicit declaration of built-in function ‘memset’
png2rgb.c:103:2: note: include ‘<string.h>’ or provide a declaration of ‘memset’
cc1: some warnings being treated as errors
Makefile:14: recipe for target 'png2rgb.o' failed
make[1]: *** [png2rgb.o] Error 1

Then I had the error

Your branch is up-to-date with 'origin/ps2-v2.6.35.14'.
/bin/sh: mips64r5900el-linux-gnu-gcc: command not found
/bin/sh: mips64r5900el-linux-gnu-gcc: command not found
make: mips64r5900el-linux-gnu-gcc: Command not found

and so I tried to use the compiler in buildroot/output/host/usr/bin instead, via a few symlinks and setting TOOLCHAIN=mipsel in config/config.cfg (I’m not sure this actually works in practice). Next,

Can't use 'defined(@array)' (Maybe you should just omit the defined()?) at kernel/timeconst.pl line 373.
/home/noring/Dokument/linux-dev/linux/kernel/Makefile:138: recipe for target 'kernel/timeconst.h' failed
make[1]: *** [kernel/timeconst.h] Error 255
Makefile:888: recipe for target 'kernel' failed
make: *** [kernel] Error 2
make: *** Waiting for unfinished jobs....

seems to be a problem fixed here. Then the error

  LD      vmlinux
  SYSMAP  System.map
  SYSMAP  .tmp_System.map
Inconsistent kallsyms data
Try setting CONFIG_KALLSYMS_EXTRA_PASS
Makefile:863: recipe for target 'vmlinux' failed
make: *** [vmlinux] Error 1

came up, but it disappeared after running ./build-all.sh again, which completed the build.

I’d like to avoid the final step of formatting the entire USB stick, and instead create a boot file in the existing USB file system. I’ll look into the ./create script and see if I can figure out how to change that.

[frno7]

Linux v2.6.35.14 booted properly on the PS2, to my surprise. 😄 I borrowed initrd.gz from the BlackRhino Linux Distribution just to start with something and avoid formatting the USB. I’m considering updating to Linux v4.12.

[rickgaiser]

buildroot
I think the iperf problem can be quickly solved by rebasing buildroot to a newer version (assuming there is one), or cherry picking the commit you pointed out. I'll see what I can do.

linux-dev
The missing folders can be improved. Can you fix this and do a pull-request?

The compilers are a bit of a mess. There are too many compilers needed right now:

• buildroot builds its own compiler (v6.3.0)
• ./build-toolchain.sh builds the same compiler (v6.3.0) for linux
• ps2dsdk needs v3.2.3 (for compiling IOP modules)

But it seems the compiler built by ./build-toolchain.sh was not found in your path. Using the buildroot compiler is perhaps a much better idea. Do you think you can somehow get linux-dev scripts to use the buildroot compilers?
The toolchain folder and scripts can stay for toolchain development but they will then no longer be needed for compiling the kernel. It can then also be removed form ./build-all.sh. Nice work!

The usb formatting is a personal choise I guess. For more rapid development cycles I suggest loading the kernel from HTTP and copying the kernel to /var/www. And changing the kernelloader config.

buildroot
Thank you for the patch, I don't know what version of libpng I have, so I'll have to test compatibility first. Ideally I would like to remove all the conversion tools and just have buildroot load the png files.

linux
Never had the timeconst problem, but I'm using ubuntu. We can cherry pick the commit, thanks for tracking it down.

Linux v2.6.35.14 booted properly on the PS2, to my surprise. 😄 I borrowed initrd.gz from the BlackRhino Linux Distribution just to start with something and avoid formatting the USB. I’m considering updating to Linux v4.12.

Congratulations adn good luck!

[frno7]

Thanks! I will need to go through all changes eventually. I squashed all commits on your ps2-v2.6.35.14 branch and sorted it out for 4.12. Most of the issues seemed straight forward to fix, but it’s easy to make mistakes of course. I now have a 4.12 kernel that compiles in a PS2 configuration. However, it doesn’t boot yet, as it stops printing here:

Started loader
Load kernel......
Stack 0x01ffebf0
rom file TGE/sbios_old.elf at 0x01087fc0 size 0x0000ad48
entry is 0x80001000
VAddr: 0x80001000 PAddr: 0x80001000 Offset 0x00001000 Size 0x00009b28
VAddr: 0x80001000 PAddr: 0x80001000 Offset 0x00001000 Size 0x00open name mass0:/boot/vmlinux-v4.12.gz flag 1 data 41378
open fd = 3
ma/boot/vmlinux-v4.12.gz size 6196028
entry is 0x804e2750
VAddr: 0x80010000 PAddr: 0x80010000 Offset 0x00001000 Size 0x0e771
10240000 bytes for initrd available.
mass0:boot/initrd.gz size 724847
initrd_start 0x8063c000 0x000b0f6f
Try to reboot IOP.
Get Reboot Request From EE
PS2HTTP: dummy function called

Your 2.6.35.14 kernel (that works) doesn’t print more than this either, so I suspect there is some patches that need to be applied to get the kernel to continue printing via the network console (ps2link). The frame buffer isn’t initialised so I believe it hangs somewhere very early, but I don’t know where.

[frno7]

I’ve pushed v4.12 for PS2. It’s on the ps2-v4.12-squashed branch and should compile but is not quite functional since it immediately hangs. I’ve used the ps2_defconfig as .config which certainly needs improvements.

[rickgaiser]

De debug messages stop as soon as the IOP drivers are loaded. I believe you can still get debug messages if you disable the IOP modules, but I'm not sure becouse I don't use ps2link for debugging.

For kernel development the best debugging is a serial cable (requires some soldering and a usb-serial converter). I use it all the time, you can also login over the serial cable. Search psx-scene for instructions on how to make such a cable.

But nice work already! When I have some time I'll see if I can build your kernel and get some debug messages.

[frno7]

Thanks Rick, that would be great! I’m somewhat stuck without effective ability to debug at the moment.

It seems the shared memory printk for PS2 link patch ported from kernelloader at the top of my branch is causing crashes as well because when I put an iop_prints early in ps2_board_setup not even Get Reboot Request From EE prints. I’ll revert that patch from my branch now. It might be ps2sif_init that hangs, but I’m unsure about it.

[frno7]

Some obvious files related to firmware and frame buffer were missing so I’ve updated those now. Some drivers were lost too, will look into them next.

[frno7]

I believe all important drivers compile in the latest push to ps2-v4.12-squashed. Still no progress on getting the frame buffer to initialise though.

[rickgaiser]

I tried your kernel but it doesn't compile:

  CC      arch/mips/kernel/asm-offsets.s
In file included from ./arch/mips/include/asm/thread_info.h:15:0,
                 from ./include/linux/thread_info.h:37,
                 from ./include/asm-generic/current.h:4,
                 from ./arch/mips/include/generated/asm/current.h:1,
                 from ./include/linux/sched.h:11,
                 from arch/mips/kernel/asm-offsets.c:13:
./arch/mips/include/asm/processor.h:265:3: error: unknown type name 'r5900_reg_t'
   r5900_reg_t reg16;
   ^~~~~~~~~~~
./arch/mips/include/asm/processor.h:266:3: error: unknown type name 'r5900_reg_t'
   r5900_reg_t reg17, reg18, reg19, reg20, reg21, reg22, reg23;
   ^~~~~~~~~~~
./arch/mips/include/asm/processor.h:267:3: error: unknown type name 'r5900_reg_t'
   r5900_reg_t reg29, reg30, reg31;
   ^~~~~~~~~~~
In file included from ./include/linux/irq.h:27:0,
                 from ./include/asm-generic/hardirq.h:12,
                 from ./arch/mips/include/asm/hardirq.h:16,
                 from ./include/linux/hardirq.h:8,
                 from ./include/linux/memcontrol.h:24,
                 from ./include/linux/swap.h:8,
                 from ./include/linux/suspend.h:4,
                 from arch/mips/kernel/asm-offsets.c:16:
./arch/mips/include/asm/ptrace.h: In function 'kernel_stack_pointer':
./arch/mips/include/asm/ptrace.h:66:19: error: incompatible types when returning type 'r5900_reg_t {aka struct <anonymous>}' but 'long unsigned int' was expected
  return regs->regs[31];
         ~~~~~~~~~~^~~~
./arch/mips/include/asm/ptrace.h: In function 'is_syscall_success':
./arch/mips/include/asm/ptrace.h:160:9: error: wrong type argument to unary exclamation mark
  return !regs->regs[7];
         ^
./arch/mips/include/asm/ptrace.h: In function 'regs_return_value':
./arch/mips/include/asm/ptrace.h:166:20: error: incompatible types when returning type 'r5900_reg_t {aka struct <anonymous>}' but 'long int' was expected
   return regs->regs[2];
          ~~~~~~~~~~^~~
./arch/mips/include/asm/ptrace.h:168:10: error: wrong type argument to unary minus
   return -regs->regs[2];
          ^
./arch/mips/include/asm/ptrace.h: In function 'user_stack_pointer':
./arch/mips/include/asm/ptrace.h:195:19: error: incompatible types when returning type 'r5900_reg_t {aka struct <anonymous>}' but 'long unsigned int' was expected
  return regs->regs[29];
         ~~~~~~~~~~^~~~
./arch/mips/include/asm/ptrace.h: In function 'user_stack_pointer_set':
./arch/mips/include/asm/ptrace.h:201:17: error: incompatible types when assigning to type 'r5900_reg_t {aka struct <anonymous>}' from type 'long unsigned int'
  regs->regs[29] = val;
                 ^

What config are you using?

[frno7]

Thanks! I’m using arch/mips/configs/ps2_defconfig which I try to keep current on the branch. I can try make clean with it and see if it still works for me.

I’m trying to get http work with kernelloader, but it has issues and mostly hangs (tcpdump shows traffic, sometimes with incorrect network settings unless “reloaded” in the Advanced Menu in kernelloader...). Python SimpleHTTPServer might be too simple for it as well, and perhaps contribute to the crashes because when accepting connections, only a few kilobytes is transferred before it hangs, perhaps some part of the http protocol isn’t implemented properly.

I’m considering ways to proceed, for example provisionally removing CONFIG_R5900_128BIT_SUPPORT which seems to complicate a lot of things. Some drivers could also be disabled to start with.

[frno7]

A make clean rebuild worked for me. I’m basically using make ARCH=mips CROSS_COMPILE=mipsel-linux- with some light editing of your build-linux.sh script. The compiler lives in linux-dev/buildroot/output/host/usr/bin/mipsel-linux-*.

% mipsel-linux-gcc --version
mipsel-linux-gcc.br_real (Buildroot 2017.02-00008-g0de152b) 6.3.0

[frno7]

Now kernel and initrd loading via http work. 😄 Apparently one must do “Save Config on MC0” in kernelloader once, and then restart (a couple of times?). Not having a saved config on MC0 seems to be buggy. I briefly replaced SimpleHTTPServer with Nginx but then ps2link flooded with:

IOP: sifSendCmd c failed
IOP: sifSendCmd c failed
IOP: sifSendCmd c failed
...

[rickgaiser]

The defconfig does build. I had the serial driver added to it (you just committed the driver), but there are zero messages from Linux over the serial port. Note that you see a little more messages from the IOP before it crashes.

Status: Setup iopaddr
Status: Buffer check
Status: Stop RPC
Status: Copying files and start...
Kernel mode print
Stack 0x81ffebf0
CXD9611 (Expansion Bay type) initialized.
Driver loaded.
Copy kernel
entry is 0x804e85f0
VAddr: 0x80010000 PAddr: 0x80010000 Offset 0x00001000 Size 0x005efba4
First bytes 0x00 0x00
entry is 0x804e85f0
VAddr: 0x80010000 PAddr: 0x80010000 Offset 0x00001000 Size 0x005efba4
First bytes 0x00 0x00
Code verified
Patched sbios_iopaddr 0x00019600
Flush TLBs (printf will not work after this).
TLBs flushed.
Jump to kernel!

[frno7]

Parts of % git di v2.6.35.14..ps2-v2.6.35.14 arch/mips/mm are missing. Not much code but nontrivial. This command essentially makes a to-do of files to be updated (neglecting that some things moved to other locations during the years):

% diff -u <(git di --name-status v2.6.35.14..ps2-v2.6.35.14) <(git di --name-status v4.12..ps2-v4.12-squashed) | less

I’d guess these are of interest:

-M arch/mips/mm/Makefile
 M arch/mips/mm/c-r4k.c
-M arch/mips/mm/cache.c
-M arch/mips/mm/cex-gen.S
-M arch/mips/mm/fault.c
-A arch/mips/mm/page-funcs.S
-M arch/mips/mm/page.c
-M arch/mips/mm/tlbex-fault.S
 M arch/mips/mm/tlbex.c
+M arch/mips/mm/uasm-micromips.c
 M arch/mips/mm/uasm.c

[frno7]

I backtracked a bit and now have a compiling and booting ps2-v3.8-squashed branch with a minimal config. Proceeding to v3.9 conflicts with

        both modified:   arch/mips/include/asm/cacheops.h
        both modified:   arch/mips/include/asm/mipsregs.h
        both modified:   arch/mips/include/asm/processor.h
        both modified:   arch/mips/include/asm/stackframe.h
        both modified:   arch/mips/kernel/genex.S
        both modified:   arch/mips/kernel/process.c
        both modified:   arch/mips/kernel/scall32-o32.S
        both modified:   arch/mips/kernel/signal.c
        both modified:   arch/mips/kernel/syscall.c
        both modified:   arch/mips/kernel/traps.c
        both modified:   arch/mips/lib/memcpy.S
        both modified:   arch/mips/lib/memset.S

and some simpler files. It will take a few moments to resolve this carefully. 😄

[rickgaiser]

Yes, I got stuck at 3.8 for a reason. I was actually hoping to reach 3.10 becouse of the long term support. Rebasing one kernel version at a time is probably a better way to see what's changed, and fix it before continuing.

[frno7]

It turns out most conflicts to v3.9 are whitespace. However, there is a kernel panic during boot (a memory fault). A Git bisect reveals that this happens in exactly v3.9-rc1 commit 64b3122 which is actually a very small change:

commit 64b3122df48b81a40366a11f299ab819138c96e8
Author: Al Viro <[email protected]>
Date:   Thu Dec 27 11:52:32 2012 -0500

    mips: take the "zero newsp means inherit the parent's one" to copy_thread()
    
    Signed-off-by: Al Viro <[email protected]>

[frno7]

I have v3.9, v3.10 and also v4.2 compiling and booting to Busybox. The kernels are not entirely stable, as there are occasional freezes. The R5900 has hardware bugs that require NOPs in special places and some of those might be missing.

[citronalco]

I've had some trouble compiling the toolchain, too. Both binutils and gcc have some not-so-clear dependencies, where the configure script does not complain but compiliation breaks with misleading errors.
If I remember correctly I had to install bison for binutils and libicl-dev for gcc.

Edit: No, I don't... Building toolchain's binutils still fails with

ibtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../../../binutils/opcodes -I. -I../../../binutils/opcodes -I../bfd -I../../../binutils/opcodes/../include -I../../../binutils/opcodes/../bfd -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT micromips-opc.lo -MD -MP -MF .deps/micromips-opc.Tpo -c ../../../binutils/opcodes/micromips-opc.c  -fPIC -DPIC -o .libs/micromips-opc.o
../../../binutils/opcodes/mips-dis.c:154:38: error: ‘mips_cp0sel_names_mipsr5900’ defined but not used [-Werror=unused-const-variable=]
 static const struct mips_cp0sel_name mips_cp0sel_names_mipsr5900[] =
                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~

But even after commenting out this struct binutils won't compile - I guess my GCC 6.3 is too new for binutils 2.24.
Rick, I've seen you have some newer binutils versions as branches in your repository. Would it be worth trying to use one of them as drop in replacement for the old 2.24?