SteamOS 3.6 Requires innoextract 1.9-9

[AlfredoDinunzioTR]

System Information

• SteamTinkerLaunch version: 'v14.0.20240624-1'
• Distribution: Steam OS
• Installation Method: Manual / git clone

Issue Description

Today I updated my steam deck and innoextract issues have started up again. I have read through all the other posts regarding innoextract and I understand what it is used for and that the dependencies change as the OS/STL Update. Last time this happened, simply removing /home/deck/stl/deps/usr/bin/innoextract and running STL again resolved. This time that doesn't seem to work. If I run STL mo2 start from desktop mode , it opens and I can launch my game no problem. But running from steam/game mode causes it to try and reinstall innoextract and it fails due to missing file.

What does seam to work for running from steam, is making sure that /home/deck/stl/deps/innoextract-1.9-8-x86_64.pkg.tar.zs doesn't get removed when running STL. I made the file immutable and it resolved my issue.

Logs

before setting +i innoextract.txt

after setting +i innoextract.txt

[adinunzio10]

Let's pretend I posted that from my non-work account lol

[sonic2kk]

Thanks for bringing this up, this seems different from before and ultimately a SteamOS issue. Since this I assume works outside of the SteamOS Beta Channel I would say this is a SteamOS problem.

For what it's worth, modding games on SteamOS is a bad idea in my opinion and also I am planning to end official support for SteamOS around v14.0's release.

innoextract is only used to install ModOrganizer 2. However from your description it sounds like when SteamTinkerLaunch is launched in Game Mode, regardless of whether ModOrganizer 2 is installed or not, it tries and fails to extract innoextract? I don't understand what's going on here, sorry.

If this only affects installing ModOrganizer 2, this should be done from Desktop Mode anyway. Ideally all tinkering should be done from Desktop Mode and Game Mode should only be used to launch games, but if games can't launch, even those not using MO2, then that's pretty serious.

Any more clarification you can give on the process would be appreciated!

[adinunzio10]

Totally understand you’re reasoning for ending steam OS support. For me STL has been a crucial step that made modding on my steam deck much easier. I’ve honestly had no problems with modding or modded games. Just this abnormal steam OS behavior. I still might do a fork of STL specific to steam deck but we’ll see.

It does happen both in desktop mode and game mode when launching a game from steam that has compatibility set to STL. From desktop mode if I launch mo 2 via terminal I can launch my game no issue. It seems when launching from steam it can’t find innoextraxt but when launching via terminal it can find it. Should be able to debug it when I have some time.

Making the innoextract file immutable prevents it from being moved or deleted which completely resolves the issue for me. Might be a bandaid but like you said it shouldn’t even try to install it if it’s already installed. Just wanted to document it , if I happen to not be the only person using STL on their deck 😂

thanks again for your work on this awesome utility !

[sonic2kk]

My guess is that it's trying to install it because it isn't able to add innoextract to PATH. But I don't know why it can do this for Yad (and cabextract). I haven't had a chance to inspect the logs given yet.

Since this happens only when launching from the Steam Client, I wonder if something is going wrong with the PATH variable from Steam... There were some changes around environment variables I heard in the latest Beta. We don't manage dependencies on Desktop Mode in quite the same way so I'm not sure if there's a direct way to reproduce this behaviour on PC, but inspecting PATH across both Linux Desktop and PC from SteamTinkerLaunch itself would be interesting.(To see what STL sees, we log this I think but not sure if only on SteamOS).

So perhaps something is going wrong with PATH when launched from Steam. Does this also happen with native titles out of interest, which don't have SteamTinkerLaunch set as a compatibility tool? (This does not work from Game Mode last I tried a long time ago, #630, so we can only test in Desktop mode) - I ask because I am interested if on SteamOS Valve are now enforcing the Compatibility Tool Manifest v2 which would kill this project because it would try to run SteamTinkerLaunch in a container (and that could play havoc with setting the PATH variable). Running it as a launch option for native games could run the script directly and not as a compatibility tool and so could skip the Steam Linux Runtime, unless native games on SteamOS are always wrapped in it (unsure how the native start command looks on SteamOS).

For me STL has been a crucial step that made modding on my steam deck much easier.

I am glad it has helped! But please do keep in mind, SteamTinkerLaunch is a tinkering tool, not a tool for modding games, and the modding tools are probably just barely more supported than SteamOS heh.

I still might do a fork of STL specific to steam deck but we’ll see

You are of course welcome to, but I'd appreciate contributions here. Advertised support is ending but no code is going to be stripped out. The reason advertised support is ending is because, after over a year of asking outright for contributions (#859), and for over 2 years since Steam Deck support was first introduced, there have been very few contributions and only two people have made any significant contributions (excluding myself before I took over as maintainer).

By all means you can create a fork, there will be zero bad blood here, but just letting you know I am receptive to contributions if you are open to merging back upstream here!

Just wanted to document it , if I happen to not be the only person using STL on their deck

You are the only person so far to report such an issue, and given that most Deck users using SteamTinkerLaunch are likely running Beta or Preview, I am interested to see how widespread this is.

---

A very mind contributor was working on Steam Deck dependency enhancement. I have flagged this issue to them in that PR, because this could get in the way of automatic dependency updates when running from Steam.

[sonic2kk]

The log before making innoextract immutable is very strange indeed, and very short. There is no issue in the second log with setting PATH luckily.

The log lines of interest are:

Tue Jul  2 23:56:10 EDT 2024 INFO - installDependencyVersionFromURL - Downloading innoextract version automatically from URL 'https://archive.archlinux.org/packages/i/innoextract/innoextract-1.9-8-x86_64.pkg.tar.zst'
Tue Jul  2 23:56:11 EDT 2024 INFO - installDependencyVersionFromURL - curl -Lq "https://archive.archlinux.org/packages/i/innoextract/innoextract-1.9-8-x86_64.pkg.tar.zst" -o "/home/deck/stl/deps/innoextract-1.9-8-x86_64.pkg.tar.zst"
Tue Jul  2 23:56:11 EDT 2024 INFO - fetchAndExtractDependency - Installing 'innoextract-1.9-8-x86_64.pkg.tar.zst' from 'https://archive.archlinux.org/packages/i/innoextract/innoextract-1.9-8-x86_64.pkg.tar.zst' to installation directory '/home/deck/stl/deps'
Tue Jul  2 23:56:11 EDT 2024 WARN - fetchAndExtractDependency - Failed to download dependency 'innoextract-1.9-8-x86_64.pkg.tar.zst', will attempt to continue with installation in case dependency archives already exist at '/home/deck/stl/deps'
Tue Jul  2 23:56:11 EDT 2024 INFO - fetchAndExtractDependency - Could not find any matching dependency archives in '/home/deck/stl/deps' for 'innoextract-1.9-8-x86_64.pkg.tar.zst' - Assuming this dependency is missing and installation cannot continue

It would only do this if it cannot find the binary in the first place. I'll try explain the steps so me, you, and aliens millennia in the future reading this are on the same page:

• (Relevant Line) Unconditionally when running SteamTinkerLaunch, we run this to make sure innoextract is installed from a given URL: installDependencyVersionFromURL "$INNOEXTRACT" "$INNOEXTRACTFILE" "$STLDEPS" "$INNOEXTRACTURL" || steamDeckInstallFail
• (Relevant Lines) The installDependencyVersionFromURL function, before downloading anything, first checks if innoextract exists and if it can be executed, in an attempt to make sure innoextract is valid.
  • In your "before" log, STL logs out that it tries to download and install innoextract, meaning that it falls into the else block here, suggesting that this check has for whatever reason failed: https://github.com/sonic2kk/steamtinkerlaunch/blob/144347b1c1b322bda8d3cdc5559ad284a9f8578c/steamtinkerlaunch#L26290-L26300
  • Whereas in your "after" log, STL can actually find that innoextract is installed and it correctly falls into the if block that makes sure innoextract is successfully installed: https://github.com/sonic2kk/steamtinkerlaunch/blob/144347b1c1b322bda8d3cdc5559ad284a9f8578c/steamtinkerlaunch#L26287-L26289

So for some reason, SteamTinkerLaunch when launched from Steam on SteamOS cannot see this file. The reason it doesn't complain about cabextract and yad is because innoextract is checked first, you can see the order dependencies are checked on these lines. If it can't install innoextract then it fails (in hindsight, it probably shouldn't take innoextract as a hard dependency, but eh).

---

There are two things I suggest trying:

• Removing the innoextract archive from ~/stl/deps and the binary from ~/stl/deps/usr/bin. Then, keeping an eye on the deps folder, run SteamTinkerLaunch from Steam with a game with STL selected as a compatibility tool (i.e.run Fallout 4 with STL). Check to see if innoextract gets extracted at all into the deps folder.
  • If it does, try inspecting its size to see if it looks sane (approx 570kb). You can also try opening a terminal in this folder (Shift+F4, or right click -> Open Terminal Here) and running ./innoextract --version and seeing if it returns anything. It should return something like innoextract 1.9 Extracts installers blah blah.
  • If you get a libboost related error, then we need to update innoextract to 1.9.9, see below.
  • If the archive does not get extracted, then that is very interesting, and it would be interesting then to see if it can be extracted when running SteamTinkerLaunch with a native game. You can try removing the archive and binary as above when testing a native game, and also if you'd like you can then just try running a native game. Running games with Proton from Steam do not work, so this would let us know if the issue is specific to using SteamTinkerLaunch as a compatibility tool or if it is something to do with the Steam environment.
    • If Steam is running STL as a compat tool inside the Steam Linux Runtime container then it probably wouldn't be able to extract the archive and so would fail (possibly permissions, issues running the extract command, etc).
    • Whereas if it's running as a launch option for native games it wouldn't have this restriction and should be able to extract properly.
• If you have a keyboard and mouse you can hook up, you could try closing Steam and then running it again from a Terminal client with steam to see the output when running a game. When running a game, inspect the output. I won't ask you to attach the output here as I am not sure what potentially sensitive information a Steam log might have, but you could take a scan for anything that seems out-of-place, such as tar: command not found. or something to this effect. You might see some "integer expression expected" complaints coming from the STL script but these are probably nothing to worry about in this context.
  • The reason you need a keyboard is because when you close Steam, the virtual keyboard disappears with it, so you can't enter the steam command very easily. You could type it beforehand and find some way to send a "Return" key signal to the terminal so it would run the command, but a keyboard is probably more straightforward :-)

Below are some steps to try if you get a libboost-related error message.

• If you get a libboost error, try checking if innoextract 1.9.9 works. You can download it manually from the Arch mirrors (not a download link, and feel free to verify the sums if you'd like) and extract it into ~/stl/deps/usr/bin.
  • SteamOS 3.6 Preview a long time back updated the SteamOS Arch snapshot to a newer version. When this happens we usually have to update the innoextract version because one of its dependencies, libboost, updates as part of the SteamOS image and so innoextract no longer works with that newer library version.
    • Similarly, bumping to a newer version of innoextract prematurely would break compatibility with older SteamOS versions. So I figured I would wait for someone to report the libboost issue if it came up again. SteamOS 3.6 has been in Preview for a long time now so I figured with no reports it wasn't a problem.
  • I misread your reply earlier, you said last time removing the files just worked, but I assumed you had meant you did this to manually update innoextract. I also did not think this was relevant before because this issue only affects STL when running from Steam, and works fine outside of Steam. This does not sound immediately like it would be a problem with libboost as historically it would be, but the new dependency validation would fail if there was a libboost issue.

Personally, I don't think this is a libboost issue. I think innoextract is unable to be extracted properly when ran from Steam and that is the part we are failing on. I think if it was libboost we would see failures when trying to run STL regardless.

---

Something else of note from your "before" log is that it's failing to download innoextract as well as not being able to find the binary, which is very strange. And it seems to fail very fast, all within seconds, so I don't think it's a timeout. Odd for sure!

[adinunzio10]

Personally, I don't think this is a libboost issue. I think innoextract is unable to be extracted properly when ran from Steam and that is the part we are failing on. I think if it was libboost we would see failures when trying to run STL regardless.

Nope I believe you were right. It is a libboost issue. You mentioned 'installDependencyVersionFromURL' will try to run innoextract to see if it works and if not , it determines it is not installed. The binary was in the location already. If I try to execute it manually like /home/deck/stl/deps/usr/bin/innoextract --version I get this: error while loading shared libraries: libboost_iostreams.so.1.81.0: cannot open shared object file: No such file or directory It also started after an update to Steam OS.

If I run find /usr/lib/ -name "libboost_iostreams.so.*" I get: /usr/lib/libboost_iostreams.so.1.83.0

I am glad it has helped! But please do keep in mind, SteamTinkerLaunch is a tinkering tool, not a tool for modding games, and the modding tools are probably just barely more supported than SteamOS heh.

Yeah totally makes sense. I know there are other ways to install Mod Organizer 2 on steam deck. A while back when I was setting up Fallout TTW, I was struggling hard to get MO2 setup until I found STL and it just worked. So it has been my sole way to setup and launch MO2. So you are correct I am mainly using STL as a tool to make it easier to install & access MO2. From there it's just modding and thankfully Steam OS has not caused me any issues with modding. I later used it to play Viva new Vegas and now the midnight ride (fallout 4).

You are of course welcome to, but I'd appreciate contributions here. Advertised support is ending but no code is going to be stripped out. The reason advertised support is ending is because, after over a year of asking outright for contributions (#859), and for over 2 years since Steam Deck support was first introduced, there have been very few contributions and only two people have made any significant contributions (excluding myself before I took over as maintainer).
By all means you can create a fork, there will be zero bad blood here, but just letting you know I am receptive to contributions if you are open to merging back upstream here!

Awesome! I did jump to conclusions and think you meant dropping support, you were yanking it out lol so I would do a fork to at least keep it going for me. Totally makes sense you are going to keep it in there just not advertise it as supported. Meaning when issues arise, I can try to resolve it for myself via a PR so others can also take advantage.

You are the only person so far to report such an issue, and given that most Deck users using SteamTinkerLaunch are likely running Beta or Preview, I am interested to see how widespread this is.

The issue did start immediately after I updated Steam OS which seems to go with the other details you shared regarding innoextract.

A very mind contributor was working on Steam Deck dependency enhancement. I have flagged this issue to them in that PR, because this could get in the way of automatic dependency updates when running from Steam.

I did notice that and while I did not look into exactly what changes were made in his version, using it did not resolve my issue.

[sonic2kk]

Thanks for checking that! That is rather surprising. I don't know why this only happens from Steam.

You can try manually updating innoextract for now. If you manually place the 1.9.9 binary in the ~/stl/deps/usr/bin folder then it should be resolved. We can't update the innoextract version yet because if we do it'll stop working on SteamOS Beta and Stable since 3.6 is only in Preview. But I can make a branch tomorrow that bumps the version.

I'm not sure why this only affects things when running from the Steam Client, I guess the dependency check triggers differently if we aren't launching a game.

This presents an interesting problem. The dependency validation is relatively new, but if the dependency doesn't work on the current system, it'll fail to install SteamTinkerLaunch. I think after #1111 we should add better logging for why the dependency failed. For example here we should check for and log if we get a libboost error.

Thanks for the testing. I think we've gotten to the bottom of this then:

• innoextract needs bumped as SteamOS 3.6 Preview updates the snapshot base. This can be done manually for now by updating the innoextract binary manually.
• The install fails because of the innoextract failure, because we can't run --version against the binary and get a success code and assume the dependency needs to be redownloaded which is not correct, we need to replace this with better handling.
• For some reason this failover only happens when running from Steam, I guess the dependency checking is different when running outside of Steam for some reason...? Not too sure to be honest.
• The dependency won't be checked at all after Steam Deck dependency enhancement #1111 if the SteamTinkerLaunch version did not change which will help this as well.

I'll update the issue name if you don't mind. I'll push the updated dependency version tomorrow to a new branch and open a draft PR in preparation for SteamOS 3.6. This was a great help in troubleshooting I am just again quite surprised it didn't come up before now 😄

[sonic2kk]

I guess this issue can also be closed since this is a known and common thing we have to do, but we didn't know until now that it was the same issue that innoextract just needed bumped to accommodate SteamOS 3.6.

[adinunzio10]

Thanks for checking that! That is rather surprising. I don't know why this only happens from Steam.

You can try manually updating libboost for now. If you manually place the 1.9.9 binary in the ~/stl/deps/usr/bin folder then it should be resolved. We can't update the innoextract version yet because if we do it'll stop working on SteamOS Beta and Stable since 3.6 is only in Preview. But I can make a branch tomorrow that bumps the version.

Confirmed that replacing the innoextract in ~/stl/deps/usr/bin with the 1.9.9 binary resolves the issue in the proper way.

I'm not sure why this only affects things when running from the Steam Client, I guess the dependency check triggers differently if we aren't launching a game.

This presents an interesting problem. The dependency validation is relatively new, but if the dependency doesn't work on the current system, it'll fail to install SteamTinkerLaunch. I think after #1111 we should add better logging for why the dependency failed. For example here we should check for and log if we get a libboost error.

Yeah I was thinking about that too. The dependency check does seem to be handled different when launched via a game in steam compat mode. Vs just running stl mo2 s which seemingly is not checking that innoextract works before moving on.

Thanks for the testing. I think we've gotten to the bottom of this then:

• innoextract needs bumped as SteamOS 3.6 Preview updates the snapshot base. This can be done manually for now by updating the innoextract binary manually.
• The install fails because of the innoextract failure, because we can't run --version against the binary and get a success code and assume the dependency needs to be redownloaded which is not correct, we need to replace this with better handling.
• For some reason this failover only happens when running from Steam, I guess the dependency checking is different when running outside of Steam for some reason...? Not too sure to be honest.
• The dependency won't be checked at all after Steam Deck dependency enhancement #1111 if the SteamTinkerLaunch version did not change which will help this as well.

I'll update the issue name if you don't mind. I'll push the updated dependency version tomorrow to a new branch and open a draft PR in preparation for SteamOS 3.6. This was a great help in troubleshooting I am just again quite surprised it didn't come up before now 😄

Thanks for helping me clear all of this up and I am always glad to do my best to help.

[sonic2kk]

Oops I didn't see your other responses, my bad.

Yeah totally makes sense. I know there are other ways to install Mod Organizer 2 on steam deck. A while back when I was setting up Fallout TTW, I was struggling hard to get MO2 setup until I found STL and it just worked.

I'm glad it works for you, the reason I am cautious of modding or really any tinkering on SteamOS is because I know how brittle it is if you do anything beyond installing games and Flatpaks. I'm not here to tell people how to use their devices I guess the thing I prefer to convey is "you can do this but be aware it's fragile."

You seem to know your way around, there have been a few people coming to me for modding support with the Steam Deck as their first PC. So I tend to give a blanket disclaimer 😅 It keeps the insults I get to a minimum and helps me sleep at night.

Awesome! I did jump to conclusions and think you meant dropping support, you were yanking it out lol so I would do a fork to at least keep it going for me.

Any fork-specific / local-specific changes are totally fine! This project is open source so anything you want to do on a fork you're welcome to. Applying quick-fixes for your own use is no problem and/or if you feel like discussing or opening a PR for longer term fixes upstream here that's fine too!

Basically, if you want to do your own thing with a fork I fully encourage it! I write this project to have fun so others that fork it or contribute back should too :-)

If there ever is anything you feel like upstreaming I'll gladly review any PR.

Confirmed that replacing the innoextract in ~/stl/deps/usr/bin with the 1.9.9 binary resolves the issue in the proper way.

Awesome! I'll get the branch up tomorrow. I'll try rebase it regularly against master so anyone relying on it won't be too far behind.

. The dependency check does seem to be handled different when launched via a game in steam compat mode. Vs just running stl mo2 s

Ah good point, plus we don't need innoextract at this point so it makes sense. It's only needed to install MO2 initially.

Innoextract is optional so we should probably treat it that way by removing the fail part for that dependency installation.

---

Apologies that the fix for this can't go into master just yet. It's purely in the interest of compatibility with SteamOS versions that aren't 3.6. I double-checked and I don't think 3.6 is in Stable yet. Once it is though I'll bump the innoextract version on master. I hope the reasoning makes sense.