Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated BuildCompat.py #3199

Draft
wants to merge 1 commit into
base: Development
Choose a base branch
from
Draft

Updated BuildCompat.py #3199

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 35 additions & 25 deletions BuildCompat.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
#!/usr/bin/python3
import os
from twisted.python.filepath import FilePath
from subprocess import Popen
import re
import sys
import asyncio
from pathlib import Path

tm = '-m' in sys.argv

parallel = '-j' in sys.argv

PROJECT_PATTERN = re.compile(r'''Project.".[0-9A-Za-z]{8}-[0-9A-Za-z]{4}-[0-9A-Za-z]{4}-[0-9A-Za-z]{4}-[0-9A-Za-z]{12}.". = '''
Expand All @@ -16,30 +15,41 @@

PUBLICIZER = os.environ.get("PUBLICIZER", "./AssemblyPublicizer")
DOWNLOAD_LIBS = os.environ.get("DOWNLOAD_LIBS", "--download-libs")
tasks = []

def system(*cmd):
sp = Popen(cmd)
if not parallel:
sp.wait()
ec = sp.poll()
if ec:
raise SystemExit(ec)
else:
tasks.append(sp)

with open("Source/CombatExtended.sln") as f:
for line in f.readlines():
line = line.strip()

async def run_command(*cmd):
proc = await asyncio.create_subprocess_exec(*cmd)
await proc.wait()
if proc.returncode != 0:
raise SystemExit(proc.returncode)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This exits non-zero even in parallel build mode. That is probably a good thing as it would let us use parallel mode in the github action and not incorrectly ship a broken build. However it leaves the other tasks hanging. They then spew their standard out and standard error messages after the main script terminates.

The return values from run_command are accumulated by asyncio.gather, so run_command should simply return the failure code, and its caller should check that error code and handle it. This could be done via try/except, assuming asyncio can be told to wait on all even after the first exception is thrown, but that does not appear to be the default from asyncio.gather. Printing a summary (passed / total) at the end is probably the correct solution.


async def main():
tasks = []

async def process_line(line):
match = PROJECT_PATTERN.match(line)
if match:
name, csproj = match.groups()
if tm and name not in sys.argv: continue
csproj = csproj.replace('\\', '/').split('/')
csproj = FilePath("Source").descendant(csproj)
if tm and name not in sys.argv:
return
csproj_path = Path("Source").joinpath(*csproj.split('/')).as_posix()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is worth noting that joinpath does not sanitize input, so this does not enforce that csproj_path is a child of "Source". It does not appear that pathlib has a way to enforce that constraint. In this case, I don't see an obvious attack vector exposed by relaxing this constraint, and FilePath is the only thing we're using out of tiwsted, so removing it makes the code more portable (as pathlib is part of the standard library).

print(f"Building {name}")
output = FilePath("AssembliesCompat").child(name+".dll")
system("python3", "Make.py", "--csproj", csproj.path, "--output", output.path, DOWNLOAD_LIBS, "--all-libs", "--publicizer", PUBLICIZER, "--", "-r:Assemblies/CombatExtended.dll")
output_path = Path("AssembliesCompat").joinpath(f"{name}.dll").as_posix()
cmd = [
"python3", "Make.py", "--csproj", csproj_path, "--output", output_path,
DOWNLOAD_LIBS, "--all-libs", "--publicizer", PUBLICIZER, "--",
"-r:Assemblies/CombatExtended.dll"
]
if parallel:
tasks.append(run_command(*cmd))
else:
await run_command(*cmd)

with open("Source/CombatExtended.sln") as f:
for line in f:
await process_line(line.strip())

if parallel:
await asyncio.gather(*tasks)

for t in tasks:
t.wait()
if __name__ == "__main__":
asyncio.run(main())
Loading