Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for ImplMap entries and relationships for .NET binaries #54

Merged
merged 43 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
66ffa76
Basic temporary output that lists PInvoked module names and methods.
Czatar Sep 24, 2023
fde4bdf
Changed output format to a list of methods from each module
Czatar Sep 25, 2023
6cd0be6
Don't need the import, used setdefault instead of defaultdict
Czatar Sep 25, 2023
e9c4b6c
Merge branch 'main' into CYT-579-dotNet-ImplMap-Support
Czatar Oct 9, 2023
bb26d92
Updated output format for ImplMap
Czatar Oct 9, 2023
d0f396b
Combined import search from assemblyRef and implMap
Czatar Oct 9, 2023
1d1187e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 9, 2023
a119d2d
Merge branch 'main' into CYT-579-dotNet-ImplMap-Support
Czatar Oct 9, 2023
c0d0e64
Added sources and updated function name
Czatar Oct 14, 2023
4cf1444
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 14, 2023
3b69388
Fixed spelling errors
Czatar Oct 14, 2023
ef8b1fa
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 14, 2023
825e9b0
Separate dotnetImplMap into its own if branch
Czatar Oct 14, 2023
dfdcefd
Merge branch 'CYT-579-dotNet-ImplMap-Support' of github.com:LLNL/Surf…
Czatar Oct 14, 2023
ec8f5e4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 14, 2023
e2915bd
Added information about the search path enum
Czatar Oct 14, 2023
4872cc5
Merge branch 'main' into CYT-579-dotNet-ImplMap-Support
Czatar Oct 16, 2023
a19a9c3
Merge branch 'main' into CYT-579-dotNet-ImplMap-Support
Czatar Nov 6, 2023
e5c982a
Create function to generate list of common Program Files or System32 …
Czatar Nov 6, 2023
13980d7
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 6, 2023
82fecd2
Remove function for probing System and Program Files directories
Czatar Nov 6, 2023
54270c0
Remove function to probe system and program files directories
Czatar Nov 6, 2023
85265c8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 6, 2023
bbf999c
modified find_installed_software to support list of strings, create h…
Czatar Nov 9, 2023
b1c49d9
Merge fixes from autofixer
Czatar Nov 9, 2023
dcebb77
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 9, 2023
e67b336
Fix pe_relationships function import
Czatar Nov 9, 2023
13aa2aa
Merge autofixes
Czatar Nov 9, 2023
6509705
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 9, 2023
bbb7917
Fixed import line
Czatar Nov 9, 2023
592c4dd
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 9, 2023
02bbcf0
Check for instance of str instead of Iterable since strings are iterable
Czatar Nov 9, 2023
eee6f55
Construct list of libary name combinations to probe
Czatar Nov 12, 2023
fc5002e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 12, 2023
ee1ffb3
Removed version checking and changed absolute path checking
Czatar Nov 19, 2023
37b89e2
Merge branch 'main' into CYT-579-dotNet-ImplMap-Support
Czatar Nov 19, 2023
1d491ec
Merge branch 'CYT-579-dotNet-ImplMap-Support' of github.com:LLNL/Surf…
Czatar Nov 19, 2023
de662d9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 19, 2023
93ff3ca
Remove call to get_windows_pe_dependencies since the loop above that …
Czatar Nov 20, 2023
5a8eb82
Removed unused import line
Czatar Nov 20, 2023
2bd8e9d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 20, 2023
399ae8c
Added references to GitHub Issues in corresponding locations
Czatar Nov 20, 2023
a2828b2
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions surfactant/infoextractors/pe_file.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,11 @@ def extract_pe_info(filename):
for ar_info in assemblyref_info:
assembly_refs.append(get_assemblyref_info(ar_info))
file_details["dotnetAssemblyRef"] = assembly_refs
if implmap_info := getattr(dnet_mdtables, "ImplMap", None):
imp_modules = []
for im_info in implmap_info:
insert_implmap_info(im_info, imp_modules)
file_details["dotnetImplMap"] = imp_modules

# TODO for a custom intermediate SBOM format, the information read from the manifest and app config files
# should be tied to a specific "<install path>/<file name>", in case the same file appears in separate
Expand Down Expand Up @@ -233,6 +238,17 @@ def get_assemblyref_info(asmref_info):
return asmref


def insert_implmap_info(im_info, imp_modules):
dllName = im_info.ImportScope.row.Name
methodName = im_info.ImportName
if dllName:
for imp_module in imp_modules:
if imp_module["Name"] == dllName:
imp_module["Functions"].append(methodName)
return
imp_modules.append({"Name": dllName, "Functions": [methodName]})


def get_xmlns_and_tag(uri):
check_xmlns = re.match(r"\{.*\}", uri.tag)
xmlns = check_xmlns.group(0) if check_xmlns else ""
Expand Down
25 changes: 15 additions & 10 deletions surfactant/relationships/_internal/windows_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# SPDX-License-Identifier: MIT
import pathlib
from collections.abc import Iterable
from typing import Any, List
from typing import Any, List, Union

from surfactant.sbomtypes import SBOM, Software

Expand All @@ -15,20 +15,25 @@
# and not accidentally mix and match cultures/app config info that could differ for different copies of the same
# file (due to app config files pointing to different assemblies despite DLL having same hash)
# culture information to find the right assembly from app config file is likely to vary (though almost always neutral/none)
def find_installed_software(sbom: SBOM, probedirs: List[Any], filename: str) -> List[Software]:
def find_installed_software(
sbom: SBOM, probedirs: List[Any], filename: Union[str, List[str]]
) -> List[Software]:
possible_matches = []
# iterate through all sbom entries
for e in sbom.software:
# Skip if no install path (e.g. installer/temporary file)
if e.installPath is None:
continue
for pdir in probedirs:
# installPath contains full path+filename, so check for all combinations of probedirs+filename
pfile = pathlib.PureWindowsPath(pdir, filename)
if isinstance(e.installPath, Iterable):
for ifile in e.installPath:
# PureWindowsPath is case-insensitive for file/directory names
if pfile == pathlib.PureWindowsPath(ifile):
# matching probe directory and filename, add software to list
possible_matches.append(e)
if isinstance(filename, str):
filename = [filename]
for fname in filename:
# installPath contains full path+filename, so check for all combinations of probedirs+filename
pfile = pathlib.PureWindowsPath(pdir, fname)
if isinstance(e.installPath, Iterable):
for ifile in e.installPath:
# PureWindowsPath is case-insensitive for file/directory names
if pfile == pathlib.PureWindowsPath(ifile):
# matching probe directory and filename, add software to list
possible_matches.append(e)
return possible_matches
66 changes: 65 additions & 1 deletion surfactant/relationships/dotnet_relationship.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,63 @@ def establish_relationships(
dnProbingPaths = []
dnProbingPaths.append(pathlib.PureWindowsPath(path).as_posix())

# https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/loading-unmanaged
# 1. Check the active AssemblyLoadContext cache
# 2. Calling the import resolver set by the setDllImportResolver function
# - a. Example using SetDllImportResolver: https://learn.microsoft.com/en-us/dotnet/standard/native-interop/native-library-loading
# - b. Checks PInvoke's or Assembly's DefaultDllImportSearchPathsAttribute, then the assembly's directory, then LoadLibraryEx with LOAD_WITH_ALTERED_SEARCH_PATH flag
# - This attribute has no effect on non-Windows platforms / Mono runtime
# - https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.defaultdllimportsearchpathsattribute?view=net-7.0
# - i. This has a "Paths" property which is a bitwise combination of paths specified in ii:
# - ii. https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.dllimportsearchpath?view=net-7.0
# 3. The active AssemblyLoadContext calls its LoadUnmanagedDll function (Default behavior is the same as AssemblyRef probing?)
nightlark marked this conversation as resolved.
Show resolved Hide resolved
# - a. Can be overridden, but the default implementation returns IntPtr.Zero, which tells the runtime to load with its default policy.
# - b. https://learn.microsoft.com/en-us/dotnet/api/system.runtime.loader.assemblyloadcontext.loadunmanageddll?view=net-7.0
# 4. Run default unmanaged library probing logic by parsing *.deps.json probing properties
# - a. If the json file isn't present, assume the calling assembly's directory contains the library
# - b. https://learn.microsoft.com/en-us/dotnet/core/dependency-loading/default-probing#unmanaged-native-library-probing
if "dotnetImplMap" in metadata:
for asmRef in metadata["dotnetImplMap"]:
if "Name" not in asmRef:
continue
refName = asmRef["Name"]

# Check absolute path against entries in software
if is_absolute_path(refName):
ref_abspath = pathlib.PureWindowsPath(refName)
for e in sbom.software:
if e.installPath is None:
continue
if isinstance(e.installPath, Iterable):
for ifile in e.installPath:
if ref_abspath == pathlib.PureWindowsPath(ifile):
relationships.extend(Relationship(dependent_uuid, e.uuid, "Uses"))
continue

probedirs = []
if isinstance(software.installPath, Iterable):
for ipath in software.installPath:
probedirs.append(pathlib.PureWindowsPath(ipath).parent.as_posix())
# Construct a list of combinations specified in (2.a)
# Refer to Issue #80 - Need to verify that this conforms with cross-platform behavior
combinations = [refName]
if not (refName.endswith(".dll") or refName.endswith(".exe")):
combinations.append(f"{refName}.dll")
combinations.extend(
[
f"{refName}.so",
f"{refName}.dylib",
f"lib{refName}.so",
f"lib{refName}.dylib",
f"lib{refName}",
]
)
nightlark marked this conversation as resolved.
Show resolved Hide resolved
# On Linux, if the libname ends with .so or has .so. then version variations are tried
# Refer to Issue #79 - Need regex matching for version variations
for e in find_installed_software(sbom, probedirs, combinations):
dependency_uuid = e.UUID
relationships.append(Relationship(dependent_uuid, dependency_uuid, "Uses"))

# https://learn.microsoft.com/en-us/dotnet/framework/deployment/how-the-runtime-locates-assemblies
# 1. Determine correct assembly version using configuration files (binding redirects, code location, etc)
# 2. Check if assembly name bound before; if it is use previously loaded assembly
Expand Down Expand Up @@ -121,7 +178,9 @@ def establish_relationships(
dependency_uuid = e.UUID
relationships.append(
Relationship(
dependent_uuid, dependency_uuid, "Uses"
dependent_uuid,
dependency_uuid,
"Uses",
)
)

Expand All @@ -135,6 +194,11 @@ def establish_relationships(
return relationships


def is_absolute_path(fname: str) -> bool:
givenpath = pathlib.PureWindowsPath(fname)
return givenpath.is_absolute()


# construct a list of directories to probe for establishing dotnet relationships
def get_dotnet_probedirs(software: Software, refCulture, refName, dnProbingPaths):
probedirs = []
Expand Down