purl2dwarf: Get DWARFs from a PURL

[pombredanne]

Create a PURL a service backed by new SCIO analysis pipelines that takes a PURL for a native binary package as an input, and returns the DWARF debug symbol compilation unit paths when available. This will specifically be worked out for Debian native binary packages.

[armijnhemel]

As a package can contain multiple binaries I guess that the result would be a list of "DWARF information per binary"?

[pombredanne]

As a package can contain multiple binaries I guess that the result would be a list of "DWARF information per binary"?

Yes, that's per binary file

[armijnhemel]

As a package can contain multiple binaries I guess that the result would be a list of "DWARF information per binary"?

Yes, that's per binary file

What would this look like for a package like coreutils (with ~100 binaries)?

[pombredanne]

What would this look like for a package like coreutils (with ~100 binaries)?

more or less like this:

- files:
  - path: foo/bar/find
    dwarf_paths:
      - /home/joe/coreutil/includes/baz.h
      - /home/joe/coreutil/find.c
  - path: foo/bar/ls
    dwarf_paths:
      - /home/joe/coreutil/includes/baz.h
      - /home/joe/coreutil/js.c

[TG1999]

It should be an add-on pipeline in the data model, it should be used for d2d on elfs. This part is just to collect dwarfs.

[pombredanne]

The basis would be provided by the https://github.com/nexb/elf-inspector

[TG1999]

Will be closed by aboutcode-org/scancode.io#1068

[TG1999]

Now we have an add-on pipeline in scancode.io to get dwarfs from a codebase, but we need this support for a package URL, so some of the ways we could achieve this, we can check if there is a download URL associated with that purl in qualifiers. If there is no download URL, we can try to get it from purlDB, and if there is nothing in purldb regarding that purl, get it from purl2url module from purl.

[pombredanne]

@TG1999 you should sync up with @keshav-space on this #260 (comment) as he has some things in the works for purl2sym

[pombredanne]

A good example of a package with DWARFs is https://repo1.maven.org/maven2/com/sun/jna/jna/3.0.9/jna-3.0.9.jar which has the PURL: pkg:maven/com.sun.jna/[email protected]

[pombredanne]

Related is this issue: aboutcode-org/scancode.io#1121

Another example pkg:generic/[email protected]?download_url=https://github.com/nexB/elf-inspector/raw/4333e1601229da87fa88961389d7397af6e027c4/tests/data/dwarf_and_elf/analyze.so.debug

[JonoYang]

@TG1999 I've merged #353

[TG1999]

This is done. We now have:

a pipeline inspect_elf_binaries from scancode.io that can get dwarf data for ELFs
These PRs have been merged

• Add service for purl2git #299
• Add an add-on pipeline for collecting dwarfs from elfs scancode.io#1068

To test this feature:

• Install PurlDB
• Make a request for a purl that contains ELFs at /api/packages/ and then check it's "resources", you will find dwarf data in "extra_data" field of every ELF resource.

And to test this first part of purl2dwarf, install the latest scancode.io

• Open http://localhost/project in your browser, Create a new project:
• Name this “test”
• Add https://github.com/nexB/elf-inspector/raw/main/tests/data/dwarf_and_elf/analyze.so.debug as an input URL
• Select scan_single_package pipeline
• Click “Create”
• Press F5
• Click Add pipeline and add inspect_elf_binaries pipeline
• Click on Resources and click on the analyze.so.debug
• On the extra tab, you will see the DWARF compilation unit paths