[FR] [Simpleperf] Java source lines annotation

[RajatChauhan-0575]

Description

Simpleperf annotates only the Dex, oat, c++ files at present. Request to support line annotations and source attribution for Java source code (i.e .java files)

[RajatChauhan-0575]

Hi @yabinc,
Can you please let me know the approach that will be used to achieve this?

[yabinc]

Sure! The plan is to handle three types of Java code separately:

1. For Fully compiled Java code (compiled into oat files), check whether we can get source line info with additional debug info. The debug info can be generated via https://android.googlesource.com/platform/system/extras/+/refs/heads/main/simpleperf/scripts/app_profiler.py#304.

2. For interpreted Java code, we need to parse source line info from dex files, that may be achieved in different ways:
   a) Extend the interface of libdexfile to get source lines from dex files, in https://source.corp.google.com/h/android/platform/superproject/main/+/main:art/libdexfile/external/include/art_api/dex_file_support.h.
   b) Since we only need the source annotation function on host, we can write Python scripts to parse dex files, maybe using some existing libraries, like https://pypi.org/project/dexparser/. I prefer this way. Because python scripts is more flexible. But need to consider compact dex files used by ART.

3. For JITed Java code, we need to generate debug info via https://android.googlesource.com/platform/system/extras/+/refs/heads/main/simpleperf/scripts/app_profiler.py#304. Also need to keep that debug info in file after recording. So we can use it when doing source line annotation on host.

[RajatChauhan-0575]

Thanks for your reply!
For point 3, I have set the property debug.generate-debug-info to true before profiling, also I have tried to store the data in a file, and it has the .debug_info, .debug_line and .sym_tab in the elf file but I could not see the line number info inside it as we see in normal elf files on linux.
I have attached the objdump -w and hexdump output of the elf file for reference

Misc:
I have tried the objdump -Cdl also on the elf but there is not output as we see on normal elf file on linux

objdumpdwarf.txt
hexdump.txt

[yabinc]

In the objdumpdwarf.txt, I saw .debug_line section had file name and line number changes:

`The File Name Table (offset 0x2d):
Entry Dir Time Size Name
1 1 0 0 MainActivity.java

Line Number Statements:
[0x00000043] Extended opcode 2: set Address to 0x58eb01f0
[0x0000004e] Advance Line by 40 to 41
[0x00000050] Special opcode 5: advance Address by 0 to 0x58eb01f0 and Line by 0 to 41
[0x00000051] Set prologue_end to true
[0x00000052] Set is_stmt to 1
[0x00000053] Advance PC by 94 to 0x58eb024e
`

Maybe that's enough.
The elf file only exports debug info, it doesn't contain .text section. That is probably fine. Because we don't need instructions for source lines annotation. And that's why we see no instructions in objdump -Cdl output.

[RajatChauhan-0575]

ok, got it. I am able to map the IP from perf samples with the JIT Dump using addr2Line.
Is there any way we can show the assembly as well for the JIT code?

[RajatChauhan-0575]

Hi @yabinc,
It would be really helpful if you could please respond to my query above?

[yabinc]

Sorry for the late reply!
To get disassembly for JIT code, we need to dump native instructions of JIT code from memory. Otherwise they are gone when the process exits.
I don't plan to add that function in simpleperf. Because it's complex and not very relevant to the core functions of simpleperf. However, if you are really interested, you can write a separate tool to dump JIT code from memory. JIT code function addresses can be found in below steps:

1. Write code to read JITCodeEntries, similar to https://cs.android.com/android/platform/superproject/main/+/main:system/extras/simpleperf/JITDebugReader.cpp;l=113.
2. Parse symfile, get in memory ranges of JIT functions.
3. Read function instructions from the app process. Save them to disk.
4. Put the function instructions and the symfile in one ELF file, figure out a way to make it work with llvm-objdump.

[RajatChauhan-0575]

sure, thanks for your response