Fix Windows CI failures. #2143

ni4 · 2023-11-08T13:42:35Z

codecov · 2023-11-08T14:20:34Z

Codecov Report

Attention: 8 lines in your changes are missing coverage. Please review.

Comparison is base (05f59e9) 76.88% compared to head (6abc2ec) 11.76%.
Report is 1 commits behind head on main.

❗ Current head 6abc2ec differs from pull request most recent head 979de1e. Consider uploading reports for the commit 979de1e to get more accurate results

Files	Patch %	Lines
src/librepgp/stream-common.cpp	20.00%	4 Missing ⚠️
src/librepgp/stream-parse.cpp	40.00%	3 Missing ⚠️
src/lib/rnp.cpp	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #2143       +/-   ##
===========================================
- Coverage   76.88%   11.76%   -65.12%     
===========================================
  Files         193      193               
  Lines       36982    47247    +10265     
===========================================
- Hits        28432     5558    -22874     
- Misses       8550    41689    +33139

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

src/librepgp/stream-common.cpp

                return false;
            }
+            // RNP_LOG("here: %p %zu", src->read, read);


src/librepgp/stream-common.cpp

                return false;
            }
+            // RNP_LOG("here: %p %zu", src->read, read);


src/librepgp/stream-common.cpp

@@ -133,6 +141,7 @@
    if (src->knownsize && (src->readb == src->size)) {
        src->eof = 1;
    }
+    //RNP_LOG("read bytes: %zu", len);


src/librepgp/stream-parse.cpp

ni4 · 2023-11-14T12:56:27Z

@maxirmx Do you have an idea why for windows-native runner cache hits for openssl backend, but always misses botan, taking 20m to install vcpkg packages? Thanks!

maxirmx · 2023-11-14T14:25:36Z

@maxirmx Do you have an idea why for windows-native runner cache hits for openssl backend, but always misses botan, taking 20m to install vcpkg packages? Thanks!

@ni4, I guess it happens because there is no cache. Cache is created only if its job completes sucessfully and the latest botan runs either fails (test) or are cancelled. Caches are invalidated often because of vcpkg updates. Vcpkg does not have versions or releases so I am just fetching the latest version.

IMHO it would be difficult to fix this using GHA without local debugger. I tried to run it locally but cli tests (python) can run in MSys environment only so I was not able to create any working environment.

If you think it is feasible I can fix the tests and go through it with debugger. It will probably take 2-3 days. Not sure I will be able to fix fast on my own the issue but at least we will know what is broken.

ni4 · 2023-11-14T14:28:39Z

@maxirmx Thanks for the reply! Didn't know that cache is only submitted on successfull run. Actually with the latest iteration got much closer to the solution for the issue, now I (most likely) see point of failure. So hopefully few more iterations and got this resolved.
Don't have Windows environment at the moment, so this way seems to be faster.

src/librepgp/stream-parse.cpp

@@ -276,6 +286,7 @@
        param->pleft -= read;
    }

+    // RNP_LOG("partial end of cycle");


src/librepgp/stream-parse.cpp

        if (!src_read(param->readsrc, buf, read, &read)) {
            RNP_LOG("failed to read data chunk");
            return false;
        }
+        // RNP_LOG("partial read %zu", read);


src/librepgp/stream-parse.cpp

            param->psize = read;
            param->pleft = read;
        }

        if (!param->pleft) {
+            // RNP_LOG("partial zero chunk");


src/librepgp/stream-parse.cpp

                return false;
            }
+            // RNP_LOG("partial new chunk: %zu %d", read, (int) param->last);


src/librepgp/stream-parse.cpp

@@ -228,6 +228,7 @@
 static bool
 partial_pkt_src_read(pgp_source_t *src, void *buf, size_t len, size_t *readres)
 {
+    // RNP_LOG("partial %p start: %zu", src, len);


src/tests/cli_tests.py

ni4 · 2023-12-05T13:33:15Z

Closing this as problem is on the Botan's side, on our side is aimed to be fixed via PR #2148

ni4 force-pushed the ni4-2139-fix-windows-ci-failures branch 2 times, most recently from 8bfeb7d to 9d4c9fc Compare November 8, 2023 16:02

Attempt to fix Windows CI failures.

1f7759f

ni4 force-pushed the ni4-2139-fix-windows-ci-failures branch 10 times, most recently from 029e22e to b806bac Compare November 13, 2023 16:52

github-advanced-security bot found potential problems Nov 13, 2023

View reviewed changes

src/librepgp/stream-common.cpp Fixed Show fixed Hide fixed

src/librepgp/stream-common.cpp Fixed Show fixed Hide fixed

src/librepgp/stream-common.cpp Fixed Show fixed Hide fixed

ni4 force-pushed the ni4-2139-fix-windows-ci-failures branch 4 times, most recently from dc32ec4 to dd52f66 Compare November 13, 2023 20:30

github-advanced-security bot found potential problems Nov 13, 2023

View reviewed changes

ni4 force-pushed the ni4-2139-fix-windows-ci-failures branch 3 times, most recently from c69a703 to 8b393a0 Compare November 14, 2023 11:28

github-advanced-security bot found potential problems Nov 14, 2023

View reviewed changes

src/librepgp/stream-common.cpp

@@ -133,6 +141,7 @@

if (src->knownsize && (src->readb == src->size)) {

src->eof = 1;

}

//RNP_LOG("read bytes: %zu", len);

Check notice

Code scanning / CodeQL

Commented-out code Note

This comment appears to contain commented-out code.

ni4 force-pushed the ni4-2139-fix-windows-ci-failures branch from 8b393a0 to 6fd3d71 Compare November 14, 2023 12:24

github-advanced-security bot found potential problems Nov 14, 2023

View reviewed changes

src/librepgp/stream-parse.cpp Fixed Show fixed Hide fixed

ni4 force-pushed the ni4-2139-fix-windows-ci-failures branch 3 times, most recently from 3138fb0 to 44c5ea5 Compare November 14, 2023 13:49

Add some debug logging.

15a4b63

ni4 force-pushed the ni4-2139-fix-windows-ci-failures branch from 44c5ea5 to 15a4b63 Compare November 14, 2023 14:32

Logging update.

51646f7

github-advanced-security bot found potential problems Nov 14, 2023

View reviewed changes

Some tests fixes.

d169d84

github-advanced-security bot found potential problems Nov 15, 2023

View reviewed changes

src/tests/cli_tests.py Fixed Show fixed Hide fixed

src/tests/cli_tests.py Fixed Show fixed Hide fixed

ni4 added 4 commits November 15, 2023 13:44

Some more fixes.

b8b4533

Check for memory corruption.

e91fad2

Another try.

fd1e221

Update test.

6583954

github-advanced-security bot found potential problems Nov 15, 2023

View reviewed changes

src/tests/cli_tests.py Fixed Show fixed Hide fixed

ni4 force-pushed the ni4-2139-fix-windows-ci-failures branch 3 times, most recently from 8522f18 to 5195247 Compare November 15, 2023 16:12

ni4 mentioned this pull request Nov 15, 2023

CI: Add windows runner with enabled address sanitizer. #2147

Closed

ni4 force-pushed the ni4-2139-fix-windows-ci-failures branch 3 times, most recently from 0fe4fad to 04e81a4 Compare November 16, 2023 10:24

Another try.

979de1e

ni4 force-pushed the ni4-2139-fix-windows-ci-failures branch from 04e81a4 to 979de1e Compare November 16, 2023 10:35

ni4 closed this Dec 5, 2023

ni4 deleted the ni4-2139-fix-windows-ci-failures branch January 22, 2024 12:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix Windows CI failures. #2143

Fix Windows CI failures. #2143

ni4 commented Nov 8, 2023

codecov bot commented Nov 8, 2023 •

edited

Loading

ni4 commented Nov 14, 2023

maxirmx commented Nov 14, 2023

ni4 commented Nov 14, 2023

ni4 commented Dec 5, 2023

Fix Windows CI failures. #2143

Fix Windows CI failures. #2143

Conversation

ni4 commented Nov 8, 2023

codecov bot commented Nov 8, 2023 • edited Loading

Codecov Report

ni4 commented Nov 14, 2023

maxirmx commented Nov 14, 2023

ni4 commented Nov 14, 2023

ni4 commented Dec 5, 2023

codecov bot commented Nov 8, 2023 •

edited

Loading