Error encountered while parsing package_config.json: Unexpected end of input

[FMorschel]

I've been contributing for months now and on Thursday last week I updated my SDK again following https://gsdview.appspot.com/dart-archive/channels/main/raw/latest/sdk/. Now after that change, even my open CLs and when I checkout origin/main, I'm unable to run any code (basically tests) on my clone (though VS Code). I've tried debugging this with @DanTup but neither of us has an idea on how to continue this.

I've tried:

• git rebase-update
• git reset --hard
• git clean
• Deleting the full dir (except .git) and git reset --hard
• Deleting only .dart_tool folders and git reset --hard

All of the above running gclient sync. I downloaded a new SDK on Friday and today again, but still no luck.

This is the output:

Connecting to VM Service at ws://127.0.0.1:50337/vrDwLsXNRVY=/ws
Connected to the VM Service.
Error encountered while parsing package_config.json: Unexpected end of input

Exited (255).

Here is a capture of debugging logs (even though @DanTup said there was nothing that would help) Dart-Code-Log-2025-00-01 09-08-44.txt (added the package_config.json file below but from my human parsing it seems alright).

Seems like this error might be happening inside warnOnNativeAssets() in dartdev. However, it does seem to only hit this code if there is a package_config.json and then a FormatException occurs.

Here is my vscode workspace file (had to add the trailing .txt since GitHub doesn't accept the extension) server.code-workspace.txt which is basically a copy of what @DanTup uses.

On the debugging logs above, I tried running pkg\analysis_server\test\services\completion\dart\completion_test.dart and created and tried running pkg/analysis_server/bin/a.dart which was a file containing only:

void main() {
  print('Hello, World!');
}

Running either on the terminal with my SDK downloaded from the link above showed no issue.

---

• The above error message tells me nothing on what package_config.json it found the problem but our guess is .dart_tool/package_config.json from the root of the project (package_config.json)
• I could not find any way of enabling a verbose read of that file even adding -v to dart run on the workspace file above at the launch.
• I'm on Windows 11 Pro with the latest stable VS Code and the latest pre-release of Dart-Code

[dart-github-bot]

Summary: After updating the Dart SDK, the user encounters "Unexpected end of input" while parsing package_config.json, preventing code execution. Troubleshooting steps, including SDK reinstallation, failed.

[DanTup]

The only place I can find this error string is here:

sdk/pkg/dartdev/lib/src/native_assets.dart

Lines 120 to 122 in 6417e1c

	log.stderr(
	'Error encountered while parsing package_config.json: ${e.message}',
	);

But it's not clear what's going wrong. I'm unable to reproduce the issue. Both the original stack trace that led to this call and also the path of the package_config.json that was being read might help.

(cc @bkonyi in case you have any ideas, I think you've been worked on some of this code).

[DanTup]

I noticed you included your package_config file too, so I diffed it, and it is identical to mine, so I don't think there's anything wrong with that file.

[bkonyi]

@dcharkes owns the native assets work, so maybe he'll know?

[dcharkes]

Thanks for the report @FMorschel!

I've recently reorganized this code to be more strict in requiring a packge_config.json. I've added support for pub workspaces in the native assets feature, which means we need to start being explicit about where the package_config.json is (and we need to read its conents).

The error message sounds like the package config is read while it's being written to by some other process.

I'll add the path to the error message.

I also think we should consider not failing early in this function, but let dartdev fail later, if the package config isn't valid.

https://dart-review.googlesource.com/c/sdk/+/406280

[FMorschel]

The error message sounds like the package config is read while it's being written to by some other process.

Not to me, can this message be improved @dcharkes, so that it is clearer to the user? Hopefully, this won't happen again but still.

[dcharkes]

The error message sounds like the package config is read while it's being written to by some other process.

Not to me, can this message be improved @dcharkes, so that it is clearer to the user? Hopefully, this won't happen again but still.

The message comes straight from dart:convert I believe. package:package_config emits it. We could add the stack trace to figure out where exactly. But that will most likely just lead to the code that tries to parse the bytes that are supposed to contain JSON. If I would guess, json.decode would be the exact place it's thrown.

(Yes not concurrent modification, but an empty file or a file that was partially flushed or something.)

[FMorschel]

Alright, more weirdness for us:

I just ran pkg\analysis_server\test\src\services\correction\fix\create_method_test.dart and the output on Debug Console was:

Connecting to VM Service at ws://127.0.0.1:52972/Ux9X7SyE-vw=/ws
Connected to the VM Service.
Error encountered while parsing C:\Users\felip_0vh5fa6\Documents\Projects\clones\dart-sdk2\sdk\.dart_tool\package_config.json: Unexpected end of input.
00:00 +0: AddMissingHashOrEqualsTest | test_equals

00:03 +1: AddMissingHashOrEqualsTest | test_equals_fieldHashCode

00:04 +2: AddMissingHashOrEqualsTest | test_hashCode

00:04 +3: CreateMethodMixinTest | test_createQualified_instance

00:05 +4: CreateMethodMixinTest | test_createQualified_static

00:06 +5: CreateMethodMixinTest | test_createUnqualified

00:06 +6: CreateMethodMixinTest | test_functionType_method_enclosingMixin_static

00:06 +7: CreateMethodMixinTest | test_functionType_method_targetMixin

00:07 +8: CreateMethodTest | test_createQualified_emptyClassBody

00:07 +9: CreateMethodTest | test_createQualified_fromClass

00:07 +10: CreateMethodTest | test_createQualified_fromClass_hasOtherMember

00:07 +11: CreateMethodTest | test_createQualified_fromExtensionType

00:07 +12: CreateMethodTest | test_createQualified_fromInstance

00:08 +13: CreateMethodTest | test_createQualified_instance_fromExtensionType

00:08 +14: CreateMethodTest | test_createQualified_targetIsFunctionType

00:08 +15: CreateMethodTest | test_createQualified_targetIsUnresolved

00:08 +16: CreateMethodTest | test_createUnqualified_duplicateArgumentNames

00:09 +17: CreateMethodTest | test_createUnqualified_instanceField

00:09 +18: CreateMethodTest | test_createUnqualified_instanceField_late

00:09 +19: CreateMethodTest | test_createUnqualified_parameters

00:09 +20: CreateMethodTest | test_createUnqualified_parameters_named

00:09 +21: CreateMethodTest | test_createUnqualified_returnType

00:09 +22: CreateMethodTest | test_createUnqualified_staticField

00:10 +23: CreateMethodTest | test_createUnqualified_staticField_late

00:10 +24: CreateMethodTest | test_createUnqualified_staticFromMethod

00:10 +25: CreateMethodTest | test_functionType_argument

00:10 +26: CreateMethodTest | test_functionType_method_enclosingClass_instance

00:11 +27: CreateMethodTest | test_functionType_method_enclosingClass_static

00:11 +28: CreateMethodTest | test_functionType_method_enclosingClass_static2

00:11 +29: CreateMethodTest | test_functionType_method_FunctionCall

00:12 +30: CreateMethodTest | test_functionType_method_inside_conditional_operator_condition

00:12 +31: CreateMethodTest | test_functionType_method_inside_conditional_operator_condition_FunctionCall

00:12 +32: CreateMethodTest | test_functionType_method_inside_conditional_operator_else

00:12 +33: CreateMethodTest | test_functionType_method_inside_conditional_operator_else_FunctionCall

00:13 +34: CreateMethodTest | test_functionType_method_inside_conditional_operator_then

00:13 +35: CreateMethodTest | test_functionType_method_inside_conditional_operator_then_FunctionCall

00:13 +36: CreateMethodTest | test_functionType_method_inside_record_functionType

00:14 +37: CreateMethodTest | test_functionType_method_inside_record_functionType_named

00:14 +38: CreateMethodTest | test_functionType_method_targetClass

00:14 +39: CreateMethodTest | test_functionType_method_targetClass_hasOtherMember

00:14 +40: CreateMethodTest | test_functionType_notFunctionType

00:14 +41: CreateMethodTest | test_functionType_unknownTarget

00:15 +42: CreateMethodTest | test_generic_argumentType

00:15 +43: CreateMethodTest | test_generic_literal

00:15 +44: CreateMethodTest | test_generic_local

00:15 +45: CreateMethodTest | test_generic_returnType

00:16 +46: CreateMethodTest | test_hint_createQualified_fromInstance

00:16 +47: CreateMethodTest | test_inSDK

00:16 +48: CreateMethodTest | test_internal_instance

00:16 +49: CreateMethodTest | test_internal_static

00:17 +50: CreateMethodTest | test_override

00:17 +51: CreateMethodTest | test_parameterType_differentPrefixInTargetUnit

00:17 +52: CreateMethodTest | test_parameterType_inTargetUnit

00:17 +53: CreateMethodTest | test_static

00:17 +54: CreateMethodTest | test_targetIsEnum

00:18 +55: All tests passed!


Exited.

There was a big delay between the error message and everything else. But as I mentioned in a talk on Discord, the file in question is the one I sent over and the last modification date is yesterday morning (when I did the deletion of all .dart_tool folders and recreated with git reset --hard). I'm not sure what should I do to help us fix this, although this is already better since now I can debug things again.

[DanTup]

That is odd. I wonder if a stack trace would help to know exactly who is throwing this. I thought this was coming from jsonDecode, however in a quick test on Dartpad, the error when trying to decode an empty string or truncated JSON is slightly different (it contains "JSON" in the text):

FormatException: SyntaxError: Unexpected end of JSON input

(@FMorschel have you ever built the SDK locally before? (something like ./tools/build.py --mode release --arch x64 create_sdk). It might be easier to add some temp debug logging locally and build an SDK than it is to land something.

[FMorschel]

have you ever built the SDK locally before?

Yes, I have. It is slow but doable.

[DanTup]

In that case, you could try changing the code here:

sdk/pkg/dartdev/lib/src/native_assets.dart

Line 117 in 6417e1c

} on FormatException catch (e) {

To have catch (e, s) { to get the stack trace and also write that to log.stderr. That should help understand exactly which call is failing, and from there you might be able to print the content out.

Perhaps it's also worth verifying the encoding of that file - for me it shows UTF-8, but I don't know if the encoding it's written with might be influenced by settings on the system.

[FMorschel]

Perhaps it's also worth verifying the encoding of that file - for me it shows UTF-8, but I don't know if the encoding it's written with might be influenced by settings on the system.

It is also UTF-8.

To have catch (e, s) { to get the stack trace and also write that to log.stderr.

Building... Will update you with a new comment on what I find.

[FMorschel]

Here is the stacktrace:

Error encountered while parsing C:\Users\felip_0vh5fa6\Documents\Projects\clones\dart-sdk2\sdk\.dart_tool\package_config.json: Unexpected end of input.
The stack trace for the error is:
#0      _ChunkedJsonParser.fail (dart:convert-patch/convert_patch.dart:1463:5)
#1      _ChunkedJsonParser.close (dart:convert-patch/convert_patch.dart:497:7)
#2      _parseJson (dart:convert-patch/convert_patch.dart:36:10)
#3      JsonDecoder.convert (dart:convert/json.dart:641:36)
#4      JsonCodec.decode (dart:convert/json.dart:223:41)
#5      jsonDecode (dart:convert/json.dart:160:12)
#6      new PackageGraph.fromPubDepsJsonString (package:native_assets_builder/src/build_runner/build_planner.dart:151:36)
#7      NativeAssetsBuildPlanner.fromPackageConfigUri (package:native_assets_builder/src/build_runner/build_planner.dart:51:22)
<asynchronous suspension>
#8      NativeAssetsBuildRunner._planner.<anonymous closure> (package:native_assets_builder/src/build_runner/build_runner.dart:759:21)
<asynchronous suspension>
#9      NativeAssetsBuildRunner.packagesWithBuildHooks (package:native_assets_builder/src/build_runner/build_runner.dart:92:21)
<asynchronous suspension>
#10     DartNativeAssetsBuilder.warnOnNativeAssets (package:dartdev/src/native_assets.dart:159:28)
<asynchronous suspension>
#11     RunCommand.run (package:dartdev/src/commands/run.dart:389:15)
<asynchronous suspension>
#12     CommandRunner.runCommand (package:args/command_runner.dart:212:13)
<asynchronous suspension>
#13     DartdevRunner.runCommand (package:dartdev/dartdev.dart:240:18)
<asynchronous suspension>
#14     runDartdev (package:dartdev/dartdev.dart:47:16)
<asynchronous suspension>
#15     main (file:///C:/Users/felip_0vh5fa6/Documents/Projects/clones/dart-sdk2/sdk/pkg/dartdev/bin/dartdev.dart:13:5)
dartdev.dart:13
<asynchronous suspension>

I'll keep trying out different things on my end, any suggestions are appreciated.

[FMorschel]

Over at third_party\pkg\native\pkgs\native_assets_builder\lib\src\build_runner\build_planner.dart I added inside NativeAssetsBuildPlanner.fromPackageConfigUri the following:

final workingDirectory = packageConfigUri.resolve('../');
logger.severe('Running `dart pub deps --json` in $workingDirectory');  // New line
final result = await Process.run(
  dartExecutable.toFilePath(),
  [
    'pub',
    'deps',
    '--json',
  ],
  workingDirectory: workingDirectory.toFilePath(),
);
logger.severe('json data: "${result.stdout}"');  // New line
logger.severe('error: "${result.stderr}"');  // New line
final packageGraph = PackageGraph.fromPubDepsJsonString(result.stdout as String); // Throwing line

And above the previously suggested change:

Future<bool> warnOnNativeAssets() async {
  try {
    _logger.onRecord.listen((record) => log.stderr(record.message)); // New line
    final builder = await _nativeAssetsBuildRunner;
    final packageNames = await builder.packagesWithBuildHooks();
    if (packageNames.isEmpty) return false;
    log.stderr(
      'Package(s) $packageNames require the native assets feature to be enabled. '
      'Enable native assets with `--enable-experiment=native-assets`.',
    );
    return true;
  } on FormatException catch (e, s) {
// ...

These are the new lines I got on the debug console:

Running `dart pub deps --json` in file:///C:/Users/felip_0vh5fa6/Documents/Projects/clones/dart-sdk2/sdk/
json data: ""
error: "Found no `pubspec.yaml` file in `C:\Users\felip_0vh5fa6\Documents\Projects\clones\dart-sdk2\sdk` or parent directories"

Now we know why it is failing for me but I still don't think I understand why it is not failing for any of you.

[dcharkes]

Okay, my thoughts so far.

In order to know the "root package" inside a pub workspace, we need to find the nearest pub spec. Also, in order to find the order in which to run build hooks (if the experiment is enabled), we need run this pub command to get a dependency graph.

Before my change, dart run would lazily try to run dart pub get if no package_config.json could be found. After the change to support pub workspaces, we still run dart pub get even if we have found a package_config.json. This is my hypothesis why things now started failing on your machine.

Taking a step back, we don't support using packages with native assets in the SDK itself (currently). So dart debug_adapter should not have any dependencies with native assets, and neither should the dart run commands that the debug_adapter spawns. @DanTup Is there a reason the debug adapter uses dart run over dart path/to/script.dart? The latter one would not try to run native assets.

Now we know why it is failing for me but I still don't think I understand why it is not failing for any of you.

I don't understand this either, there is no top level pubspec.yaml in the SDK. So it should be failing for us as well.

@bkonyi What is the contract that we want to enforce for dartdev including dart run?

1. You must have a .dart_tool/package_config.json in your own directory or one of it's ancestor directories, otherwise you must have a pubspec.yaml in your own directory or one of it's ancestor directories.
2. You must have a pubspec.yaml in your own directory or one of it's ancestor directories, and may have a .dart_tool/package_config.json in your own directory or one of it's ancestor directories. The latter one may be overridden by running pub get on the former.

dart path/to/script has the following contract:

• You may have a .dart_tool/package_config.json in your own directory or one of it's ancestor directories.

Dart without a command has no concept of pubspecs and the package manager.

If we want to enforce contract 1. Then the SDK should not be trying to run any dart run on its own codebase, because it only has a package_config.json and no pubspec.

If we want to enforce contract 2, Then (a) The package_config.json would need to be extended to include the dependency graph, and (b) dartdev would need to find a different solution to finding the root package name from a directory. (a) feels problematic because everyone who's making package_config.jsons manually would be needing to add this extra info and it's not needed for dart path/to/script.dart. (b) feels problematic because the directory name does not have to be equal to the package name. The package name basically only exists in the pubspec.yaml name: field on disk.

So I'm leaning towards saying that the contract should be 1. And that dart debug_adapter should use dart path/to/script.dart instead of dart run.

[DanTup]

Now we know why it is failing for me but I still don't think I understand why it is not failing for any of you.

I don't understand this either, there is no top level pubspec.yaml in the SDK. So it should be failing for us as well.

I made the same changes as @FMorschel and built the SDK locally and tried to run pkg/analysis_server/bin/a.dart from VS Code (with my SDK set to the locally built version) and I got the same output!

So I tested back with my previous SDK, and I also see the error. My only conclusion is that I only previously tested with "Run" (which works) and not "Debug" (I'm surprised, since I'm sure I was debugging tests just yesterday, but it's my only good theory).

So, things are definitely broken.

@DanTup Is there a reason the debug adapter uses dart run over dart path/to/script.dart? The latter one would not try to run native assets.

I was actually mistaken - we are not using run:

Spawning D:\Tools\Dart\Latest (Bleeding Edge)\bin\dart.exe with [--no-serve-devtools, --enable-vm-service=0, -DSILENT_VM_SERVICE=true, --write-service-info=file:///C:/Users/danny/AppData/Local/Temp/dart-vm-servicec18a442f/vm.json, --pause_isolates_on_exit, --pause_isolates_on_start, --enable-asserts, D:\Dev\Google\dart-sdk\sdk\pkg\analysis_server\bin\a.dart] in D:\Dev\Google\dart-sdk\sdk\pkg\analysis_server
Connecting to VM Service at ws://127.0.0.1:50690/ovkRFbkhUGQ=/ws
Error encountered while parsing package_config.json: Unexpected end of input

However I'm not sure if internally it gets rewritten (@bkonyi?). Knowing that it only occurred in the debug mode, I was actually able to trivially reproduce this from the command line by just passing --enable-vm-service:

PS D:\Dev\Google\dart-sdk\sdk\pkg\analysis_server> & "D:\Tools\Dart\Latest (Bleeding Edge)\bin\dart.exe" --enable-vm-service=0 bin\a.dart                    
Error encountered while parsing package_config.json: Unexpected end of input
The Dart VM service is listening on http://127.0.0.1:50717/x8e6QSImwo4=/
The Dart DevTools debugger and profiler is available at: http://127.0.0.1:50717/x8e6QSImwo4=/devtools/?uri=ws://127.0.0.1:50717/x8e6QSImwo4=/ws

[DanTup]

Now I'm wondering why all of the tests aren't broken... Surely between both Dart and the DAP tests, we should be triggering this a lot?

Edit: Maybe all of the DAP tests have a pubspec.yaml in the project?

[dcharkes]

I see some interesting code here:

sdk/runtime/bin/main_options.cc

Lines 646 to 649 in 671bbe9

	// Handle the special case where the user is running a Dart program
	// without using a DartDev command and wants to use the VM service. Here
	// we'll run the program using DartDev as it's used to spawn a DDS
	// instance.

sdk/runtime/bin/main_options.cc

Lines 745 to 750 in 671bbe9

	if (implicitly_use_dart_dev && first_option) {
	// Special case where user enables VM service without using a dartdev
	// run command. If 'run' is provided, it will be the first argument
	// processed in this loop.
	dart_options->AddArgument("run");
	} else {

Maybe this is splicing in run in the command? @bkonyi Can you elaborate on this behavior?

[bkonyi]

I was literally just looking at this code when you commented @dcharkes!

I thought Siva had removed this logic once we moved to spawning DDS from the VM Service isolate instead of DartDev. Basically, since DartDev was responsible for spawning the DDS process we needed to implicitly add the run command when --enable-vm-service or --observe were provided and --no-dds and --disable-dart-dev were not present. I think this logic can be removed now.

[DanTup]

What's the intended difference between dart and dart run? I think I always assumed they were approximately equivalent (but maybe that's because the code above made them so). If the code above is removed, this bug would only occur for dart run - is the code that's triggering this intentionally only working with dart run? (eg. may users running with dart foo.dart in future be surprised that something relative to native assets stopped working?).

(I presume there's still a bug here to fix even if that's changed though?)

[dcharkes]

However I'm not sure if internally it gets rewritten ([@bkonyi](https://github.com/bkonyi)?). Knowing that it only occurred in the debug mode, I was actually able to trivially reproduce this from the command line by just passing --enable-vm-service:

PS D:\Dev\Google\dart-sdk\sdk\pkg\analysis_server> & "D:\Tools\Dart\Latest (Bleeding Edge)\bin\dart.exe" --enable-vm-service=0 bin\a.dart                    
Error encountered while parsing package_config.json: Unexpected end of input

I can't reproduce this on MacOS. 🤷 Before I fire up a Windows machine, anyone on a Windows machine wants to try removing the referenced code?

What's the intended difference between dart and dart run?

In my understanding dart path/to/script.dart is only the core language, and it only consumes a package_config.json. dart run, dart build, dart compile, dart test are more like Flutter. A framework around the core language. (An example: For the native assets work, the Dart "Framework" and Flutter Framework know how to build an app and where to bundle native assets. The core Dart language does not.)

(eg. may users running with dart foo.dart in future be surprised that something relative to native assets stopped working?).

Probably not. Because the implicit run only occurs if someone passes the --enable-vm-service flag.

[bkonyi]

WIP CL to remove the implicit DartDev behavior here.

[DanTup]

Before I fire up a Windows machine, anyone on a Windows machine wants to try removing the referenced code?

Sure, I'll test with Ben's CL shortly.

Probably not. Because the implicit run only occurs if someone passes the --enable-vm-service flag.

Ah, yes :)

[DanTup]

I built an SDK from Ben's change, but now I get a weirder error:

And the same from the terminal:

PS D:\Dev\Google\dart-sdk\sdk\pkg\analysis_server> D:\Dev\Google\dart-sdk\sdk\out\ReleaseX64\dart-sdk\bin\dart.exe --observe .\bin\a.dart
Error when reading '.\bin\a.dart': No such file or directory.
The Dart VM service is listening on http://127.0.0.1:8181/e2vrGMWn79A=/
The Dart DevTools debugger and profiler is available at: http://127.0.0.1:8181/e2vrGMWn79A=/devtools/?uri=ws://127.0.0.1:8181/e2vrGMWn79A=/ws

I don't know where that error is coming from, but in the case of the full absolute path, the file definitely exists because I can ctrl+click it in the VS Code output and it opens the file.

Edit: I tried passing a file URI in case something was expecting that, but it made no difference:

PS D:\Dev\Google\dart-sdk\sdk\pkg\analysis_server> D:\Dev\Google\dart-sdk\sdk\out\ReleaseX64\dart-sdk\bin\dart.exe --observe file:///D:/Dev/Google/dart-sdk/sdk/pkg/analysis_server/bin/a.dart
Error when reading 'file:///D:/Dev/Google/dart-sdk/sdk/pkg/analysis_server/bin/a.dart': No such file or directory.
The Dart VM service is listening on http://127.0.0.1:8181/FNWKx_CPnj8=/
The Dart DevTools debugger and profiler is available at: http://127.0.0.1:8181/FNWKx_CPnj8=/devtools/?uri=ws://127.0.0.1:8181/FNWKx_CPnj8=/ws

[bkonyi]

I might have broken something in my CL, so I wouldn't look too deeply into that error quite yet.

[bkonyi]

@DanTup I've uploaded a fix for the issue you're seeing if you'd like to try it again (I was unconditionally skipping the first argument by accident).

[DanTup]

Yep, that works as expected now :-)