Remove panics from planning logic #770

Job	Run time
build-vegafusion-jupyter-packages	15m 38s
build-vegafusion-packages	2m 1s
build-vegafusion-python-embed-linux-64	19m 17s
build-vegafusion-python-embed-linux-arm64	23m 1s
build-vegafusion-python-embed-osx-64	21m 51s
build-vegafusion-python-embed-osx-arm64	20m 39s
build-vegafusion-python-embed-win-64	37m 39s
build-vegafusion-server-linux-64	18m 16s
build-vegafusion-server-linux-arm64	22m 59s
build-vegafusion-server-osx-64	24m 47s
build-vegafusion-server-osx-arm64	23m 58s
build-vegafusion-server-win-64	23m 32s
build-vegafusion-wasm	12m 18s
check-rust	13m 18s
test-rust (ubuntu-20.04)	55m 44s
test-rust (windows-2022)	1h 17m 15s
test-rust (macos-11)	55m 39s
test-vegafusion-python-linux-64	8m 51s
test-vegafusion-jupyter-linux-64	34m 45s
test-vegafusion-python-osx-64	10m 21s
test-vegafusion-python-win-64	2m 36s
	8h 44m 25s

Provide feedback