[FLINK-37511][rest] Use Jackson serialization in JobPlanInfo.Plan

[Efrat19]

What is the purpose of the change

The pr follows refactors done over time to transition rest object classes to use modern auto-serialization instead of implementing their own serializations.
Doing the same for org.apache.flink.runtime.rest.messages.JobPlanInfo.Plan improves error handling and makes the restAPI more informative about expected response.

Brief change log

• org.apache.flink.runtime.rest.messages.JobPlanInfo.Plan type refactored to object instead of a string
• References updated as well

Verifying this change

This change is already covered by existing tests, such as (please describe tests).

• org.apache.flink.runtime.jobgraph.jsonplan.JsonGeneratorTest#testGeneratorWithoutAnyAttachements
• org.apache.flink.runtime.rest.messages.JobPlanInfoTest#testJsonMarshalling

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: Not sure which serializers we are talking about here
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

Open questions / follow ups

To be able to completely get rid of org.apache.flink.runtime.rest.messages.JobPlanInfo.RawJson#RawJson I would need to refactor StreamGraphs as well - Should probably separate it to a follow up PR to reduce blast radius? cc @zentol

[flinkbot]

CI report:

• 5e58710 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[davidradl]

• this appears to be an issue with the open api and rest docs - that are not generating correctly.
• the fix seems to be to change the way we do job serialization
• on the face of it this seems a large change to a core part of Flink, that could introduce other considerations for example how are existing job serializations effected.

Have I understood this correctly?

[Efrat19]

@flinkbot run azure