Update to support Tesla Custom Integration re-write and add battery site support

[shred86]

Overview

This update includes support for the Tesla Custom Integration re-write and adds support for Tesla solar systems with Powerwall. The general design I'm following is what I see from other libraries like simplisafe-python and Home Assistant documentation. I'm not a developer by any means, so please provide feedback/recommendations. The general flow for using teslajsonpy is:

1. Create an instance of teslajsonpy.Controller.
2. Call Controller.connect to get a list of products from Tesla to create self._vehicle_list and self._energysite_list. Two new arguments available, include_vehicles and include_energysites which defaults True.
3. Call Controller.generate_car_objects which returns a dictionary of car objects by VIN number.
4. Call Controller.generate_energysite_objects which returns a dictionary of energysite objects by energy site ID.

The car and energy site objects are the primary means to get car and energy site data and send commands.

Changes

• New module car.py that contains a TeslaCar class.
• New module energy.py that contains energy site classes.
• New method Controller.generate_car_objects that generates TeslaCar objects and stores them into self.cars by vin.
• New method Controller.generate_energysite_objects that generates SolarSite, PowerwallSite or SolarPowerwallSite objects and stores them in the self.energysites dictionary by energysite_id.
• Modified Controller.update to now just send a single request to the PRODUCT_LIST endpoint (instead of both PRODUCT_LIST and VEHICLE_LIST to get all products on a Tesla account. From there, create a list of cars self._vehicle_list and energy sites self._energysite_list.
• Storing JSON responses as-is into their own dictionary by VIN for cars and energysite_id for energysites. These are then passed to TeslaCar and EnergySite when instantiated.
• Added include_vehicles and include_energysites arguments to Controller.connect which default to True. This provides the option to completely ignore vehicles or energysites.
• Removed Controller.command, Controller.get and Controller.post methods. Only Controller._wake_up was using post which now uses Controller.api.
• Removed all Home Assistant specific modules.
• Removed some unused code specific to energy sites.
• Changes to some naming to try to better align with what the Tesla API uses.

Fixes

• Change Trunk, Frunk and Battery Door to be cover and not lock #348
• "grid_status" and "solar_power" values #334
• Generalize API and remove home-assistant specific architecture #24

Energy sites

For energy sites, there's a few variations that a user could have and we still need to get more JSON response examples from as many different setups as possible. My guess is the gateway is what determines how the data is published, whether it's a Tesla inverter, home energy gateway, Powerwall 2, etc. What I can tell right now is:

• Solar panels with a Tesla inverter (no Powerwalls), the data we need is from the SITE_DATA endpoint.
• Solar panels with a Solar Edge inverter and Home Energy Gateway with Powerwall 2, the data we need comes from BATTERY_DATA and BATTERY_SUMMARY

There's a couple variations on solar setups but in general, I'm treating it as essentially three from a Class design perspective:

• Solar only
• Powerwall only
• Solar and Powerwall

As you can see in the changes, I created a base EnergySite class and three child classes,SolarSite, PowerwallSite and SolarPowerwallSite. The first two inherit from EnergySite and SolarPowerwallSite inherit from SolarSite and PowerwallSite. There's probably a better way to do this, but this is what I came up with so far.

Notes

• Currently passing in the entire instance of the Controller (self) when calling Controller._generate_car_objects so we have access to all methods on the Controller compared to energysites, I'm only passing in self.api, the specific energysite in self.energysites and self.__power_data[energysite_id].
• Trunk, frunk and charger doors aren't really "locks" so I find it a bit confusing that if I'm "unlocking" a trunk, it opens. I'm sure this was done intentionally but I wonder if something like a Cover entity would be better.

To Do / Request Help

• Does the username and password authentication actually work? It would be nice to be able to allow users to authenticate with username and password instead of passing a refresh token in HA.
• Testing methods is simply asserting is None since they don't return anything. I'm assuming we probably want to do something like check if Controller.api is being called, but unittest.mock.patch doesn't work with async functions.
• Need to update the sphinx-docs. I updated some of the .rst in /docs/teslajsonpy based on my understanding of skimming over the sphinx documentation but I haven't figured out how to build and update everything.

[alandtse]

Makes sense to address #348

[shred86]

I just noticed the endpoint CACHED_PROTO_VEHICLE_DATA which has a note "This is cached data, pushed by the vehicle on sleep, wake and around OTAs.". When looking at the JSON response of this endpoint, it seems to have everything VEHICLE_DATA has but even more. I also noticed everything in the VEHICLE_DATA response is also in the CACHED_PROTO_VEHICLE_DATA response but listed under a legacy key, which makes me think perhaps using CACHED_PROTO_VEHICLE_DATA might be better.

Anyone happen to know the difference between these two? If the note is accurate, it seems like to me we could simply just poll this endpoint without having to worry about waking up the vehicle. If it's asleep, it should just return the cached data.

[alandtse]

That seems new. We could use that to start but do we lose the info on whether a car is sleeping? I guess the online sensor may be enough to figure that out.

This may also solve the issue of alandtse/tesla#251.

I know you've been basically working yourself on this for like a month without my help. Are we getting to a point where we should consider taking it live?

[shred86]

It appears to have the car state as well "state": "asleep", but after looking at this a bit more it appears everything listed under the data key might only be updated on certain events (i.e. on sleep, on wake, etc.) as described in the notes. As an example, I started charging my car and waited about 5 minutes. All of the information under data appears to be old (charging_state showing complete) but everything under legacy is current (charging_state showing charging). This might still be useful to just use this endpoint and if the car is awake, pull the data from the legacy key and if it's asleep, use the data key.

I've been running it 24/7 on a separate HA install and it's been working well so far. I think it's ready for use. If we could also release a beta version of the HA Tesla integration, that would be good to get some more folks to maybe test it out. There's a couple things I'd like to clean up on the integration though first but I'll continue to update that PR.

[alandtse]

Once it's in Dev it's basically beta, but you'll need to recruit people willing to test it.

This might still be useful to just use this endpoint and if the car is awake, pull the data from the legacy key and if it's asleep, use the data key.

This sounds fine. It can be the backup source when sleeping.

[shred86]

Ah bummer, it looks like CACHED_PROTO_VEHICLE_DATA returns 408 (vehicle unavailable) as well when the car is asleep, so that's not going to work.

[alandtse]

Just so it's tracked, please ensure conflicts are resolved so it merges cleanly. I can review and provide comments then. This should also make sure we don't obliterate any fixes that have been submitted by others. You can just look at closed PRs as there haven't been too many.

[alandtse]

I assume the tests we had for the HA components will be migrated to the /tesla app right? After you add the headers in and confirm the documentation state, I'll do a breaking change release based on your first post. We can then integrate your PR into dev and let it run for a week or two. We'll want to do the official release to master when you are available to handle any major bugs so let me know what your preferred schedule is. I'm based in California and we can target a weekend.

Thanks again for all your hard work on this. I can't adjust your status on this repo, but I can make you a maintainer on tesla if you'd like to help out more actively.

[shred86]

I assume the tests we had for the HA components will be migrated to the /tesla app right? After you add the headers in and confirm the documentation state, I'll do a breaking change release based on your first post. We can then integrate your PR into dev and let it run for a week or two. We'll want to do the official release to master when you are available to handle any major bugs so let me know what your preferred schedule is. I'm based in California and we can target a weekend.

Thanks again for all your hard work on this. I can't adjust your status on this repo, but I can make you a maintainer on tesla if you'd like to help out more actively.

That sounds good. I'm in the same time zone so that should work. I'm happy to continue helping out when I can so more than happy to be a maintainer on the Tesla integration.