[4.8.0] Breaks esptool import due to esptool.py in Scripts instead of esptool.exe (ESPTOOL-924)

[matthuisman]

Operating System

Windows

Esptool Version

4.8.0

Python Version

3.10

Full Esptool Command Line that Was Run

No response

Esptool Output

from esptool import get_default_connected_device, ESPLoader, DEFAULT_CONNECT_ATTEMPTS
ImportError: cannot import name 'get_default_connected_device' from 'esptool' (\\?\C:\Users\matt\Envs\py310\Scripts\esptool.py)

What is the Expected Behaviour?

Can import from esp module and use esptool.exe cmdline

More Information

setuptools 75.1.0
pip 24.2
wheel 0.44.0

Other Steps to Reproduce

the Scripts directory has to be before site-pages in sys.path
virtualenvs do that

[matthuisman]

we are running another exe in Scripts so suspect Scripts is inserted to sys.path

[matthuisman]

ok. so its just python behaviour to add the cwd first to sys.path
So when our other console script in Scripts runs, python is inserting the Scripts dir to sys.path
And due to esptool.py having same name as the module - its finding the cmdline script

Can it be reverted back to exe?

Also, for users who have setup .py files to open in editor by default, just typing esptool or esptool.py opens the file for editing :(
Need to do python esptool.py ... now

[radimkarnis]

Hi @matthuisman,

I am confused a little by your report. You have several options how to run esptool:

• We recommend installing it with pip as a python package.
• Or you can download an executable binary for Windows (esptool.exe) at the bottom of the release. In this case, you don't need Python or do any installation.

Your report shows you've tried to install it as a Python package, but you keep mentioning esptool.exe.

Can it be reverted back to exe?

We haven't changed anything in this release and I don't fully understand what you mean by this. What version did you use before v4.8.0?

Also, for users who have setup .py files to open in editor by default, just typing esptool or esptool.py opens the file for editing

Try running simply esptool (not esptool.py) on windows.

[matthuisman]

It has changed. pip install esptool now gives you esptool.py in Scripts directory instead of esptool.exe Which breaks importing esptool package as they share the same name mkvirtualenv esptool activate the env pip install esptool cd to the virtualenv Scripts directory python -c "from esptool import ESPLoader" ERROR: ImportError: cannot import name 'ESPLoader' from 'esptool' (C:\Users\Matt\Envs\esptool\Scripts\esptool.py)

[matthuisman]

4.8.dev5 works ok

4.8.0 does not

[matthuisman]

maybe due to change in release process?
eacc94a

I notice previous releases are all .tar.gz but 4.8.0 uses a whl

[matthuisman]

if I use
pip install esptool==4.8 --no-binary :all:
to force it to ignore the whl, then it builds correctly and has the correct exes (no bug)

So the issue is just in the published whl

[matthuisman]

i tried cloning the source and then using the same python -m build
and the whl I get works correctly

What is intersting, the pypi whl published by your job has an extra .data directory that contains scripts and then the python scripts

You can see in your github actions log where it is creating those
https://github.com/espressif/esptool/actions/runs/10923306835/job/30319610052#step:5:648

When i do python -m build locally, my whl does not have the .data/scripts dir and uses entry_points instead.
So when that gests installed, setuptools generates the .exe for those entry_points and it works correctly

Not sure what is different to your github build action to mine locally (maybe different setuptools version or some env variable changing behaviour?

[matthuisman]

ok, if i build in ubuntu, i get the same result .data/scripts.

[matthuisman]

OK, so the issue is https://github.com/espressif/esptool/blob/master/setup.py#L4

Your building the whl on ubuntu so its choosing the if os.name != "nt": path and using scripts instead of entry_points which causes the issue on Windows.

I think you would need to publish a whl for windows and a whl for other platforms (if thats possible)
If not, you may need to just go back to not building a whl

[matthuisman]

PR created to use --sdist with build to avoid building non-universal whl
#1011