Installation issues

[ad-si]

I get following error after installing it with conda:

$ smude -o dewarped.png photo.png
Traceback (most recent call last):
  File "/usr/local/bin/smude", line 33, in <module>
    sys.exit(load_entry_point('smude==0.1.0', 'console_scripts', 'smude')())
  File "/usr/local/bin/smude", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/local/Cellar/[email protected]/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/usr/local/Cellar/[email protected]/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/local/lib/python3.9/site-packages/smude-0.1.0-py3.9.egg/smude/__init__.py", line 10, in <module>
    import torch
  File "/usr/local/lib/python3.9/site-packages/torch-1.8.0-py3.9-macosx-10.15-x86_64.egg/torch/__init__.py", line 196, in <module>
    from torch._C import *
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd

[sonovice]

Thanks for the report. Have you tried upgrading your numpy build?

[ad-si]

Uhm what exactly does that mean? I don't really know the Python ecosystem well, but I thought conda provides the whole environment and the dependencies, so shouldn't it just work?

[sonovice]

It should, but your traceback says .../torch-1.8.0-py3.9... which tells that you haven't used the conda environment (or something went really wrong) because neither the torch nor the python version meet the numbers in the environment file.
Maybe you used the setup.py method for installation? I just noticed that the version numbers are not fixed in there.

EDIT: You could try to fix it with pip install --upgrade numpy.

[ad-si]

I first used setup.py and since that didn't work I tried it with Conda afterwards. I guess that's the problem then. How can I fix it?

I just noticed that the version numbers are not fixed in there.

But they should, right? 😉

EDIT: You could try to fix it with pip install --upgrade numpy.

Didn't help

[sonovice]

But they should, right?

Yes. That part came from an external developer and it just slipped my attention. I will look into it as soon as I find some time for testing.

I first used setup.py and since that didn't work I tried it with Conda afterwards. I guess that's the problem then. How can I fix it?

Try deleting the entire env and start over:

conda env remove -n smude
conda env create -f environment.yml
conda activate smude # <-- Do this everytime you open a new terminal!

Let me know if that helps on your end.

[ad-si]

Try deleting the entire env and start over:

Already tried that and just tried it again, but doesn't help. 😕

Maybe it's because I'm using miniconda?

$ conda info                                                                                                                   (smude)

     active environment : smude
    active env location : /usr/local/Caskroom/miniconda/base/envs/smude
            shell level : 2
       user config file : /Users/adrian/.condarc
 populated config files : /Users/adrian/.condarc
          conda version : 4.9.2
    conda-build version : not installed
         python version : 3.8.5.final.0
       virtual packages : __osx=10.15.7=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /usr/local/Caskroom/miniconda/base  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/osx-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /usr/local/Caskroom/miniconda/base/pkgs
                          /Users/adrian/.conda/pkgs
       envs directories : /usr/local/Caskroom/miniconda/base/envs
                          /Users/adrian/.conda/envs
               platform : osx-64
             user-agent : conda/4.9.2 requests/2.24.0 CPython/3.8.5 Darwin/19.6.0 OSX/10.15.7
                UID:GID : 501:20
             netrc file : /Users/adrian/.netrc
           offline mode : False

[gorgobacka]

I added the setup.py. I also was not totally sure which exact versions are needed/required. It worked for me and therefore, I thought it works for others, too. It seems not be the case. Sorry for the inconvenience.

[sonovice]

I added the setup.py. I also was not totally sure which exact versions are needed/required. It worked for me and therefore, I thought it works for others, too. It seems not be the case. Sorry for the inconvenience.

No worries, we will sort this out. It is usually best practice to use fixed versions ==version for production code rather than >=version. Even if the dependencies use some sort of semantic versioning, things can break occasionally.

Maybe it's because I'm using miniconda?

miniconda is what I use as well. Could you please post the output of conda list?

[ad-si]

Could you please post the output of conda list?

Show output

# packages in environment at /usr/local/Caskroom/miniconda/base/envs/smude:
#
# Name                    Version                   Build  Channel
absl-py                   0.12.0             pyhd8ed1ab_0    conda-forge
aiohttp                   3.7.4            py38h96a0964_0    conda-forge
async-timeout             3.0.1                   py_1000    conda-forge
attrs                     20.3.0             pyhd3deb0d_0    conda-forge
blas                      2.108                       mkl    conda-forge
blas-devel                3.9.0                     8_mkl    conda-forge
blinker                   1.4                        py_1    conda-forge
blosc                     1.21.0               h046ec9c_0    conda-forge
brotli                    1.0.9                h046ec9c_4    conda-forge
brotlipy                  0.7.0           py38h5406a74_1001    conda-forge
brunsli                   0.1                  h046ec9c_0    conda-forge
bzip2                     1.0.8                hc929b4f_4    conda-forge
c-ares                    1.17.1               h0d85af4_1    conda-forge
ca-certificates           2020.12.5            h033912b_0    conda-forge
cachetools                4.2.1              pyhd8ed1ab_0    conda-forge
certifi                   2020.12.5        py38h50d1736_1    conda-forge
cffi                      1.14.4           py38h6afc60c_0    conda-forge
chardet                   3.0.4           py38h5347e94_1008    conda-forge
charls                    2.2.0                h046ec9c_0    conda-forge
click                     7.1.2              pyh9f0ad1d_0    conda-forge
cloudpickle               1.6.0                      py_0    conda-forge
cryptography              3.4.6            py38h1fa4640_0    conda-forge
cycler                    0.10.0                     py_2    conda-forge
cytoolz                   0.11.0           py38h5406a74_3    conda-forge
dask-core                 2021.3.0           pyhd8ed1ab_0    conda-forge
decorator                 4.4.2                      py_0    conda-forge
freetype                  2.10.4               h4cff582_1    conda-forge
future                    0.18.2           py38h50d1736_3    conda-forge
giflib                    5.2.1                hbcb3906_2    conda-forge
google-auth               1.24.0             pyhd3deb0d_0    conda-forge
google-auth-oauthlib      0.4.1                      py_2    conda-forge
grpcio                    1.36.1           py38ha263829_0    conda-forge
idna                      2.10               pyh9f0ad1d_0    conda-forge
imagecodecs               2021.1.28        py38hf9d61db_0    conda-forge
imageio                   2.9.0                      py_0    conda-forge
importlib-metadata        3.7.3            py38h50d1736_0    conda-forge
jpeg                      9d                   hbcb3906_0    conda-forge
jxrlib                    1.1                  h35c211d_2    conda-forge
kiwisolver                1.3.1            py38hd9c93a9_1    conda-forge
lcms2                     2.12                 h577c468_0    conda-forge
lerc                      2.2.1                h046ec9c_0    conda-forge
libaec                    1.0.4                h046ec9c_1    conda-forge
libblas                   3.9.0                     8_mkl    conda-forge
libcblas                  3.9.0                     8_mkl    conda-forge
libcxx                    11.1.0               habf9029_0    conda-forge
libdeflate                1.7                  h35c211d_5    conda-forge
libffi                    3.2.1             hb1e8313_1007    conda-forge
libgfortran               5.0.0           9_3_0_h6c81a4c_20    conda-forge
libgfortran5              9.3.0               h6c81a4c_20    conda-forge
liblapack                 3.9.0                     8_mkl    conda-forge
liblapacke                3.9.0                     8_mkl    conda-forge
libpng                    1.6.37               h7cec526_2    conda-forge
libprotobuf               3.15.6               hcf210ce_0    conda-forge
libtiff                   4.2.0                h355d032_0    conda-forge
libwebp-base              1.2.0                h0d85af4_2    conda-forge
libzopfli                 1.0.3                h046ec9c_0    conda-forge
llvm-openmp               11.1.0               hda6cdc1_0    conda-forge
lz4-c                     1.9.3                h046ec9c_0    conda-forge
markdown                  3.3.4              pyhd8ed1ab_0    conda-forge
matplotlib-base           3.3.4            py38hb24f337_0    conda-forge
mkl                       2020.4             h08c4f10_301    conda-forge
mkl-devel                 2020.4             h694c41f_302    conda-forge
mkl-include               2020.4             h08c4f10_301    conda-forge
multidict                 5.1.0            py38h5406a74_1    conda-forge
ncurses                   6.2                  h2e338ed_4    conda-forge
networkx                  2.5                        py_0    conda-forge
ninja                     1.10.2               h9a9d8cb_0    conda-forge
numpy                     1.19.1           py38h8ccc501_2    conda-forge
oauthlib                  3.0.1                      py_0    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
opencv-contrib-python     4.4.0.42                 pypi_0    pypi
openjpeg                  2.4.0                h6cbf5cd_0    conda-forge
openssl                   1.1.1j               hbcf498f_0    conda-forge
packaging                 20.9               pyh44b312d_0    conda-forge
pillow                    8.1.2            py38h83525de_0    conda-forge
pip                       21.0.1             pyhd8ed1ab_0    conda-forge
protobuf                  3.15.6           py38ha048514_0    conda-forge
pyasn1                    0.4.8                      py_0    conda-forge
pyasn1-modules            0.2.7                      py_0    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pyjwt                     2.0.1              pyhd8ed1ab_0    conda-forge
pyopenssl                 20.0.1             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pysocks                   1.7.1            py38h50d1736_3    conda-forge
python                    3.8.5           h0ed32c4_9_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.8                      1_cp38    conda-forge
pytorch                   1.6.0                   py3.8_0    pytorch
pytorch-lightning         0.9.0                      py_0    conda-forge
pywavelets                1.1.1            py38hc7193ba_3    conda-forge
pyyaml                    5.4.1            py38h5406a74_0    conda-forge
readline                  8.0                  h0678c8f_2    conda-forge
requests                  2.24.0             pyh9f0ad1d_0    conda-forge
requests-oauthlib         1.3.0              pyh9f0ad1d_0    conda-forge
rsa                       4.7.2              pyh44b312d_0    conda-forge
scikit-image              0.17.2           py38he9f00de_4    conda-forge
scipy                     1.5.2            py38hb6cd474_2    conda-forge
setuptools                49.6.0           py38h50d1736_3    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
snappy                    1.1.8                hb1e8313_3    conda-forge
sqlite                    3.34.0               h17101e1_0    conda-forge
tensorboard               2.2.0                      py_0    conda-forge
tensorboard-plugin-wit    1.8.0              pyh44b312d_0    conda-forge
tifffile                  2021.3.17          pyhd8ed1ab_0    conda-forge
tk                        8.6.10               h0419947_1    conda-forge
toolz                     0.11.1                     py_0    conda-forge
torchvision               0.7.0                  py38_cpu    pytorch
tornado                   6.1              py38h5406a74_1    conda-forge
tqdm                      4.48.2             pyh9f0ad1d_0    conda-forge
typing                    3.7.4.3            pyhd8ed1ab_2    conda-forge
typing-extensions         3.7.4.2                       0    conda-forge
typing_extensions         3.7.4.2                    py_0    conda-forge
urllib3                   1.25.11                    py_0    conda-forge
werkzeug                  1.0.1              pyh9f0ad1d_0    conda-forge
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
xz                        5.2.5                haf1e3a3_1    conda-forge
yaml                      0.2.5                haf1e3a3_0    conda-forge
yarl                      1.6.3            py38h5406a74_1    conda-forge
zfp                       0.5.5                h046ec9c_4    conda-forge
zipp                      3.4.1              pyhd8ed1ab_0    conda-forge
zlib                      1.2.11            h7795811_1010    conda-forge
zstd                      1.4.9                h582d3a0_0    conda-forge

[sonovice]

Thanks! The versions in that look correct and very different from your opening post. I will try to reproduce your error on a mac this weekend.

[sonovice]

Unfortunately, I couldn't get my hands on a Mac in the last few days.

Nevertheless, I have an idea what might went wrong. Pip requirements are updated, please give it another go. :)

EDIT:
I would recommend doing this:

conda env create -n smude python=3.8
conda activate smude
git clone https://github.com/sonovice/smude.git
cd smude
pip install .

[ad-si]

Traceback (most recent call last):
  File "/usr/local/Caskroom/miniconda/base/envs/smude/bin/smude", line 8, in <module>
    sys.exit(main())
  File "/usr/local/Caskroom/miniconda/base/envs/smude/lib/python3.8/site-packages/smude/__init__.py", line 198, in main
    result = smude.process(image)
  File "/usr/local/Caskroom/miniconda/base/envs/smude/lib/python3.8/site-packages/smude/__init__.py", line 141, in process
    cols, rows = mrcdi(
  File "/usr/local/Caskroom/miniconda/base/envs/smude/lib/python3.8/site-packages/smude/mrcdi.py", line 664, in mrcdi
    left, right = get_outer_barlines(barlines_img)
  File "/usr/local/Caskroom/miniconda/base/envs/smude/lib/python3.8/site-packages/smude/mrcdi.py", line 56, in get_outer_barlines
    l_idx = np.argmin(peaks[2])
  File "<__array_function__ internals>", line 5, in argmin
  File "/usr/local/Caskroom/miniconda/base/envs/smude/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 1269, in argmin
    return _wrapfunc(a, 'argmin', axis=axis, out=out)
  File "/usr/local/Caskroom/miniconda/base/envs/smude/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 58, in _wrapfunc
    return bound(*args, **kwds)
ValueError: attempt to get argmin of an empty sequence

But mixing pip and Conda like this sounds like something is off already anyways, right? 😅
What about pipenv? Would that make things easier?

Also: Maybe you should simply provide a Dockerfile and Docker image? I guess that would be the easiest way to use it.

[sonovice]

Your error is not a dependency error, but a "bad" input image. The script could not detect some geometry and simply quit with an awful error message. This software is a raw research proof of concept and not polished in any way, I'm afraid... 😒

But mixing pip and Conda like this sounds like something is off already anyways, right? sweat_smile
What about pipenv? Would that make things easier?

Nothing wrong with mixing 'em, happens alot. pipenv would work as well, though you would probably still use pip in your env.

Also: Maybe you should simply provide a Dockerfile and Docker image? I guess that would be the easiest way to use it.

Any contributions are more than welcome! 😉

[ad-si]

Ah I see.
Sorry, by the way, if I sounded too demanding. I'm really grateful for your work on this! I think it's still a quite important problem and unfortunately the open source solutions available so far are mostly still not production ready.
In my quest to find something useful I created https://github.com/ad-si/awesome-sheet-music and https://github.com/ad-si/awesome-scanning, and I started to develop https://github.com/ad-si/perspec.
Actually Perspec had a predecessor written in Python where I also tried a more automatic approach to crop the images, but in the end I realized I don't care if it's automatic or manual and I was more interested in reaching 100% correctness in cropping.

I have also been thinking about integrating a manual dewarping feature, where the user traces a few lines which should be horizontal / vertical in the output image and then I use something like Imagemagick's Shephard's Distortion to apply the fixes.

Any contributions are more than welcome! 😉

I actually tried it, but hit some Docker resource limits during building. Would net some more trial and error to figure out if increases the limits is enough, or if the ML pipeline is simply too much for Docker.

[sonovice]

I have also been thinking about integrating a manual dewarping feature, where the user traces a few lines which should be horizontal / vertical in the output image and then I use something like Imagemagick's Shephard's Distortion to apply the fixes.

Using smude in the background for this should be pretty doable. The important two code parts are these:

1. get_stafflines(...) takes some output of the neural net and returns the topmost and bottommost staff lines as splines. Replacing this with manually traced curves should be a piece of cake.

smude/smude/mrcdi.py

Line 72 in 037051d

def get_stafflines(upper_img: np.ndarray, lower_img: np.ndarray, step_size: int) -> List[UnivariateSpline]:

2. In the same manner get_outer_barlines(...) could be replaced. This method returns the left and right barlines on the page, so basically the score boundaries. Again, it simply returns a tuple of two line functions. Easy to replace with manual data.

smude/smude/mrcdi.py

Line 29 in 037051d

def get_outer_barlines(barline_img: np.ndarray) -> Tuple[Callable[[float], float], Callable[[float], float]]:

I actually tried it, but hit some Docker resource limits during building. Would net some more trial and error to figure out if increases the limits is enough, or if the ML pipeline is simply too much for Docker.

Using ML pipelines in Docker is pretty common and nowadays used for basically all server side ML stuff. I will eventually add it to this repo, but probably not this weekend.

Sorry, by the way, if I sounded too demanding. I'm really grateful for your work on this! I think it's still a quite important problem and unfortunately the open source solutions available so far are mostly still not production ready.

No worries, I can always choose to not meet your request if it goes beyond my time. True, open source code for such topics rarely surpasses their alpha stage. If it works for the scientific publication, it "works" - Mischief managed. 😉