How to resolve issues with light grey colours in PDF or TIFF files at conversion to binary

[BertMatser]

Hi there!

New on GitHub and new to Audiveris. Installed everything yesterday and working fine (version 5.3.1). Also read the manual and tried to convert my PDF score.

It seems grey tones do not convert very well into binary. More specifically, I have grey beams in my PDF thet seem to be invisible when turned into binary:

I have tried a lot of things to get the grey beams into a darker tone or plain black but this turns out to be an enormous hassle (so anything to help me do that for an entire PDF at once is welcome!).

PS: The origin of this PDF is a website that I have a paid subscription for. What I ultimately would like to do is turn this score into MIDI via MusicXML.

Is there any setting in Audiveris that influences the Grey to Binary process in such a way that grey tones are properly recognised and turned into binary?

Or is there anything else I can do to make this work?

Much appreciated!

[hbitteur]

I just tried to process your example.
I can't see any problem with the beams.
On the opposite, the staff lines are drawn in a pale gray color, and the most part get discarded by the BINARY step.

The binarization uses an adaptive model, whereby the pixel threshold applied at a given location depends of the main and standard deviation of pixels values in the neighborhood of the location.

We can change the coefficients for main and std dev. This is low level stuff, use the Tools | Options menu and search for AdaptiveDescriptor. Then change the values as follows (you can try on your own with different values and see what the BINARY step gives you):

These values gave me rather good results:

The GRID step suffers at the start of the staff, but this is something you can easily fix manually (via staff editing)

[BertMatser]

Hi hbitteur,

Thanks a lot for this explanation!

Yes I have mixed up the wording (bar / staff) as I am not completely familiar with the right terminology in English. But you got it right from the example, so that's great! :)

I'll play around with these settings on the entire multipage PDF and try to manually correct the staffs that don't come through properly.

I'll let you know the result within a day or so.

Thanks again!

[hbitteur]

The funny thing is that in 5.4-alpha (the version I'm using) I just dropped the possibility for the end-user to select the binarization policy.
Up until 5.3.1, you still have the choice. Go to Book | Set book parameters and select the Binarization field. Select the GLOBAL mode (rather than the ADAPTIVE mode) and set the global threshold to 225 (I determined this precise value by inspecting the staff lines in your image with an external utility).

The binarization now gives:

And the final transcription is:

Note the OCR gives poor results because the lyrics appear as small bold characters, much less readable than the ones on the gray image.

I'm ready to bet that for years nobody had used the global threshold, since the adaptive mode used to give really good results in all cases. Hence my decision to remove it for the sake of complexity reduction.

My decision was OK until you came up with such pale staff lines.
I'm afraid I will have to re-introduce the binarization option.
I don't thank you !!! :-)

[BertMatser]

Hi hbitteur,

Sorry to be a pain :) I seem to have this effect on other software too...

I tried your suggestion from your first post and that worked very well! The staves are all recognised this way. Great stuff!

A bit further down the first page of my PDF, it seems the OCR has difficulty recognising beams and heads. Could that be solved by using larger prints (like in A3 size) or would there be another setting I could adjust to optimise the recognition of the heads and beams?

This is the same part from the PDF (I hope this image is of usable quality):

Thanks a lot for your help with this, it's really great stuff you're doing here!

[hbitteur]

Beams recognition is generally very good in Audiveris, provided that the typical beam height has been correctly detected in the image.
I can't download your image and thus cannot see what is happening.

I'd suggest to restart the processing until the SCALE step.

1. In the binary tab, manually measure the height of a typical beam. You can do that, using a lasso (keeping the shift key pressed while dragging the mouse). This gives a rectangle that you can inspect in the Pixel board. Like I said, I suspect a wrong beam height, either too low or too high.
2. Use the pulldown menu Sheet | Set scaling data, and there check for the beam height value. If needed, manually modify the beam height value, according to your manual measurement.
3. Then run until the BEAMS step. You should get better results.

[BertMatser]

I took another PDF, which happens to be the Double Bass part of Mercy, the same song as above by Duffy. I printed that PDF in A3, at very large scale. Then put it in. Here is one of the pages from the PDF (most of the Bass goes on like this in this song):

I made some changes to the Book Parameters:

Then ran the transcription of this page. The binary looks very good!

Transcription worked very well, also the beams are recognised correctly this way. Only thing not recognised in one go are the augentation dots behind and above the heads of some of the notes:

This would be very time-consuming to correct so would be great if you have some suggestions how to get these recognised automatically somehow.

Thanks again, this is could become a major time saver for me.

[hbitteur]

Running 5.3.1 on your example (still using the GLOBAL binarization with a 225 threshold), I got the following results (read at the end of the LINKS step):

Obviously, all augmentation and staccato dots have been correctly recognized and linked to their related heads.

Are you sure you did not stop before the LINKS step?

[BertMatser]

Hi Hbitteur,

Tried again but same result.

Seems I am doing something wrong. I think I have not set the GLOBAL binarization value as you mentioned but I can't find it... (I did find and set the meanCoeff and stdDevCoef ratio's to 0.95 and .0.25 respectively which worked very well but it does not recognise the augmentation dots).

I can't find it the GLOBAL binarization in the book parameters as you suggested:

Nor under Options, but I do find 4 parameters with 'binarization' in it:

Should I set the last one to value 225?

Maybe I can't find it becuase you showed a setting that is only available in version 5.4.1?

Hope you can help, there are so many parameters I'm a bit worried to just go about and play with them.

Regards,

Bert

[hbitteur]

Oops! My mistake, sorry.

In 5.3.1 version (the one you are using), binarization papameters are still present but reserved for the "advanced" user.
Here is what you need to do;

1. Go to Tools | Advanced topics, and there tick the field "Specific items".
2. Restart Audiveris (so that the specific items are taken into account)
3. In the Book | Set book parameters, you now can see the binarization parameters at the top.
4. Select the GLOBAL mode and set the threshold value to 225.

[BertMatser]

Hi Hbitteur,

Thanks! I finally had some time to try again.

I found the parameter and set that to GLOBAL and tested using various values.

This is what my default book parameters look like:

Everything below binarization level 225 did not work very well. 225 itself worked but still have the same issue with the augmentation dots as before and it seems to have a hard time recognising most of the rests on a more complicated sheet. See examples below.

Binary at GLOBAL 225:

Recognised result:

Higher binarization settings do not seem to improve things, as the binary results get more 'rough' or 'grainy'.

For instance, binary at GLOBAL 240:

Recognised result:

Maybe there are more parameters I can tweak, as it seems you get different results than I.

Would be great as manually setting all the rests and aug dots is going to be tedious :) (in reality I had to because this project could not wait but there will be many more music sheets of other songs I would like to process so no waste of time at all having this conversation :)

Thanks again!

[hbitteur]

@BertMatser
Could you provide the input file for your last example?
So that I can process it on my own and see what is happening.

[BertMatser]

Mercy - Double Bass.pdf

Here you go!

[hbitteur]

I'm having problems with your double bass input file.
I have not tried all sheets, but at least sheet 2 and sheet 3 do not recognize all staves in sheet.

The scale step there gives two values for interline:

[Mercy.-.Double.Bass#3] Scale{ small_interline(43,44,44) interline(47,47,47) line(2,3,3) beam(23)}

This is the direct consequence of the so-called "combo histograms", where we can see two peaks instead of one expected.

And indeed, in the missing staves, the interline between lines is not constant, while a staff is expected to exhibit lines regularly spaced vertically. Consequently, the staff gets discarded.

I think the root cause is the too high value of interline. Audiveris expects an interline value around 20 pixels; Here we are at 44 and 47 pixels.
Please try to use a smaller zoom factor, half the one you have used.
And make sure the resulting interline is close to 20 pixels.
I bet we won't see these 2 peaks anymore, but just a single peak.

[BertMatser]

Hi hbitteur,

Here is the original sized file (100%). When I use this one, it gets stuck during 'grid' step and goes into 'invalid' status.

Mercy - Double Bass - normal size.pdf

I also tried a file at 150%. Worked better, as it gave a valid result, but ran into issues with the measures:

Mercy - Double Bass - size 150%.pdf

Trace:
[Mercy - Double Bass - size 150%#1] Loaded image 1 2480x3508 from C:\Users\BertMatserSkoposRend\Desktop\Bladmuziek\Mercy - Double Bass - size 150%.pdf
[Mercy - Double Bass - size 150%#1] Beam guessed height: 20 -- 0.50 of 40 interline
[Mercy - Double Bass - size 150%#1] Beam measured height: 22 -- 0.38 of [11..40] range at 197% of needed quorum
[Mercy - Double Bass - size 150%#1] Scale{ interline(39,40,41) line(3,3,4) beam(22)}
[Mercy - Double Bass - size 150%#1] Global slope: 0.00000
[Mercy - Double Bass - size 150%#1] Retrieved line clusters: 4 of sizes [5] with interline(39,40,41)
[Mercy - Double Bass - size 150%#1] Systems: #1[1] #2[2] #3[3] #4[4]
[Mercy - Double Bass - size 150%#1] 1 part along 4 systems
[Mercy - Double Bass - size 150%#1] Created scores: [{Score 1}]
[Mercy - Double Bass - size 150%#1] Discarding alpha band ...
[Mercy - Double Bass - size 150%#1] Converting max RGB to gray
[Mercy - Double Bass - size 150%#1] stem(8 max:9)
[Mercy - Double Bass - size 150%#1] Scale information: HeadSeeds{NOTEHEAD_BLACK[L:-0.6]}
[Mercy - Double Bass - size 150%#1] Stems free length median value: 140 pixels, 3.5 interlines
[Mercy - Double Bass - size 150%#1] Step CUE_BEAMS is skipped because small heads switch is off
[Mercy - Double Bass - size 150%#1] Could not initialize Tesseract lang: deu+eng+fra result: -1
[Mercy - Double Bass - size 150%#1] No OCR'ed lines
[Mercy - Double Bass - size 150%#1] 12 raw measures: [3 in system#1, 3 in system#2, 3 in system#3, 3 in system#4]
[Mercy - Double Bass - size 150%#1] Slurs: 0
[Mercy - Double Bass - size 150%#1] Segments: 0
[Mercy - Double Bass - size 150%#1] Measure{#2P1} Voice{#2 excess:1/8} too long
[Mercy - Double Bass - size 150%#1] Measure{#7P1} Voice{#2 excess:1/8} too long
[Mercy - Double Bass - size 150%#1] Measure{#9P1} Voice{#2 excess:1/8} too long
[Mercy - Double Bass - size 150%#1] Measure{#2P1} Voice{#2 excess:1/8} too long
[Mercy - Double Bass - size 150%#1] Measure{#7P1} Voice{#2 excess:1/8} too long
[Mercy - Double Bass - size 150%#1] Measure{#9P1} Voice{#2 excess:1/8} too long

The used a file at 130%. Same as 100% it went into error during 'grid' step:

Mercy - Double Bass - size 130%.pdf

Then tried 140%. This gave decent results, comparable to the really big sized file I gave you earlier, but same issue with augmentations.

Mercy - Double Bass - size 140%.pdf

Just for the record I tried 160% up to 200% as well. These are the files if you want to test with them. 160%, 170% and 180% did yield results but not as good as the 140% sized file.

190% and 200% worked pretty decent by the way. Similar results as 140%.

Mercy - Double Bass - size 160%.pdf
Mercy - Double Bass - size 170%.pdf
Mercy - Double Bass - size 180%.pdf
Mercy - Double Bass - size 190%.pdf
Mercy - Double Bass - size 200%.pdf

Again thanks a lot for your time and effort!

[hbitteur]

I checked the "normal size 100%" image.
On sheet 1, the SCALE step gives 2 families of interline values clearly separated:

INFO [Mercy.-.Double.Bass.-.normal.size#1] ScaleStep 65 | Scale{ small_interline(24,25,25) interline(27,28,28) line(1,2,2) beam(13)}

And indeed, when visually checking the sheet 1 image we can see them.
Interlines for staff 1: 28, 25, 28, 25
Interlines for staff 2: 25, 28, 25, 25
Interlines for staff 3: 28, 25, 25, 28

Since Audiveris engine expects the staff lines to be regularly spaced within the same staff, none of the staves here get recognized.
Consequenly, the sheet is flagged as invalid, because it contains no music staves.

In other words, the initial image violates the assumption on a staff geometry.

I have no clear solution for that (except getting a better image of course).
A workaround might be to twist the engine so that it "merges" the two interline peaks (25 and 28).
I'm not motivated at all to go cheating this way.

Could you further scale down your image (90%, 80%, 70%, ...) and test the result of the SCALE step, until it gives you a single value.

[hbitteur]

I made a simple test, displaying your 200% image within a PDF viewer and taking a snapshot of the screen into a .png file
Here is this sheet-1.png file

Running the SCALE step gave:

[sheet-1] Scale{ interline(19,20,21) line(1,1,2) beam(10)}

BINGO! single interline of 20 pixels!
From that point on, recognition went like a charm:

[hbitteur]

Note on your input image, that the staccato dots are abnormally close to the staff line and to the note head below.
In the example above, we have been rather lucky, all staccado dots have been correctly recognized and linked to the related note head.
With a margin of 0 or 1 pixel!

Other staccato dots on other staves may be less lucky...