Error/Wrong display of foreign languages and symbols

[CDScissor]

Says it all, pretty much. In my case, Japanese characters are messed up. Can't provide images, for that I'll apologize.

[joel16]

Version 3.20 was my final release for this project. I'm not sure where or why it's getting so much attention now after the project is complete than when it was still under development. Considering this may is rather necessary I'll try to squeeze in some time for a bug fix at some point. I believe this is an issue with the current font (Roboto) which may not support Japanese characters. I could be wrong.

[CDScissor]

To answer your first question, I don't know, I just stumbled across your project when I got the need to have a file manager and it was already at version 3.20 by then. It's surprising how rediculously quick your response was considering that this is a complete project. Your theory seems to be correct from my own experimenting on Roboto, information on Wikipedia proves it as well.

[CDScissor]

Even so, that doesn't explain why some symbols, particularly this one ☸ isn't displayed right, unless Roboto doesn't support them either. Also, funny thing, I just discovered a new issue as I was exploring this one, that is that files that have long names aren't recognized as what file they are but rather as a "null" file, so says the properties anyway. I think this occurs in part of the main issue, for me atleast, as I've noticed because Roboto doesn't know how to handle Japanese characters, it puts who knows how many spaces on the file name making it pass some threshold. If you should change the font, I suggest Google's open-source Noto font, as mentioned on Wikipedia. It looks dead-close to Roboto.

[joel16]

Yes I saw that as well. I'll try to make some time to switch to the Noto font and see how it goes. Perhaps this weekend I'll get to it.

[CDScissor]

Hope to see that, closing this soon would be great. See ya then, bye.

[CDScissor]

Any news? Or is it a bad time where you are? Also, does this mean your other file managers (on 3DS and Vita) have the same issue? If so, I assume they're also going to be fixed.

I'll be back in about 8, it's sleepy night night.

[joel16]

I currently do not have the time. Also I can't find a working TTF to PGF converter, if someone can provide the Noto font converted to PGF I can apply the changes and have travis build it. (I no longer have the PSP development environment on my PC either)

[CDScissor]

I see, I'll try what I can do since it is supposed to be my mess. I don't think I'll have any better luck than you but I can atleast try.

[joel16]

All I need is the font in pgf format. I can't find a copy that works on Ubuntu, and I don't remember how I got Roboto converted sadly.

[CDScissor]

After checking everything I could, as expected, I got no luck as well. Perhaps if possible, we could get someone else to get involved. Maybe there's someone in the PSP hombrew Discord that could help but I'd like to bother as little amount of people as I can.

Edit: I'd like to but if I have to, I will bother someone. Not annoy them, just you know, tell them.

[joel16]

Finally got around to it. I made the changes on this branch: 7791983

Currently I don't have the PSP development envoirnemnt installed so I only tested it in PPSSPP after travis built it. Please test if this works on your PSP: https://github.com/joel16/CMFileManager-PSP/raw/gh-pages/CMFileManager-PSP.zip

[CDScissor]

I regretfully report that the issue still persists. That atleast eliminates the font as the one at fault. Tested on a PSP-2000.
My next assumption would be that there's something wrong with the character encoding. Two standards I know of are ASCII and Unicode, the former being well-known with only supporting the English alphabet as well as commonly-used symbols like numbers and punctuation marks.

[CDScissor]

Honestly, this just feels like some annoying piece of food stuck between your teeth. Anyway, take a look at what's supposed to be displayed.

[joel16]

Sorry for the hold up, I think I might be able to fix this. Will test my theory in the coming days and let you know.

[CDScissor]

Sounds good. Can't wait to see if your theory is correct. Good Luck.

[joel16]

Can you test this? https://github.com/joel16/CMFileManager-PSP/raw/gh-pages/CMFileManager-PSP.zip
There was a "set alternate font" functionality in the case some symbols don't show in the main font. I'm not sure if this works. I tried it on my PSP and got a one or two Japanese characters to show up.

[CDScissor]

No luck, but I think I've found a promising fix. Further study into the Noto fonts reveal that there's a completely different branch of Noto named Noto CJK that's specifically designed to support Chinese, Japanese and Korean characters (hence CJK, if only I could abbreviate as smoothly as that).

The normal Noto font covers just about everything except those covered by Noto CJK. Only reason I could think for them to seperate CJK is because not all use cases will need support for CJK characters or that the CJK fonts are big. It also shouldn't matter whether you get "Noto CJK JP" or "Noto CJK KR" or what-have-you since Google has clearly documented that all varients cover the same amount of glyphs as they call it and the varient simply states what the default language is.

If the character codes are the same as what the PSP understands, and very likely they are, then this should fix the problem for good.

To recap:

There's a varient of Noto supporting CJK characters. Either set it to replace the current Noto font or set it as the alternate font, which I would prefer because the writing with the alphabet using the CJK varient is too spacey(for me at least)

[joel16]

Thanks for the research. I'll give it a try when I get some time.

[CDScissor]

Yeah, this better be the final nail in the coffin for this pesky problem.