Issues with Android 4.4-5.x/6.x

[BeeeWall]

I tried taking the .ttf from the release and replacing the current font, but the emoji do not show. A few of the emoji show as black and white (I'm not sure if it's all the black and white emoji or if only some work), but the rest just do not show up. The Oreo NotoColorEmoji font works fine, as do the Nougat emoji, so I'm pretty sure it's an issue with this font. (Just to test, I also tried the EmojiCompat font, and got the exact same results).

[C1710]

I know that there are some problems with using this font on a 4.4 device.
But when I tested it, only the ZWJ sequences didn't show up, so this is a bit confusing.
I can't promise anything – since I'm currently busy doing other things, but I'll try to have a look at it at the weekend.

[C1710]

But just for testing purposes:
Could you please tell me if these emojis work or not?

😂
🖖🏽
🤠
👩🏽‍💻
🤯
🥳
🍞🏼

[BeeeWall]

Inside Firefox:

• 😂 is a black and white tears of joy
• 🖖🏽 is a dark skinned hand in a "V" (isn't that Star Trek or something?)
• 🤠 is a yellow blob cowboy
• 👩🏽‍💻 is a black and white woman, a dark skinned square, and a colored computer
• 🤯 is a yellow blob with a blowey-upy mushroom cloud head
• 🥳 is a yellow blob with a party hat and those things you blow and confetti
• 🍞🏼 is black and white bread and a light skinned square

If I copy and paste any outside Firefox, none show up (haven't tried Chrome though).

Here are all the emoji that show up in Gboard (all are black and white), separated by category:

• ✌✍❤❣
• ♨✈⭐❄
• ♠♥♦♣♟
• ☎✉✏✒✂
• ↗➡↘↙↖↕↔↩↪✡✝▶◀♀♂✔✖✳✴❇‼⁉©®™ℹⓂ🈁🈂🈷🈶🈯🉐🈹🈚🈲🉑🈸🈴🈳㊗㊙🈺🈵▪▫⚪⚫

In Firefox, ℹ and Ⓜ have circles around them but don't in Gboard, as do 🉐, 🉑, ㊗, and ㊙. Besides those, all the others from 🈁 to 🈵 have boxes around them in Firefox but not Gboard.

That was probably more than you needed, but I wanted to be sure.

[C1710]

Well, that looks like emojis which were already supported in 4.4 don't get overridden correctly.

So just to be 100% sure: You installed this font by copying NotoColorEmoji.ttf into your /system/fonts-directory and you tried an app which uses the EmojiCompat font (like Blobmoji Preview)?

Did you change any other font besides the emoji font? And btw, what device are you using? I only tested it on (almost) vanilla Android devices.

Oh, and that the hand, the woman in front of the computer and the light bread don't show up correctly seems to be an issue with KitKat not being able to display emoji sequences. This is something I already noticed.

[C1710]

Wait a minute... Which shape does the black and white Tears of joy emoji have?
Is it:

• a grey Blob?
• a circular emoji which has only a black outline?
• some kind of Android head with these two antennas?

While the first one would mean that there are issues with the color rendering of older emoji, the second one would mean that another (non-emoji) font is overriding these emojis and the third one would mean, that you somehow still have the pre-KitKat b/w emojis installed 🤔

[BeeeWall]

I didn't try any special apps, just viewing it in Firefox, Octodroid, and MiXplorer. When I said I tried the EmojiCompat font, I meant I downloaded the NotoColorEmojiCompat.ttf file, copied it to /system/fonts, and renamed it to NotoColorEmoji.ttf to see if it worked better than the normal font. You're correct about the installation menthod. I'm using a Galaxy Tab 4 7.0 using the Acherom ROM (a TouchWiz based ROM), but I don't think it should affect it, because I downloaded and tested both the Nougat and Oreo emoji and they worked.

Edit: Just tried Blobmoji Preview and it didn't work at all. Though that might be a bug with EmojiCompat or Blobmoji Preview rather than Blobmoji.

[C1710]

Okay. That there's no difference between the standard and the EmojiCompat font when using them as your system font is normal.
I'm still confused that the other fonts work as I didn't change anything (besides some assets and some region flags) when it comes to the build tools.

That would mean it's rather unlikely that something went wrong with font priority.
If it is like that, you might check this thread about that issue. But this wouldn't explain why the other fonts work...

[BeeeWall]

The Tears of Joy emoji is a circle, so the second one.

[C1710]

I just got an idea: There's one thing I altered which could cause such an priority issue which is not present for the other fonts:
As far as I know font names are not specified by their file name but by the name specified inside the font file. I'll need to check that, but I would suggest you to try out this modification of that XML file as described in the XDA thread I mentioned above but not with NotoColorEmoji but with Blobmoji, since I called the font like that (as far as I remember).

[BeeeWall]

Nope, that thread didn't fix it. Though maybe that is because I don't have a fonts.xml, so I only changed it in fallback_fonts.xml? There is a system_fonts.xml, but NotoColorEmoji isn't in there, so I doubt it's a priority issue.
Edit: Just saw your new suggestion, will test.

[C1710]

I can only guess, but I think a font which isn't present in the main fonts configuration (which is the case for Blobmoji) will automatically get a priority which is much lower than all the other fonts. I'll try to add a fix which adds a new file to the fonts directory which has the actual name NotoColorEmoji, but currently I don't have very much time 😅

[BeeeWall]

Renaming it in fallback_fonts.xml doesn't seem to have fixed it, so either I messed up (possible, especially since I couldn't completely follow the instructions), or you named it something besides Blobmoji (did you have some other name when you first forked?). I also tried renaming the font to Blobmoji.ttf, just in case the filename needed to match the name in the file.

[C1710]

Originally the font had the exact same name as Noto Color Emoji (I guess that's the name).

But the current name is definitely Blobmoji. I don't actually know how to set the font priority on Samsung devices correctly (I haven't done anything about that on any Android device I own) so I need to google it myself 🤷

[C1710]

I'm sorry for this delay, but this should work...

[BeeeWall]

Sorry, can't test. I gave the device to my brother. I'll try to convince him to let me test it though.

[C1710]

So, did this solution work for you? 😅

[BeeeWall]

I completely forgot about this, sorry. I seem to have lost the device I was using, I'll try and find it and report back. Sorry.

[C1710]

No hurry 😄

As long as you don't have any issues, everything is fine.

[Yowlen]

I can confirm that this does not work on Lollipop either. The kitkatcompat one linked above also does not work.

That said, having paid attention when others ran into this issue back during either Marshmallow or Nougat's release, I can say for certain that this is caused by the changes to the font file that were made to include modifiers. I've never actually worked on them, though, so I don't know what needs to be done to revert back to the old font style without the modifiers, but that should be the solution.

Just as a side note: The smaller file size on the compatibility version you linked suggests you already know this, but that something is still going wrong. (Unless it's just an old build that didn't have all the newest stuff.)

Edit: Forgot to mention that I'll be happy to test anything to help get this working for older devices.

[C1710]

I can confirm that this does not work on Lollipop either. The kitkatcompat one linked above also does not work.

That said, having paid attention when others ran into this issue back during either Marshmallow or Nougat's release, I can say for certain that this is caused by the changes to the font file that were made to include modifiers. I've never actually worked on them, though, so I don't know what needs to be done to revert back to the old font style without the modifiers, but that should be the solution.

Just as a side note: The smaller file size on the compatibility version you linked suggests you already know this, but that something is still going wrong. (Unless it's just an old build that didn't have all the newest stuff.)

Edit: Forgot to mention that I'll be happy to test anything to help get this working for older devices.

Do you have a screenshot of the exact issue(s) you have?

[Yowlen]

I can grab a screenshot later, but what Android 5.x and below end up showing is the generic version of the emoji (if one exists), followed by a square block of any skin color modifier that may have been used, and after that, a gender marker if the gender modifier was also used. Basically, up to three separate characters for a given emoji.

But that's only for emoji that are displayed via other means. Google's keyboard won't display any of them because it can't get the proper modifier support from the system in order to categorize them. The emoji keyboard generally fails to load any of them at that point, though depending on the version, it might display the black & white emoji and a few other unicode symbols from other built-in fonts.

Anyway, I'll work on grabbing a screenshot and edit it in here once I get it.

[C1710]

I can grab a screenshot later, but what Android 5.x and below end up showing is the generic version of the emoji (if one exists), followed by a square block of any skin color modifier that may have been used, and after that, a gender marker if the gender modifier was also used. Basically, up to three separate characters for a given emoji.

That's normal. Support for (ZWJ) sequences was introduced in Android 6.0.1 (at least that was the first time they were used).

But that's only for emoji that are displayed via other means. Google's keyboard won't display any of them because it can't get the proper modifier support from the system in order to categorize them. The emoji keyboard generally fails to load any of them at that point, though depending on the version, it might display the black & white emoji and a few other unicode symbols from other built-in fonts.

That's something I'm rather confused about. So you get something like 🦸♀ (with the second one not being shown as an emoji)? That's probably also something I guess I can't do anything about.
Have you tried using the official (current) Noto emoji font? Just to make sure it's not one of those minor non-artistic changes I made to the font.

[C1710]

And about Gboard not displaying them at all: That might be an implementation issue on their side.
I don't think removing sequences would improve anything.

[Yowlen]

Sorry, I should've been more clear. The extra symbols is actually normal for that kind of stuff when it's displayed. And yes, it shows up like you said there, at least when the font actually works.

As for the actual erroneous behavior, what it was doing was showing the missing symbol thing, followed by those modifier symbols instead of the generic emoji. Google's keyboard also wouldn't recognize that stuff.

But... for some reason, when we went to go grab a screenshot, it actually worked in Google's keyboard this time. It's probably because we installed it via TWRP this time instead of using root in the main system. Some of them are still using the other font, which suggests to me that a cache issue is still persistent somewhere that I'll have to track down.

But while Google's keyboard displays them okay now, and oddly enough, Firefox does too, any system stuff -- including the Play Store -- does not. In the screenshot below, you can see the old emoji font mixed with some from blobmoji in Google's keyboard, but the Play Store's search box just shows the missing character symbols -- two of them, as the most recent one was the redhaired woman one, made up of redhaired person + female sex symbol.

Edit: Hit the send button too soon.

So it might just be a cache issue for everything, but it'll take some more time to sort that out. At the very least, it was most definitely the way I explained in the past, but it could be that a recent update to Google Play Services implemented a catch for the outdated system functions and is using its own updated version in its place.

[C1710]

Oh, yeah, one thing about Gboard: They always use Noto emoji nowadays.
I recently noticed that when I was using my Nexus 9 after a while (Android 7.1). Downgraded Gboard to some older version, and it was fine again 😅.
So it was probably an app update that caused the different behavior for the keyboard.

[Yowlen]

Now that isn't true. Maybe it was with a given version that was later revoked, but here's a screenshot of the latest version of Gboard running on Pie/9.0. Blobmoji all the way. ^_^

Edit: Forgot to mention that it functioned the same way on our 7.0 ROM too.

[C1710]

Well, that makes everything even stranger...
But without Gboard (or anything app-specific), there's no way for the Android 8+ emojis to show up on older Android versions.

[C1710]

• Gboard shows Noto emoji (i.e. the Android 8+ design), which is not something I can influence.
• Firefox and other apps which probably use some form of custom rendering properly show the Blobs as they are supposed to look, including sequences
• Any app that lets the system itself handle emoji rendering displays single character emojis correctly (at least mostly; some symbols even have issues on Android 9 with Noto emoji), but messes up sequences that they are not only split up into different emojis (as you would expect), but into boxes (instead of the emojis that these sequences are composed of).

Is that the correct summary of your issue?

[Yowlen]

The thing is, if you look at the screenshot from our Lollipop device, it actually shows the new emoji just fine, both in the keyboard and elsewhere -- even in the base system dialogs. The problem is that it didn't show modified versions in many cases, which is why the special versions of emoji fonts existed for pre-Marshmallow devices.

It's strange that it'd show now, but the ROM I'm using could probably stand to be refreshed anyway, so I'll do that when I get the chance and get back to you. It should at the very least fix Gboard.

Also, for clarity, this was using the regular Blobmoji.ttf font from the latest release, not the test font listed up above.

Edit: Wrote that before seeing the last reply. Not quite. In addition to the stuff I wrote above, Firefox actually showed the modifiers separately with the old Oreo beta font. It isn't showing it with blobmoji, which tells me that Firefox does use the system emoji renderer, same as others. This is why I suspected an update to Google Play Services overriding the outdated system function and that the individual apps just hadn't cleared their font caches for whatever reason.

[C1710]

Wait, Firefox shows the Oreo emojis?
That sounds a bit like either your ROM provides a different font than the one that was used in Lollipop OR that Firefox started using the EmojiCompat library (or something similar).
However, my Android 9 phone doesn't display the latest emojis at all on Firefox, which makes option no. 2 unlikely (and the bug entry for that is still marked open as well).

I also just created a new branch (nosequences) that does not include any sequences in the png/128 folder. But since I recently reinstalled my laptop, I haven't yet set up the build environment and I won't do that until tomorrow evening (I'm already a bit tired and setting this stuff up is often very annoying 😅), so the font hasn't changed yet...

[Yowlen]

Sorry. I should've made it clear that I was using a different custom emoji font already. Obviously the standard font included with the ROM doesn't have those emojis. Sorry for any confusion that omission caused. 😅

Edit: Also, I'm using the Magisk module on my 9.0 device, so I'm not installing it to the system image itself. I should also clarify that I'm in no way referring to the default fonts for any ROM in any of my posts. Only the default functions and updated/blobmoji fonts.

[C1710]

Here's an experimental build without any emoji sequences:
Blobmoji_no_seqs.zip

[Yowlen]

Tested on my Lollipop device and nope, not working. I know that some sequences work (and are therefore required) but not all.

I know racial sequences are supported, but things start to get sketchy with gendered and hair color sequences. Looking at Emojipedia, Emoji v4.0, 2016 is when gendered sequences were introduced, and given that Lollipop is from 2015, this would make sense.

That's not to say that the emojis themselves have to be removed from the font, though. I recall seeing many of the 4.0 emojis like the blonde hair woman show up properly. I'm just not sure how those are done.

I wish I could be of more help. I just never studied the inner workings of how the sequences are implemented, so I don't know what else to do.

Edit: Actually, I think I remember one more piece of information that may be of help. "Man" was considered the "default" back then, so it was only the "woman" gendered sequence that was implemented. Ergo, the "man" gendered sequence would have to be removed for sure, but the "woman" one should be able to be kept.

[C1710]

As Lollipop is now about 6 years old and Emoji 4 is almost 5 years old, is this really a relevant issue anymore?