Skip to content

Creating Your Own Instrument

Jump to bottom
StavWasPlayZ edited this page Jun 11, 2023 · 28 revisions

Let's say you're a fellow modder who'd like to use this mod as a dependency for making a new instrument. While I heavily doubt anyone would actually do it, I still left a pretty neat API for you to use. In any case, this section is mostly for me to remember my stuff :P

Anywho, this tutorial will assume that you are a complete beginner to the subject of modding and such in general, as it makes it easier for me to write it that way.
Of course, you will still need basic Java, Forge (and Minecraft) knowledge.

Notices

  • Instrument mod extensions are untested!

  • Modifying the number of rows/columns in an instrument is highly experimental. You may, however, do so by overriding the columns/rows method respectively.
    You will then be required to re-map your instruments. For that, you may override the noteMap method.
    Grid instruments will have an easier time, since they just need to pass in to noteGrid.genKeyboardMap their key mappings of choice in the form of a 2D array; and simply return its value.

    • A note map MUST have all note buttons in it!

If you encounter any issues with the above points and is sure that it is nothing but my stupidity at rise, make sure to report it to this project's issue tracker.

Resources

All resources for any instrument's screen must be kept under assets\<modid>\textures\gui\instrument\<instrument>. If this file structure is not followed, the API will not be able to detect your resources.

Our Goal

We're going to start simple, and assume that all you have in mind is a simple grid instrument that inherits the 21 notes style - per se, a piano.

*Quick sidenote, I'd actually love to implement a piano into this mod but I just don't have the sounds for it. If you actually make it that'd be very epic and I'll be in a life's debt to you.

NOTE: If you wish to implement an instrument that does not inherit this style, I suggest seeing the implementation of AratakisGreatAndGloriousDrumScreen.
More than that - just go see any of the present instruments' implementations to gain better understanding of the API. You're more than welcome to do so.

Creating an Instrument Screen

Firstly, we'll need to create an instrument screen. As the name may suggest, it is the container for our instrument's GUI. Such a screen must extend the AbstractInstrumentScreen class.

NOTE: This is a client only class; screens do not exist on dedicated servers.

This mod has a convenient implementation for grid-specific instruments called AbstractGridInstrument - so let us use this in our example.
We will therefore annotate it as such:

@OnlyIn(Dist.CLIENT)
public class PianoScreen extends AbstractGridInstrumentScreen {
    public static final String INSTRUMENT_ID = "piano";
    //...
}

The INSTRUMENT_ID field is not required, however we will have to reference the ID more than once, so it is recommended to store it there. The ID must match that of your item's ID.

This class will kindly ask you to implement some methods. Let's explore their purposes and how you should handle them:

Constructor Method

All that the AbstractInstrumentScreen constructor requires is an InterationHand instance. This hand represents the hand of which the player currently plays the instrument with.

getInstrumentId()

Pretty self-explanatory. Just return the INSTRUMENT_ID we made, paired with your mod ID:

@Override
public ResourceLocation getInstrumentId() {
    return new ResourceLocation(Main.MODID, INSTRUMENT_ID);
}

Yet again, this ID (location) must match that of your item's.

getThemeLoader()

This method should return an InstrumentThemeLoader.

This class is responsible for handling the color palettes of the instrument, and will automatically assign them according to the JSON styler associated in the instrument's resources directory, under the name of instrument_style.json. We will explore its contents later.
For now, just know that our lad AbstractInstrumentScreen has got our back, and stores in it a neat method to create one for you. Just call initThemeLoader, passing both your mod and instrument's ID.

Triggering the Theme Loader

The initiator for this class must be called in about the same time as of the loading of the Mod event bus. This is so because when you call the initiator, it automatically subscribes the given JSON for any resource load event on the client, and the first one is just about when the game fires.
This means that if you do not call it at that time, your mod's and any resource pack that is modifying your piano will not have its effects applied until a resource load event occurs (such F3+T being pressed).

The trick I use for it is to make the theme loader a static property of the screen class, and annotate the screen class as an EventBusSubscriber for the mod event bus. This makes the FML load the class early on in search for any methods trying to subscribe to the specified event bus.
We must also declare the value property to be of CLIENT for it to not attempt to load on the server.

Thus, we end up with the following snippet:

@OnlyIn(Dist.CLIENT)
//NOTE: There just to make it load on mod setup
@EventBusSubscriber(bus = Bus.MOD, value = Dist.CLIENT)
public class PianoScreen extends AbstractGridInstrumentScreen {
    //...

    private static final InstrumentThemeLoader THEME_LOADER = initThemeLoader(PianoMod.MODID, INSTRUMENT_ID);
    @Override
    public InstrumentThemeLoader getThemeLoader() {
        return THEME_LOADER;
    }

    //...

getSounds()

This method should return an array of NoteSounds. Their length must be equal to the total notes that can be pressed in such instrument - by default 21.
Each one will be taken in the order of the note pressed by the player, as specified in the associated NoteGrid.

For now, at least, we don't have such array, so let's help ourselves by simply returning null.

To Conclude

With all said and done, we end up with this class at hand:

@OnlyIn(Dist.CLIENT)
//NOTE: There just to make it load on mod setup
@EventBusSubscriber(bus = Bus.MOD, value = Dist.CLIENT)
public class PianoScreen extends AbstractGridInstrumentScreen {
    public static final String INSTRUMENT_ID = "piano";

    public PianoScreen(InteractionHand hand) {
        super(hand);
    }
    @Override
    public ResourceLocation getInstrumentId() {
        return new ResourceLocation(Main.MODID, INSTRUMENT_ID);
    }


    @Override
    public NoteSound[] getSounds() {
        return null;
    }


    private static final InstrumentThemeLoader THEME_LOADER = initThemeLoader(Main.MODID, INSTRUMENT_ID);
    @Override
    public InstrumentThemeLoader getThemeLoader() {
        return THEME_LOADER;
    }
    
}

Adding Sounds

NOTE: Keep in mind that, according to this very clever Reddit user, quote:

The Genshin lyre is in C major, so transpose everything into 1-4-5 chords and play in C major

NoteSound

Mono and Stereo

Because of some annoying specification by OpenAL, the sound library used by Minecraft - all sounds produced on Stereo will be stuck on 100%.
I have thus made a system to balance quality and your bloody ears, by implementing the NoteSound class. There, you may define one mono sound and an optional surround version. Which one is chosen depends on multiple factors, but ultimately boils down to the user's preference, as set by their instrument's options.

If the player has chosen either Mono or Stereo sounds only, all instrument sounds are forced into the respected version - no matter the factors.

However, when Mixed (the default options) is chosen:

  • If the instruments volume is lesser than 100%, mono sounds will always play. This does not account for the master volume.
  • When requested by the client, stereo sounds will always by chosen.
  • When requested by the server, only if the distance between the player and the client is lesser than the set stereo range (5 blocks), stereo will be chosen.

Note that if a surround is not present, the quite obviously - mono will always be chosen.

Embedding Sounds

Location & Convention

All instrument sounds should be kept under: assets/<modid>/sounds/<instrument>/<note_number>.ogg. If you have their stereo version too, name them <note_number>.stereo.ogg by convention.

NOTE: The sounds, in a grid, must be ordered from the first visual row to the last, as described here: image

In our case, we would need 21 piano key sounds. Let us assume that we also happen to have a stereo version of the piano as well - we will thus include all 42 audio files in our instruments folder, as described earlier.

NOTE: If you only have the stereo versions of your sounds, you may use FFMPEG to channel them down to a singular mono channel. This command was generated by ChatGPT, and it worked quite well for me. Make sure to run it in your instrument sounds directory as the active directory.

PowerShell:

Get-ChildItem -Filter "*.ogg" | ForEach-Object { ffmpeg -i $_.FullName -ac 1 -c:v copy ($_.BaseName + "_mono" + $_.Extension) }

Unix Bash (Untested!):

for file in *.ogg; do ffmpeg -i "$file" -ac 1 -c:v copy "${file%.*}_mono.${file##*.}"; done

In summary, the command takes all the ".ogg" files in the current directory and converts each file's audio to mono using ffmpeg. The converted files will have "_mono" appended to their filenames before the original file extension.
It is still recommended to rename your mono output files to *.ogg and the stereo ones to *.stereo.ogg in order to follow the mod's conventions, though.

sounds.json

Forge requires you make a JSON file describing all your sounds that the mod adds, as described here.
The keys of these sounds will be what Minecraft and the GI-API will use to reference your sounds. The naming of the sounds must follow this naming principal: <instrument>_note_<note number>, and for stereo types: <instrument>_note_<note number>_stereo.
All instrument sounds should channel to the records category.

So, your file should be constructed as something like this:

{
    "piano_note_0": {
        "catrgory": "record",
        "sounds": ["pianomod:piano/0"]
    },
    "piano_note_1": {
        "category": "record",
        "sounds": ["pianomod:piano/1"]
    },
    //...
    "piano_note_20": {
        "category": "record",
        "sounds": ["pianomod:piano/1"]
    },

    // If you have stereo versions:
    "piano_note_0_stereo": {
        "catrgory": "record",
        "sounds": ["pianomod:piano/0.stereo"]
    },
    "piano_note_1_stereo": {
        "category": "record",
        "sounds": ["pianomod:piano/1.stereo"]
    },
    //...
    "piano_note_20_stereo": {
        "category": "record",
        "sounds": ["pianomod:piano/1.stereo"]
    }
}

Do notice how the sounds array refers to the location inside of our sounds directory: piano's the parent directory and the files range from 0 to 20.

Registering the sounds to the API

Registering the sounds is made extremely easy on your end. And I really mean it - all you need is but a single method to handle it all.
You will need the general registration class, of type SoundEvent, and its contents will be as followed:

public class ModSounds {
    
    public static final DeferredRegister<SoundEvent> SOUNDS = DeferredRegister.create(ForgeRegistries.SOUND_EVENTS, PianoMod.MODID);
    public static void register(final IEventBus bus) {
        SOUNDS.register(bus);
    }

    public static final NoteSound[] PIANO_NOTE_SOUNDS = NoteSoundRegistrer.createInstrumentNotes(
        SOUNDS, new ResourceLocation(PianoMod.MODID, "piano"), true
    )

}

createInstrumentNotes will simply register to the specified registrer all the sounds necessary for the instrument to function - those that you put in the sound directory previously.
The true argument at the end defines whether this instrument has stereo support. You can either replace it with false or remove this argument to specify otherwise.
Additionally, if your instrument is not a 3x7 grid, you may specify the rows and columns for your instruments after the stereo boolean, such as:

NoteSoundRegistrer.createInstrumentNotes(SOUNDS, new ResourceLocation(PianoMod.MODID, "piano"), true, 5, 6)

Where the grid here would be 5x6.

NOTE: You may also define singular note sounds and not a grid of them. An example can be found on the glorious drum's implementation:

GLORIOUS_DRUM = new NoteSound[] {
    NoteSoundRegistrer.registerNote(SOUNDS, loc("glorious_drum_don")),
    NoteSoundRegistrer.registerNote(SOUNDS, loc("glorious_drum_ka"))
}

NOTE: loc() is just a shorthand for the resource location within my mod ID.

You may append a boolean argument for this method, too, to define stereo support for it.

Wrapping it up

Let's get back to our PianoScreen. now, All we now need to do is instead of returning null in our getSounds() method - we will return the array of sounds we've just made:

@Override
public NoteSound[] getSounds() {
    return ModSounds.PIANO_NOTE_SOUNDS;
}

Texturing

Instrument Styler

instrument_style.json is a JSON styler file that should be present under your instrument's resources directory. It (currently only) defines the color scheme of our instrument. 3 of its properties are customizable:

{
    "note_theme": [30, 30, 30],
    "label_theme": [20, 20, 20],
    "note_pressed_theme": [255, 249, 239]
}
  • note_theme defines the RGB color of the note text when the note is unpressed, as well as the ring produced when pressing the notes. It should be about the same color of the note button's pressed state. I'm thinking of black-ish piano keys, so I'll put all values on 30.
  • label_theme defines the RGB color of the note labels when the note is unpressed. It should be the same color of the note button's pressed state, so I'll put the values on 20.
  • note_theme_pressed defines the RGB color of the note text when the note is pressed. It should be the same as the main color of the note button's unpressed note state. By default, this value is set to 255, 249, 239, so if you did not change the aforementioned note color - you can simply omit this property.

NOTE: If you want to add any other design elements to the styler, you may subscribe to a theme load event via InstrumentThemeLoader#addListener in a subclass, replacing it in your instrument's screen:

public class PianoThemeLoader extends InstrumentThemeLoader {

   private Color nailsColor;
   private int parentDissapointmentAmount;
   //...

   public PianoThemeLoader(ResourceLocation instrumentStyleLocation) {
       super(instrumentStyleLocation);
       addListener(this::loadPianoStuff);
   }

   private void loadPianoStuff(JsonObject themeLoader) {
       setNailsColor(getTheme(themeLoader.get("nails_color"), Color.BLACK));

       final JsonElement dissapointment = themeLoader.get("dissapointment");
       if (dissapointment.isJsonPrimitive())
           setDissapointmentAmount(tryGetProperty(dissapointment.getAsJsonPrimitive(), JsonPermitive::getAsInt, 1));

       //...
   }

   // Getters & setters etc.

}

It is recommended that you make use of InstrumentOptionsScreen for most options, though - simply override initInstrumentOptionsScreen.
Only use the styler for properties that make sense for a resource pack to modify.

Note Background

The note background refers to the note texture by itself. You must place the texture in your instrument's resources folder under the note directory, named "note_bg.png". An example of such texture may be found here:
Windsong Lyre's note background sprites

It is divided into 3 parts:

  1. The unpressed note state
  2. The pressed note state
  3. Hovered/focused/other player state

Note Labels

By default, the note grid provides you with a label sprite of 7 items. It is located under genshinstrument:textures/gui/instrument/grid_notes.png. You may change this to your liking by overwriting the getNotesLocation() method in your instrument's screen. The colors taken for the textures are provided in the JSON styler, as described earlier.

Texture Model

The Genshin Instruments mod provides you with an instrument model to inherit: genshinstrument:item/instrument. This is a base model that inherits the regular minecraft:item/generated base model, however it also applies the transformations that make the instrument appear correctly in the player's hands.
Here is an example for a piano.json item model:

{
    "parent": "genshinstrument:item/instrument",
    "textures": {
        "layer0": "pianomod:item/piano"
    }
}

Sending an Open Packet

To open our instrument, through any means, we need to send a packet from the server to the client. This is done so that the server could register the player opening it, as well as to provide a way of the client classes to not leak to the server.

I have made a packet manager for my mod to use, you may see it here. It will essentially take any instrument's ID as an argument, and call Minecraft's setScreen() method with a new instance of your screen via a map. Feel free to use this implementation or another that suits your needs. Do note that you'll need to consider the hand of the player in your packet as well.

Making an instrument item

An instrument item is an extension of the InstrumentItem class. There is no need to make a subclass for it - you may simply initiate it directly in your registrater class, like so:

public static final RegistryObject<Item> PIANO = ITEMS.register("piano", () ->
    new InstrumentItem(
        (player, hand) -> ModPacketHandler.sendToClient(new OpenInstrumentPacket("piano", hand), player)
    )
)

NOTE: You can also add item properties in a 2nd argument, however the item stack will always be set to 1.

The method you must provide to the constructor is responsible for sending an open instrument packet to the given player. DO NOT USE ANY OF THE PROVIDED PACKETS / PACKET HANDLER. Make sure to implement your own.

And that should be it! All you now need to do is texture your item nice and neat, and your instrument is now fully implemented into the game. Enjoy!

Clone this wiki locally