This project is at heart an osu-queue-processor that handles various book-keeping tasks in response to scores arriving from users. This includes:
- Awarding medals
- Updating the pp values of individual scores
- Updating user statistics (total scores / pp / rank counts / play time / play count / max combo)
It also offers commands to perform common maintenance on all the above, including tasks like:
- Re-running pp calculations on scores (and users) after a change in algorithm
- Re-running processing of individual scores after a change in processing (ie. a new medal or new tracked statistic).
- Migrating scores from legacy tables to the new format
Some things are not yet fully fleshed out:
- Not all medals are handled
- Ranked score processor is not added
To start the queue processing functionality:
dotnet run -- queue watchTo reprocess pp of all scores:
dotnet run -- performance allTo pump all scores in scores back through the queue for reprocessing:
dotnet run -- queue pump-allNote that you will want a queue processor running to handle all the scores that are going to appear in the queue.
Processing is versioned (see ScoreProcessed.ProcessedVersion), so re-queueing scores which have already been processed previously is a safe operation – they will either be skipped or upgraded (revert-apply).
Each processor has an apply and revert command, so in theory, it should be possible to upgrade scores to a newer version of processing. This was made with the assumption we would gradually be adding new pieces of the puzzle in until we had everything online. If everything is in a good state, this may be less useful.
This is part of a group of projects which are used in live deployments where the deployed version is critical to producing correct results. The master branch tracks ongoing developments. If looking to use the correct version for matching live values, please consult this wiki page for the latest information.
Contributions can be made via pull requests to this repository. We hope to credit and reward larger contributions via a bounty system. If you're unsure of what you can help with, check out the list of open issues.
Note that while we already have certain standards in place, nothing is set in stone. If you have an issue with the way code is structured; with any libraries we are using; with any processes involved with contributing, please bring it up. I welcome all feedback so we can make contributing to this project as pain-free as possible.
URI template to download .osu beatmap files. Used for realtime processing.
Defaults to https://osu.ppy.sh/osu/{0}.
Host for MySQL.
Defaults to localhost.
Database name.
Defaults to osu.
Database username.
Defaults to root.
Database password.
Database users table name; if using the dumps, set this environment variable to "sample_users".
Defaults to phpbb_users.
Host to submit DataDog/StatsD metrics to.
Defaults to localhost.
Enables DataDog origin detection when running in a container. See DataDog documentation.
Root domain to which LegacyIO requests should be directed.
- In Debug configuration, this parameter is not required; when missing, all LIO requests are no-ops.
- In Release configuration, this parameter is required; all LIO requests will hard-fail with an exception if missing.
Secret key used to sign LegacyIO requests to osu-web. Required to award medals.
- In Debug configuration, this parameter is not required; when missing, all LIO requests are no-ops.
- In Release configuration, this parameter is required; all LIO requests will hard-fail with an exception if missing.
Whether to process user medals. Set to 0 to disable processing.
Default is unset (processing enabled).
Whether to process user total stats. Set to 0 to disable processing.
Default is unset (processing enabled).
Whether to use realtime processing (download beatmaps and compute their difficulty attributes on every processed score), or to rely on database data. Set to 0 to disable processing.
Default is unset (processing enabled).
Redis connection string; see here for configuration options.
Defaults to localhost
Schema version for the queue; see Schema.
Whether to write a legacy score's PP to osu_scores_high if applicable to the score. Set to 0 to disable writing.
Default is unset (writing enabled).
To setup a testing environment, the minimum requirements are having a MySQL instance available at localhost:3306 and a redis instance available at localhost:6379.
Make sure all data is expendable. It may will be nuked without remorse.
You can setup an environment and have it map to the correct localhost ports:
docker-compose up -dYou can stop the environment using:
docker-compose downAlternatively, you can stop and remove all volumes (ie. nuke your test database) using:
docker-compose down -vThe osu! client code, framework, and server-side components are licensed under the MIT licence. Please see the licence file for more information. tl;dr you can do whatever you want as long as you include the original copyright and license notice in any copy of the software/source.
Please note that this does not cover the usage of the "osu!" or "ppy" branding in any software, resources, advertising or promotion, as this is protected by trademark law.